N2WS Support Engineer Elizabeth Lewis gave an amazing session at CloudOps Summit August 2020 on Data Lifecycle Management and the 5 key pieces you may be missing if you have your workloads and data on AWS.
Data is now more valuable than oil, many say. Our Data Lifecycle Management session clears up the confusion and vague understanding about a concept that has entered almost every company's lexicon in 2020 as enterprises are increasingly faced with the challenges of working remotely, compliance concerns, the exponential scaling of data and the storage costs associated.
Developing a comprehensive plan and strategy for data lifecycle management is an extremely overwhelming challenge. We will aim to provide best practices and main issues to think about as you begin to familiarize yourself and develop a comprehensive plan:
Learn:
• What Data Lifecycle Management means in the AWS cloud and why it will be crucial in the coming year in terms of data protection
• How to use Data Lifecycle Management to enjoy higher performance, lower storage costs and high availability
• How to avoid the five most common mistakes so your Data Lifecycle Management (or lack thereof) does not lead to catastrophic data loss
5 Key Pieces you are missing when dealing with Data Lifecycle Management in AWS
1.
2. What is the Data Cycle?
Refers to all stages of data from creation to disposal.
• Backup
• Retrieval
• Archive
• Dispose
#1 AWS Backup
3. Do you need to care about Data
Lifecycle Management? …
In 2018, the world’s datasphere was 33 zettabytes.
By 2025, it is predicted to grow to 175 zettabytes.*
Data types are increasingly varied and require different:
*Source: The Digitization of the World (IDC)
Storage requirements Retention periods / compliance
Recovery options Frequency of backup
RTO/RPO SLAs Levels of criticality
?
?
?
?
?
?
#1 AWS Backup
5. What are the consequences of poor or no
Data Lifecycle Management?
• Inability to locate data
• Human errors or bugs producing missing or wrong data
• Data finding its way into the wrong hands
• Downtime disrupting mission critical tasks and services
• Financial repercussions
• Data leaks
• Noncompliance penalties and regulation violations
Without proper DLM, you run the risk of:
!
!
!
!
!
!
!
#1 AWS Backup
6. 7 Ways to Automate your Data Lifecycle in AWS:
• Backup Enforce automated snapshot backup
• Role-Based Security Define permissions and manage access
• Archiving/Retention Automate retention time based on nature of data
• Deletion/Cost control Automate deletion of backups and/or utilize low
cost storage options like S3, Glacier
• Auditing Deliver regular reports, manage protected/unprotected data
• Retrieval Perform DR drills, confirm retrieval is fast and granular
• Data Storage Choose “Block” vs. “Object” vs. “File”1
2
3
4
5
6
7
7. • Founded in 2012 with a mission to simplify
Backup and Recovery for AWS
• Top rated on AWS Marketplace and AWS
Premier Partner
• Purpose-built for AWS & distributed via AWS
Marketplace
• Thousands of global clients, backing up
hundreds of thousands of EC2 instances
•Winner of 18+ industry awards
N2WS: A pioneer in Data Lifecycle
Management on AWS
8. Operational backup & disaster
recovery (DR) built for AWS.
Flexible policies —scheduling from
minutes to months.
Distributed as an AMI through
AWS
Marketplace.
Near-zero RTO, recover in seconds
from any type of outage across
AWS regions and accounts.
N2WS #1 Backup & Recovery for AWS
User-friendly “single pane of glass”
with dashboards, monitoring,
alerting, reporting, and third-party
integrations.
11. Automation Use Case with N2WS
COST
CONTROL
DATA STORAGE
Amazon EC2
ROLES
IAM permissions / N2WS roles
BACKUP & RETRIEVAL
N2WS cross-region/account
ARCHIVING
N2WS Freezer or automated
archiving to S3 and/or
Glacier
AUDITING
N2WS reporting
12. To get the most out of your data
lifecycle policies, avoid these
5 data lifecycle management mistakes
13. Do not ignore that we are human
#1 AWS Backup
Compliance
requirements
EBS
Failure
Ransomware AZ
Failure
Human Error/
Malicious intent
Growing costs
“Everything fails all the time” —Werner Vogels
14. Automate data lifecycle management using N2WS
#1 AWS Backup
Eliminate data loss and reduce risk with a
1-click recovery solution that works across
AWS regions and accounts to restore in
seconds.
Many organizations are required to store
data for years. By archiving data to low-
cost storage tiers, you can reduce costs by
75%.
By simply turning off non-critical instances when they’re
not needed, Gett was able to save $100,000 in a single
year.
Provide 100% availability for
workloads in AWS
Reduce compliance costs for your
customers
Save on compute costs (up to 50%
each month)
By switching from homegrown scripts to an automated
solution, Essilor was able to protect their complex
global environment with ease.
Automate backup policies for
complex environments
15. 2 Do not assume that it’s encrypted
Do not assume that data encryption is being actively managed
Do not use the same encryption key for backups AND data
Server-Side Client-Side
ENCRYPTION
16. 3 Do not assume that data is “consistent”
Let’s distinguish between “Crash Consistent” and
“Application Consistent” backup
Source: Nakivo Blog: https://www.nakivo.com/blog/crash-consistent-vs-application-consistent-backup/
17. Cross-Region
#1 AWS Backup
Protect against
regional outages with
cross-region disaster
recovery.
Securely copy
backups between 25
regions around the
world.
19. 4 Do not assume one region is enough
To truly protect your data
from any outage, failure,
or error, you should use
BOTH cross-region AND
cross-account backups
Snapshot Vault: The
ultimate peace of mind
with N2WS
20. 5 Do not treat all data as equal
Storage Classes Archiving Retention
Utilize AWS Tags to automate data lifecycle management action
• Block
• File
• Object
• Amazon S3
• Amazon S3 IA
• Amazon Glacier
• Automate generations
• Utilize freezer
• Decide which backups to delete
21. 5 Do not treat all data as equal
Compliance Recovery Options RTO/RPO
Utilize AWS Tags to automate data lifecycle management action
• Internal
requirements
• Industry
regulations
• Cross Region
• Cross account
• File level
• Instance level
• Volume level
• Maximum disruption time
• Acceptable amount of data loss
22. Optimize data lifecycle management
#1 AWS Backup
Meet any
retention period
N2WS supports multi-tiered
archiving for EBS snapshots
and archiving to S3 or
Glacier for long-term
retention
Turn instances
on/off on demand
Stay compliant
for less
Store data for as long as
you need while paying
less, by archiving to
Amazon S3, Glacier, or
Deep Archive
Reduce your AWS bill by
turning off non-critical
Amazon EC2/RDS instances
on-demand or automatically
23. DB Systel
• DB Systel UK is a division of Deutsche Bahn:
manages 0.5PB of data, backups >1,500
volumes across 700 servers
• Retains data for 7+ years as per compliance
requirements while dramatically lowering
storage costs using Copy to S3.
• Reports on thousands of routes, runs apps and
dispatches/runs trains for 12 million daily
passengers
700+ Amazon instances
1500 data volumes
CASE STUDIES
0.5 petabytes of
data
80+ Amazon instances
4 AWS accounts
$100K in cost
savings
• On-demand transport company in over 120
countries. Greater control over its backup
and restore strategy helped Gett prepare for
its IPO in 2020.
• No longer storing old snapshots. Saved
~50% or $2,000—per instance per month. IT
savings >$100k/year.
• Uses Resource Control to automate the
stop/start of instances that they don’t need.
24. Used by AWS builders, worldwide
AWS Accounts
5K+
Petabytes of Backup
13+
HUNDREDS of
THOUSANDS of
Protected Instances
25. Talk to us!
#1 AWS Backup
We value your feedback
Questions? Email: Elizabeth.Lewis@n2ws.com
AWS Cost Estimator:
https://n2ws.com/aws-cost-
savings-calculator