SlideShare a Scribd company logo
1 of 21
Download to read offline
© 2020, Amazon Web Services, Inc. or its Affiliates.
Ray Wang, Solutions Architect
April 2020
Data Migration to AWS
with DataSync & Transfer for SFTP
© 2020, Amazon Web Services, Inc. or its Affiliates.© 2020, Amazon Web Services, Inc. or its Affiliates.
Agenda
Challenges - Problem we want to solve
AWS DataSync
AWS Transfer for SFTP
Demo - DataSync & Transfer for SFTP
© 2020, Amazon Web Services, Inc. or its Affiliates.© 2020, Amazon Web Services, Inc. or its Affiliates.
Challenges
Problem we want to solve
© 2020, Amazon Web Services, Inc. or its Affiliates.
Do you want to …
Migrate application data & archives to the cloud?
Replicate files for protection and recovery?
Reduce spending on storage hardware and management?
Move data to the cloud for processing?
© 2020, Amazon Web Services, Inc. or its Affiliates.
Why can’t I just transfer data to the cloud on my own?
With DIY transfers, you own:
Assuring
security and
data validity
Network and
system
complexity
Scaling
transfers
Staff time &
resources
Managing
performance
& throughput
Data transfer at scale can quickly becomes complex and time-consuming
© 2020, Amazon Web Services, Inc. or its Affiliates.© 2020, Amazon Web Services, Inc. or its Affiliates.
AWS DataSync
© 2020, Amazon Web Services, Inc. or its Affiliates.
Introducing AWS DataSync
Online transfer service that simplifies, automates, and
accelerates moving data between on-premises storage and AWS
Fast data
transfer
Cost-
effective
The speed and reliability of network acceleration software with
the cost-effectiveness of open source tools
Easy to use Secure and
reliable
Cloud
integrated
AWS
© 2020, Amazon Web Services, Inc. or its Affiliates.
Deploy agent on
VMware or EC2 for
efficient access to
local NFS or SMB server
Secure highly parallel
transfers using
optimized network
protocol
Optimized reads and
writes to Amazon S3,
Amazon EFS, and Amazon
FSx for Windows File
Server
Fully managed service
scales to send or
receive data from agent
On-Premises Region
AWS DataSyncShared
file system
AWS DataSync
agent
NFS or SMB
TLS
AWS DataSync: How it works
AWS Storage Resources
Amazon S3
Direct to all storage classes
Amazon Elastic File
System
Amazon FSx for
Windows File Server
© 2020, Amazon Web Services, Inc. or its Affiliates.
Start transferring files quickly
Start the task and
monitor progress
3
Deploy on-premises
agent and connect to
your AWS account
1
Create a task by
configuring storage
locations and options
2
© 2020, Amazon Web Services, Inc. or its Affiliates.
What can you do with DataSync?
Migration of
active application
data
Transfers for
timely in-cloud
processing
Replication for data
protection and
recovery
Archive to free
up on-premises
storage capacity
© 2020, Amazon Web Services, Inc. or its Affiliates.
Complex management of 2.4PB of on-premises Data
Domain array
Data retention required data to be retained for many
years
Problem
Solution
Outcome
Chose Amazon S3 for low cost, pay-as-you-go
model as well as versioning support
DataSync to move data to S3 as soon as possible
Performance: transfer rate of ~500 GB/hour.
Successfully transferred and verified 700TB
dataset to S3 in 2.5 months
Leader in 3D design, engineering, and
entertainment software. Autodesk
makes software for people who make
things.
© 2020, Amazon Web Services, Inc. or its Affiliates.
Call to action
Deploy a DataSync
agent
Configure your
storage locations
Learn more at https://aws.amazon.com/datasync
Start transferring
data today!
Try it out ...
© 2020, Amazon Web Services, Inc. or its Affiliates.© 2020, Amazon Web Services, Inc. or its Affiliates.
AWS Transfer for SFTP
© 2020, Amazon Web Services, Inc. or its Affiliates.
Introducing AWS Transfer for SFTP
Cost-effective
Seamless migration of
existing SFTP workflows
Native integration
with AWS services
SimpleFully managed
in AWS
Secure
and compliant
Fully-managed SFTP service enabling transfer of data into Amazon S3
© 2020, Amazon Web Services, Inc. or its Affiliates.
Transfer for SFTP: How it works
SFTP
users
AWS Cloud
Amazon S3
No changes to end-user
credentials, firewall
configurations, or scripts
Custom identity provider
Amazon
API Gateway
AWS
Lambda
Modernize
your workflow
using cloud
native services
Amazon
Athena
Amazon
Redshift
Amazon
EMR
Amazon
SageMaker
Your VPC or
data center
DIY SFTP server
and data
No need to manage
SFTP infrastructure
AWS Transfer for
SFTP
© 2020, Amazon Web Services, Inc. or its Affiliates.
AWS Transfer for SFTP Customers
© 2020, Amazon Web Services, Inc. or its Affiliates.
Problem
• multiple SFTP endpoints
• inconsistent security
• operationally inefficient to manage
Solution
• No infrastructure to manage,
automatic scaling, built-in DR
• Reduced operational burden
• Seamless cutover; using existing
authentication systems
Seamless migration of
existing workflows
”AWS Transfer for SFTP is helping us reduce our
operational burden, keeping S3 as our data lake, while
maintaining critical connection to our existing
authentication systems for external users – so they
don’t experience any disruption as we complete our
migration of SFTP services to AWS.”
~Ranga Rajagopal, Senior Director, FINRA
© 2020, Amazon Web Services, Inc. or its Affiliates.
Problem
• multiple SFTP endpoints
• inconsistent security
• operationally inefficient to manage
Solution
• No infrastructure to manage,
automatic scaling, built-in DR
• Reduced operational burden
• Seamless cutover; using existing
authentication systems
Simple to use
“AWS Transfer for SFTP was an instant win for
Celgene…We stood up our SFTP endpoint using the
service within minutes, attached it to our bucket,
provisioned users, and were in business in under an
hour. That speed and flexibility is simply not
possible in a traditional on-prem. world.”
~Lance Smith, Associate Director, Global Planning &
Technology, Celgene
© 2020, Amazon Web Services, Inc. or its Affiliates.
Try out AWS SFTP
Deploy an SFTP
server endpoint
1 Select your target
S3 bucket(s)
2 Set up your users3
Visit console.aws.amazon.com/transfer to get started today!
Pricing
• SFTP server endpoint time: $0.30/protocol/hour
• Data uploads and downloads: $0.04/GB
© 2020, Amazon Web Services, Inc. or its Affiliates.© 2020, Amazon Web Services, Inc. or its Affiliates.
Demo
DataSync/S3 SFTP/AWSCLI S3 Sync
© 2020, Amazon Web Services, Inc. or its Affiliates.
Thank you!
Ray Wang, Solutions Architect
hsiawang@amazon.com

More Related Content

What's hot

AWS Landing Zone Deep Dive (ENT350-R2) - AWS re:Invent 2018
AWS Landing Zone Deep Dive (ENT350-R2) - AWS re:Invent 2018AWS Landing Zone Deep Dive (ENT350-R2) - AWS re:Invent 2018
AWS Landing Zone Deep Dive (ENT350-R2) - AWS re:Invent 2018Amazon Web Services
 
Getting Started with AWS Database Migration Service
Getting Started with AWS Database Migration ServiceGetting Started with AWS Database Migration Service
Getting Started with AWS Database Migration ServiceAmazon Web Services
 
AWS Security Week: AWS Secrets Manager
AWS Security Week: AWS Secrets ManagerAWS Security Week: AWS Secrets Manager
AWS Security Week: AWS Secrets ManagerAmazon Web Services
 
Introduction to the Well-Architected Framework and Tool - SVC208 - Anaheim AW...
Introduction to the Well-Architected Framework and Tool - SVC208 - Anaheim AW...Introduction to the Well-Architected Framework and Tool - SVC208 - Anaheim AW...
Introduction to the Well-Architected Framework and Tool - SVC208 - Anaheim AW...Amazon Web Services
 
Amazon EKS - Elastic Container Service for Kubernetes
Amazon EKS - Elastic Container Service for KubernetesAmazon EKS - Elastic Container Service for Kubernetes
Amazon EKS - Elastic Container Service for KubernetesAmazon Web Services
 
How to backup, restore and archive your data on AWS
How to backup, restore and archive your data on AWSHow to backup, restore and archive your data on AWS
How to backup, restore and archive your data on AWSAmazon Web Services
 
Introduction to AWS Cost Management
Introduction to AWS Cost ManagementIntroduction to AWS Cost Management
Introduction to AWS Cost ManagementAmazon Web Services
 
Cloud Migration, Application Modernization, and Security
Cloud Migration, Application Modernization, and Security Cloud Migration, Application Modernization, and Security
Cloud Migration, Application Modernization, and Security Tom Laszewski
 
AWS re:Invent 2016: Workshop: Secure Your Web Application with AWS WAF and Am...
AWS re:Invent 2016: Workshop: Secure Your Web Application with AWS WAF and Am...AWS re:Invent 2016: Workshop: Secure Your Web Application with AWS WAF and Am...
AWS re:Invent 2016: Workshop: Secure Your Web Application with AWS WAF and Am...Amazon Web Services
 
How to build a data lake with aws glue data catalog (ABD213-R) re:Invent 2017
How to build a data lake with aws glue data catalog (ABD213-R)  re:Invent 2017How to build a data lake with aws glue data catalog (ABD213-R)  re:Invent 2017
How to build a data lake with aws glue data catalog (ABD213-R) re:Invent 2017Amazon Web Services
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
 
CI-CD with AWS Developer Tools and Fargate_AWSPSSummit_Singapore
CI-CD with AWS Developer Tools and Fargate_AWSPSSummit_SingaporeCI-CD with AWS Developer Tools and Fargate_AWSPSSummit_Singapore
CI-CD with AWS Developer Tools and Fargate_AWSPSSummit_SingaporeAmazon Web Services
 
Deploy and Govern at Scale with AWS Control Tower
Deploy and Govern at Scale with AWS Control TowerDeploy and Govern at Scale with AWS Control Tower
Deploy and Govern at Scale with AWS Control TowerAmazon Web Services
 

What's hot (20)

AWS Landing Zone Deep Dive (ENT350-R2) - AWS re:Invent 2018
AWS Landing Zone Deep Dive (ENT350-R2) - AWS re:Invent 2018AWS Landing Zone Deep Dive (ENT350-R2) - AWS re:Invent 2018
AWS Landing Zone Deep Dive (ENT350-R2) - AWS re:Invent 2018
 
Getting Started with AWS Database Migration Service
Getting Started with AWS Database Migration ServiceGetting Started with AWS Database Migration Service
Getting Started with AWS Database Migration Service
 
AWS Security Week: AWS Secrets Manager
AWS Security Week: AWS Secrets ManagerAWS Security Week: AWS Secrets Manager
AWS Security Week: AWS Secrets Manager
 
AWS Secrets Manager
AWS Secrets ManagerAWS Secrets Manager
AWS Secrets Manager
 
Introduction to the Well-Architected Framework and Tool - SVC208 - Anaheim AW...
Introduction to the Well-Architected Framework and Tool - SVC208 - Anaheim AW...Introduction to the Well-Architected Framework and Tool - SVC208 - Anaheim AW...
Introduction to the Well-Architected Framework and Tool - SVC208 - Anaheim AW...
 
Amazon EKS - Elastic Container Service for Kubernetes
Amazon EKS - Elastic Container Service for KubernetesAmazon EKS - Elastic Container Service for Kubernetes
Amazon EKS - Elastic Container Service for Kubernetes
 
How to backup, restore and archive your data on AWS
How to backup, restore and archive your data on AWSHow to backup, restore and archive your data on AWS
How to backup, restore and archive your data on AWS
 
AWS for Backup and Recovery
AWS for Backup and RecoveryAWS for Backup and Recovery
AWS for Backup and Recovery
 
AWS 101
AWS 101AWS 101
AWS 101
 
Introduction to AWS Cost Management
Introduction to AWS Cost ManagementIntroduction to AWS Cost Management
Introduction to AWS Cost Management
 
AWS Security by Design
AWS Security by Design AWS Security by Design
AWS Security by Design
 
AWS Lambda
AWS LambdaAWS Lambda
AWS Lambda
 
Cloud Migration, Application Modernization, and Security
Cloud Migration, Application Modernization, and Security Cloud Migration, Application Modernization, and Security
Cloud Migration, Application Modernization, and Security
 
AWS re:Invent 2016: Workshop: Secure Your Web Application with AWS WAF and Am...
AWS re:Invent 2016: Workshop: Secure Your Web Application with AWS WAF and Am...AWS re:Invent 2016: Workshop: Secure Your Web Application with AWS WAF and Am...
AWS re:Invent 2016: Workshop: Secure Your Web Application with AWS WAF and Am...
 
Fundamentals of AWS Security
Fundamentals of AWS SecurityFundamentals of AWS Security
Fundamentals of AWS Security
 
How to build a data lake with aws glue data catalog (ABD213-R) re:Invent 2017
How to build a data lake with aws glue data catalog (ABD213-R)  re:Invent 2017How to build a data lake with aws glue data catalog (ABD213-R)  re:Invent 2017
How to build a data lake with aws glue data catalog (ABD213-R) re:Invent 2017
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Cloud Migration Workshop
Cloud Migration WorkshopCloud Migration Workshop
Cloud Migration Workshop
 
CI-CD with AWS Developer Tools and Fargate_AWSPSSummit_Singapore
CI-CD with AWS Developer Tools and Fargate_AWSPSSummit_SingaporeCI-CD with AWS Developer Tools and Fargate_AWSPSSummit_Singapore
CI-CD with AWS Developer Tools and Fargate_AWSPSSummit_Singapore
 
Deploy and Govern at Scale with AWS Control Tower
Deploy and Govern at Scale with AWS Control TowerDeploy and Govern at Scale with AWS Control Tower
Deploy and Govern at Scale with AWS Control Tower
 

Similar to Data Migration to AWS with DataSync & Transfer for SFTP

Make your data move: Best practices for migrating data to AWS - STG201 - New ...
Make your data move: Best practices for migrating data to AWS - STG201 - New ...Make your data move: Best practices for migrating data to AWS - STG201 - New ...
Make your data move: Best practices for migrating data to AWS - STG201 - New ...Amazon Web Services
 
Lessons from Migrating Oracle Databases to Amazon RDS or Amazon Aurora
Lessons from Migrating Oracle Databases to Amazon RDS or Amazon Aurora Lessons from Migrating Oracle Databases to Amazon RDS or Amazon Aurora
Lessons from Migrating Oracle Databases to Amazon RDS or Amazon Aurora Datavail
 
Migrating Your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...
Migrating Your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...Migrating Your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...
Migrating Your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...Amazon Web Services
 
Initiate Edinburgh 2019 - Migrating Data to the Cloud
Initiate Edinburgh 2019 - Migrating Data to the CloudInitiate Edinburgh 2019 - Migrating Data to the Cloud
Initiate Edinburgh 2019 - Migrating Data to the CloudAmazon Web Services
 
Migrating Massive Databases and Data Warehouses to the Cloud - ENT327 - re:In...
Migrating Massive Databases and Data Warehouses to the Cloud - ENT327 - re:In...Migrating Massive Databases and Data Warehouses to the Cloud - ENT327 - re:In...
Migrating Massive Databases and Data Warehouses to the Cloud - ENT327 - re:In...Amazon Web Services
 
Getting started on your AWS migration journey
Getting started on your AWS migration journeyGetting started on your AWS migration journey
Getting started on your AWS migration journeyAmazon Web Services
 
Make Your Data Move: Best Practices for Migrating Data to AWS
Make Your Data Move: Best Practices for Migrating Data to AWSMake Your Data Move: Best Practices for Migrating Data to AWS
Make Your Data Move: Best Practices for Migrating Data to AWSAmazon Web Services
 
Heterogenous Migration with DMS & SCT - Michael Russo
Heterogenous Migration with DMS & SCT - Michael RussoHeterogenous Migration with DMS & SCT - Michael Russo
Heterogenous Migration with DMS & SCT - Michael RussoAmazon Web Services
 
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech TalksMigrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech TalksAmazon Web Services
 
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech TalksMigrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech TalksAmazon Web Services
 
AWS Initiate Day Dublin 2019 – Migrating Data to the Cloud
AWS Initiate Day Dublin 2019 – Migrating Data to the CloudAWS Initiate Day Dublin 2019 – Migrating Data to the Cloud
AWS Initiate Day Dublin 2019 – Migrating Data to the CloudAmazon Web Services
 
SRV422 Deep Dive on AWS Database Migration Service
SRV422 Deep Dive on AWS Database Migration ServiceSRV422 Deep Dive on AWS Database Migration Service
SRV422 Deep Dive on AWS Database Migration ServiceAmazon Web Services
 
Migrating your IT - AWS Summit Cape Town 2018
Migrating your IT - AWS Summit Cape Town 2018Migrating your IT - AWS Summit Cape Town 2018
Migrating your IT - AWS Summit Cape Town 2018Amazon Web Services
 
Heterogenous Migration with DMS and SCT: Database Week San Francisco
Heterogenous Migration with DMS and SCT: Database Week San FranciscoHeterogenous Migration with DMS and SCT: Database Week San Francisco
Heterogenous Migration with DMS and SCT: Database Week San FranciscoAmazon Web Services
 
Discuss data migration with AWS experts - STG304 - Santa Clara AWS Summit
Discuss data migration with AWS experts - STG304 - Santa Clara AWS SummitDiscuss data migration with AWS experts - STG304 - Santa Clara AWS Summit
Discuss data migration with AWS experts - STG304 - Santa Clara AWS SummitAmazon Web Services
 
Modernizing Databases with DMS - Michael Russo
Modernizing Databases with DMS - Michael Russo Modernizing Databases with DMS - Michael Russo
Modernizing Databases with DMS - Michael Russo Amazon Web Services
 
Bursting on-premise analytic workloads to Amazon EMR using Alluxio
Bursting on-premise analytic workloads to Amazon EMR using AlluxioBursting on-premise analytic workloads to Amazon EMR using Alluxio
Bursting on-premise analytic workloads to Amazon EMR using AlluxioAlluxio, Inc.
 
Migrating Databases to the Cloud: Introduction to AWS DMS - SRV215 - Anaheim ...
Migrating Databases to the Cloud: Introduction to AWS DMS - SRV215 - Anaheim ...Migrating Databases to the Cloud: Introduction to AWS DMS - SRV215 - Anaheim ...
Migrating Databases to the Cloud: Introduction to AWS DMS - SRV215 - Anaheim ...Amazon Web Services
 
AWS Advanced Analytics Automation Toolkit (AAA)
AWS Advanced Analytics Automation Toolkit (AAA)AWS Advanced Analytics Automation Toolkit (AAA)
AWS Advanced Analytics Automation Toolkit (AAA)CloudHesive
 

Similar to Data Migration to AWS with DataSync & Transfer for SFTP (20)

Make your data move: Best practices for migrating data to AWS - STG201 - New ...
Make your data move: Best practices for migrating data to AWS - STG201 - New ...Make your data move: Best practices for migrating data to AWS - STG201 - New ...
Make your data move: Best practices for migrating data to AWS - STG201 - New ...
 
Lessons from Migrating Oracle Databases to Amazon RDS or Amazon Aurora
Lessons from Migrating Oracle Databases to Amazon RDS or Amazon Aurora Lessons from Migrating Oracle Databases to Amazon RDS or Amazon Aurora
Lessons from Migrating Oracle Databases to Amazon RDS or Amazon Aurora
 
Migrating Your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...
Migrating Your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...Migrating Your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...
Migrating Your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...
 
Initiate Edinburgh 2019 - Migrating Data to the Cloud
Initiate Edinburgh 2019 - Migrating Data to the CloudInitiate Edinburgh 2019 - Migrating Data to the Cloud
Initiate Edinburgh 2019 - Migrating Data to the Cloud
 
Migrating Massive Databases and Data Warehouses to the Cloud - ENT327 - re:In...
Migrating Massive Databases and Data Warehouses to the Cloud - ENT327 - re:In...Migrating Massive Databases and Data Warehouses to the Cloud - ENT327 - re:In...
Migrating Massive Databases and Data Warehouses to the Cloud - ENT327 - re:In...
 
Getting started on your AWS migration journey
Getting started on your AWS migration journeyGetting started on your AWS migration journey
Getting started on your AWS migration journey
 
Make Your Data Move: Best Practices for Migrating Data to AWS
Make Your Data Move: Best Practices for Migrating Data to AWSMake Your Data Move: Best Practices for Migrating Data to AWS
Make Your Data Move: Best Practices for Migrating Data to AWS
 
Heterogenous Migration with DMS & SCT - Michael Russo
Heterogenous Migration with DMS & SCT - Michael RussoHeterogenous Migration with DMS & SCT - Michael Russo
Heterogenous Migration with DMS & SCT - Michael Russo
 
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech TalksMigrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
 
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech TalksMigrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
 
AWS Initiate Day Dublin 2019 – Migrating Data to the Cloud
AWS Initiate Day Dublin 2019 – Migrating Data to the CloudAWS Initiate Day Dublin 2019 – Migrating Data to the Cloud
AWS Initiate Day Dublin 2019 – Migrating Data to the Cloud
 
SRV422 Deep Dive on AWS Database Migration Service
SRV422 Deep Dive on AWS Database Migration ServiceSRV422 Deep Dive on AWS Database Migration Service
SRV422 Deep Dive on AWS Database Migration Service
 
Migrating your IT - AWS Summit Cape Town 2018
Migrating your IT - AWS Summit Cape Town 2018Migrating your IT - AWS Summit Cape Town 2018
Migrating your IT - AWS Summit Cape Town 2018
 
Heterogenous Migration with DMS and SCT: Database Week San Francisco
Heterogenous Migration with DMS and SCT: Database Week San FranciscoHeterogenous Migration with DMS and SCT: Database Week San Francisco
Heterogenous Migration with DMS and SCT: Database Week San Francisco
 
Discuss data migration with AWS experts - STG304 - Santa Clara AWS Summit
Discuss data migration with AWS experts - STG304 - Santa Clara AWS SummitDiscuss data migration with AWS experts - STG304 - Santa Clara AWS Summit
Discuss data migration with AWS experts - STG304 - Santa Clara AWS Summit
 
Modernizing Databases with DMS - Michael Russo
Modernizing Databases with DMS - Michael Russo Modernizing Databases with DMS - Michael Russo
Modernizing Databases with DMS - Michael Russo
 
Bursting on-premise analytic workloads to Amazon EMR using Alluxio
Bursting on-premise analytic workloads to Amazon EMR using AlluxioBursting on-premise analytic workloads to Amazon EMR using Alluxio
Bursting on-premise analytic workloads to Amazon EMR using Alluxio
 
Migrating your IT - Final
Migrating your IT - FinalMigrating your IT - Final
Migrating your IT - Final
 
Migrating Databases to the Cloud: Introduction to AWS DMS - SRV215 - Anaheim ...
Migrating Databases to the Cloud: Introduction to AWS DMS - SRV215 - Anaheim ...Migrating Databases to the Cloud: Introduction to AWS DMS - SRV215 - Anaheim ...
Migrating Databases to the Cloud: Introduction to AWS DMS - SRV215 - Anaheim ...
 
AWS Advanced Analytics Automation Toolkit (AAA)
AWS Advanced Analytics Automation Toolkit (AAA)AWS Advanced Analytics Automation Toolkit (AAA)
AWS Advanced Analytics Automation Toolkit (AAA)
 

More from Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateAmazon Web Services
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSAmazon Web Services
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareAmazon Web Services
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAmazon Web Services
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWSAmazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckAmazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without serversAmazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceAmazon Web Services
 
Come costruire un'architettura Serverless nel Cloud AWS
Come costruire un'architettura Serverless nel Cloud AWSCome costruire un'architettura Serverless nel Cloud AWS
Come costruire un'architettura Serverless nel Cloud AWSAmazon Web Services
 

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 
Come costruire un'architettura Serverless nel Cloud AWS
Come costruire un'architettura Serverless nel Cloud AWSCome costruire un'architettura Serverless nel Cloud AWS
Come costruire un'architettura Serverless nel Cloud AWS
 

Data Migration to AWS with DataSync & Transfer for SFTP

  • 1. © 2020, Amazon Web Services, Inc. or its Affiliates. Ray Wang, Solutions Architect April 2020 Data Migration to AWS with DataSync & Transfer for SFTP
  • 2. © 2020, Amazon Web Services, Inc. or its Affiliates.© 2020, Amazon Web Services, Inc. or its Affiliates. Agenda Challenges - Problem we want to solve AWS DataSync AWS Transfer for SFTP Demo - DataSync & Transfer for SFTP
  • 3. © 2020, Amazon Web Services, Inc. or its Affiliates.© 2020, Amazon Web Services, Inc. or its Affiliates. Challenges Problem we want to solve
  • 4. © 2020, Amazon Web Services, Inc. or its Affiliates. Do you want to … Migrate application data & archives to the cloud? Replicate files for protection and recovery? Reduce spending on storage hardware and management? Move data to the cloud for processing?
  • 5. © 2020, Amazon Web Services, Inc. or its Affiliates. Why can’t I just transfer data to the cloud on my own? With DIY transfers, you own: Assuring security and data validity Network and system complexity Scaling transfers Staff time & resources Managing performance & throughput Data transfer at scale can quickly becomes complex and time-consuming
  • 6. © 2020, Amazon Web Services, Inc. or its Affiliates.© 2020, Amazon Web Services, Inc. or its Affiliates. AWS DataSync
  • 7. © 2020, Amazon Web Services, Inc. or its Affiliates. Introducing AWS DataSync Online transfer service that simplifies, automates, and accelerates moving data between on-premises storage and AWS Fast data transfer Cost- effective The speed and reliability of network acceleration software with the cost-effectiveness of open source tools Easy to use Secure and reliable Cloud integrated AWS
  • 8. © 2020, Amazon Web Services, Inc. or its Affiliates. Deploy agent on VMware or EC2 for efficient access to local NFS or SMB server Secure highly parallel transfers using optimized network protocol Optimized reads and writes to Amazon S3, Amazon EFS, and Amazon FSx for Windows File Server Fully managed service scales to send or receive data from agent On-Premises Region AWS DataSyncShared file system AWS DataSync agent NFS or SMB TLS AWS DataSync: How it works AWS Storage Resources Amazon S3 Direct to all storage classes Amazon Elastic File System Amazon FSx for Windows File Server
  • 9. © 2020, Amazon Web Services, Inc. or its Affiliates. Start transferring files quickly Start the task and monitor progress 3 Deploy on-premises agent and connect to your AWS account 1 Create a task by configuring storage locations and options 2
  • 10. © 2020, Amazon Web Services, Inc. or its Affiliates. What can you do with DataSync? Migration of active application data Transfers for timely in-cloud processing Replication for data protection and recovery Archive to free up on-premises storage capacity
  • 11. © 2020, Amazon Web Services, Inc. or its Affiliates. Complex management of 2.4PB of on-premises Data Domain array Data retention required data to be retained for many years Problem Solution Outcome Chose Amazon S3 for low cost, pay-as-you-go model as well as versioning support DataSync to move data to S3 as soon as possible Performance: transfer rate of ~500 GB/hour. Successfully transferred and verified 700TB dataset to S3 in 2.5 months Leader in 3D design, engineering, and entertainment software. Autodesk makes software for people who make things.
  • 12. © 2020, Amazon Web Services, Inc. or its Affiliates. Call to action Deploy a DataSync agent Configure your storage locations Learn more at https://aws.amazon.com/datasync Start transferring data today! Try it out ...
  • 13. © 2020, Amazon Web Services, Inc. or its Affiliates.© 2020, Amazon Web Services, Inc. or its Affiliates. AWS Transfer for SFTP
  • 14. © 2020, Amazon Web Services, Inc. or its Affiliates. Introducing AWS Transfer for SFTP Cost-effective Seamless migration of existing SFTP workflows Native integration with AWS services SimpleFully managed in AWS Secure and compliant Fully-managed SFTP service enabling transfer of data into Amazon S3
  • 15. © 2020, Amazon Web Services, Inc. or its Affiliates. Transfer for SFTP: How it works SFTP users AWS Cloud Amazon S3 No changes to end-user credentials, firewall configurations, or scripts Custom identity provider Amazon API Gateway AWS Lambda Modernize your workflow using cloud native services Amazon Athena Amazon Redshift Amazon EMR Amazon SageMaker Your VPC or data center DIY SFTP server and data No need to manage SFTP infrastructure AWS Transfer for SFTP
  • 16. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS Transfer for SFTP Customers
  • 17. © 2020, Amazon Web Services, Inc. or its Affiliates. Problem • multiple SFTP endpoints • inconsistent security • operationally inefficient to manage Solution • No infrastructure to manage, automatic scaling, built-in DR • Reduced operational burden • Seamless cutover; using existing authentication systems Seamless migration of existing workflows ”AWS Transfer for SFTP is helping us reduce our operational burden, keeping S3 as our data lake, while maintaining critical connection to our existing authentication systems for external users – so they don’t experience any disruption as we complete our migration of SFTP services to AWS.” ~Ranga Rajagopal, Senior Director, FINRA
  • 18. © 2020, Amazon Web Services, Inc. or its Affiliates. Problem • multiple SFTP endpoints • inconsistent security • operationally inefficient to manage Solution • No infrastructure to manage, automatic scaling, built-in DR • Reduced operational burden • Seamless cutover; using existing authentication systems Simple to use “AWS Transfer for SFTP was an instant win for Celgene…We stood up our SFTP endpoint using the service within minutes, attached it to our bucket, provisioned users, and were in business in under an hour. That speed and flexibility is simply not possible in a traditional on-prem. world.” ~Lance Smith, Associate Director, Global Planning & Technology, Celgene
  • 19. © 2020, Amazon Web Services, Inc. or its Affiliates. Try out AWS SFTP Deploy an SFTP server endpoint 1 Select your target S3 bucket(s) 2 Set up your users3 Visit console.aws.amazon.com/transfer to get started today! Pricing • SFTP server endpoint time: $0.30/protocol/hour • Data uploads and downloads: $0.04/GB
  • 20. © 2020, Amazon Web Services, Inc. or its Affiliates.© 2020, Amazon Web Services, Inc. or its Affiliates. Demo DataSync/S3 SFTP/AWSCLI S3 Sync
  • 21. © 2020, Amazon Web Services, Inc. or its Affiliates. Thank you! Ray Wang, Solutions Architect hsiawang@amazon.com