When evaluating and planning migrating your data from on premises to the Cloud, you might encounter physical limitations. Amazon offers a suite of tools to help you surmount these limitations by moving data using networks, roads, and technology partners. In this session, we discuss how to move large amounts of data into and out of the Cloud in batches, increments, and streams.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Data Transfer Services
Data Ingest Strategies into the AWS Cloud
Ken Chan
Product Business Development Manager
Greater China
kenchan@amazon.com

2. Storage is the gravity
for cloud applications

3. Customers have Data Centers
Depl
oyDeploy

4. Batches and Streams
Direct
Connect
Snowball,
Snowball Edge,
Snowmobile
3rd Party
Connectors
Transfer
Acceleration
Storage
Gateway
Kinesis Firehose
File
Amazon EFS
Block
Amazon EBS
(persistent)
Object
Amazon GlacierAmazon S3 Amazon EC2
Instance Store
(ephemeral)
Internet/VPN CloudFront

5. Long Term
Archive
All tiers
accessible through
a single API
Oldest content
trickles down to
glacier
automatically to
save cost
Amazon S3
S3 In-frequent Access
Amazon Glacier
LifeCyclePolicies
Onsite
Storage
Frequently
Accessed
New
S3 Life Cycle Policy Use case for Media

6. Internet/VPN ingest

7. What is Internet/VPN?
Globally available
Default method of ingesting content into Amazon S3
Simple standards-based (HTTP) connection
Use your existing internet connection
Available in a VPC for VPN connectivity
Acceleration through multipart upload
Data transfer into AWS is free
VPN connections using VPC virtual private gateway
•$0.05 per VPN connection-hour
•$0.048 per VPN connection-hour for connections to the Tokyo region

8. How does Internet/VPN ingest work?
Accelerate data transfer using
multipart upload
Ingest data directly into S3 buckets
with existing internet connectivity
S3 bucket
AWS Region
and
through the console or API
customer
gateway
endpoints
VPN
connection
Internet Internet through VPN +
VPC

9. Amazon S3 Transfer Acceleration

10. What is Transfer Acceleration?
Network- and protocol-based data transfer service
Acceleration of data ingress/egress with S3 buckets
Typically 50% to 300% faster
Feature of S3 enabled at the bucket level
Available in all S3 regions worldwide
No client/server software required
No code changes to your application
No firewall exceptions
Simple pricing model

11. Ingest & egress with Transfer Acceleration
S3 bucket
AWS edge
location
Uploader
Optimized
throughput!
Uses AWS 59 global edge locations
AWS determines best edge location
Data transfer optimized between
edge and customer, and edge and S3
Data is not stored on the edge cache

12. Customers: Frame.io, Hudl, Viocorp
Problem Statement:
• Needed to accelerate customer content ingest into their respective
applications running on AWS
• Existing ingest options were proprietary and too expensive
Use of AWS:
• S3 and S3 transfer acceleration for massively scalable ingest
• S3 for storage, CloudFront and S3 transfer acceleration for ingest
Business Benefits:
• Global highly distributed data transport available on demand
• Massive scalability and elasticity
• Lower TCO for storage and data transport infrastructure
Accelerating media content uploads to their platforms
S3 Bucket
AWS Edge
Location
Uploader
Optimized
Throughput!

13. Rio De
Janeiro
Warsaw New York Atlanta Madrid Virginia Melbourne Paris Los
Angeles
Seattle Tokyo Singapore
Time[hrs]
500 GB upload from these edge locations to a bucket in Singapore
Public internet
How fast is S3 Transfer Acceleration?
S3 transfer acceleration

14. AWS Direct Connect

15. What is AWS Direct Connect?
Dedicated, 1 or 10 GE private pipes into AWS
Create private (VPC) or public virtual interfaces to AWS
Reduced data-out rates (data-in still free)
Consistent network performance
At least 1 location to each AWS region
Option for redundant connections
Uses BGP to exchange routing information over a VLAN

16. Getting Started with DX

17. Create Connection to issue LOA

18. Pass this LOA to our DX partner to get cross connection setup

19. At the Direct Connect location
CORP
AWS Direct
Connect
Routers
Customer
Router
Colocation
DX Location
Customer
network
`
AWS backbone
network
Cross-
connect
Customer
router
Customer’s network
Demarcation

20. Dedicated port through Direct Connect partner
CORP
AWS Direct
Connect
Routers
Colocation
DX Location
Partner network
AWS backbone
network
Cross-
connect
Customer
router
Partner
network
Access
circuit
Demarcation
Partner
equipment

21. Hybrid cloud storage expansion:
Amazon EFS through Direct Connect
“Bursting”
File Workloads
Data Migration
into EFS
Amazon EFSOn-Premises AWS Direct Connect

22. AWS Storage Gateway

23. What is AWS Storage Gateway?
Works with your existing applications
Secure and durable storage in AWS
Low latency for frequently used data
Scalable and cost-effective on-premises storage - $.01/GB
written to AWS + S3/Amazon Glacier storage fees
Service connecting an on-premises software appliance
with cloud-based storage

24. Hybrid storage use cases and architectures for
AWS Storage Gateway
Enabling cloud workloads
Move data to AWS storage for Big Data, cloud bursting, or migration
Tiered cloud storage
Easily add AWS storage to your on-premises environment
Backup, archive, and disaster recovery
Cost effective storage in AWS with local or cloud restore

25. Storage Gateway hybrid storage solutions
Enables using standard storage protocols to access AWS storage services
Customer Premises
Storage
Gateway
Amazon EBS
snapshots
Amazon
S3
Amazon Glacier
AWS Identity and Access
Management (IAM)
AWS Key Management
Service (KMS)
AWS
CloudTrail
Amazon
CloudWatch
Enterprise
storage
Devices
Application
servers

26. Storage gateway – Files, volumes, and tapes
File gateway NFS (v3 and v4.1) interface **NEW!**
On-premises file storage backed by Amazon S3 objects
Volume gateway iSCSI block interface
On-premises block storage backed by Amazon S3 with EBS snapshots
Tape gateway iSCSI virtual tape library (VTL) interface
Virtual tape storage in Amazon S3 and Glacier with VTL management

27. Detail: AWS File Gateway for S3
NFS Interface ElasticityAmazon S3 Bucket
Easy Integration Cloud ScaleCloud Access

28. AWS Snowball

29. What is AWS Snowball?
Petabyte-scale data transport
E-ink shipping
label
Ruggedized case
“8.5G impact”
All data encrypted
end-to-end
Rain- and dust-
resistant
Tamper-resistant
case and
electronics
80 TB
10 GE network

30. AWS storage migration expansion:
AWS Snowball
Transfer
Capacity
Integration
Regional
Availability
80TB model
HDFS support
3rd party API
HIPAA support
Continue to expand

31. How it works

32. How fast is Snowball?
• Less than 1 day to transfer 200TB via 3x10G connections with 3
Snowballs, less than 1 week including shipping
• Number of days to transfer 200TB via the Internet at typical utilizations
Internet Connection Speed
Utilization 1Gbps 500Mbps 300Mbps 150Mbps
25% 71 141 236 471
50% 36 71 118 236
75% 24 47 225 157

33. Customer: Scripps Networks Interactive
Problem Statement:
• Need storage platform to manage active archive content
• Existing content repository too large to migrate via available
network-based ingest methods
Use of AWS:
• S3 and Snowball for massively scalable ingest
• S3 for storage, Glacier for content archive
• Snowball to securely transport existing media content from on-
premises storage and tape vault
Business Benefits:
• Petabyte-scale data transport without increased network costs
• Massive scalability and elasticity
• Lower TCO for active archive storage
Active archive transport and archival for digital content provider

34. AWS storage migration expansion:
AWS Snowmobile

35. Hybrid cloud storage expansion:
AWS Snowball Edge
On-premises
Capacity
On-premises
Integration
On-premises
Compute
Clustered local storage
100TB capacity
NFS and S3-compatible
endpoint
AWS Lambda
support for local
transformation

36. AWS Snowball Edge
Integrated
Storage and Compute
Applications
STG214

37. Storage Ecosystem Partners

38. Backup to AWS approaches
Amazon S3
Amazon
Glacier
AWS
Direct
Connect
Internet
Amazon S3-IA
Application
servers
Cloud gateway
Local disk
Media
server
Cloud gateway
Application
servers
Backup SW cloud connector
Local disk
Media
server with cloud
connector

39. Hybrid cloud storage ecosystem
Backup
AWS Storage Gateway VTL
Direct to Amazon S3
File
Systems
Object Storage

40. Remember to complete
your evaluations!

41. Thank you!