AWS Storage Gateway is a service that connects an on-premises software appliance with AWS storage. It simplifies the adoption of cloud-based storage within on-premises environments, giving customers a secure, reliable, and cost-effective alternative to local storage. In this session, we take a detailed look at how to use Storage Gateway to backup and archive on-premises data. We discuss the three types of storage and how to select the right type for your environment. We walk through setup and configuration of the on-premises gateway appliance, data restoration, and daily management, such as monitoring performance and managing storage. The session is intended for customers who perform on-premises backup and archive today, and want to learn how to include cloud storage in their environment.

1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Paul Reed, AWS Storage Gateway
October 2015
AWS Storage Gateway Deep Dive
Simple, Secure, Cost-effective Storage in AWS
STG311

2. Agenda
• AWS Storage Gateway overview
• Demo setup and activation
• Monitoring and managing
• Where to learn more

3. AWS Storage Gateway overview

4. What is AWS Storage Gateway?
Works with your existing applications
Secure and durable storage in AWS
Low-latency for frequently used data
Scalable and cost-effective on-premises storage
Service connecting an on-premises software appliance
with cloud-based storage

5. Common uses for AWS Storage Gateway
Backup and archive
Disaster recovery
Data migration or mirroring
Replace or expand on-premises storage

6. How does AWS Storage Gateway work?
Amazon EBS
snapshots
Amazon S3
Amazon
Glacier
AWS
Storage Gateway
appliance
Application
server
AWS
Storage Gateway
backend
AWS
Direct
Connect
Internet
Customer premises

7. Amazon EBS
snapshots
Amazon S3
Amazon
Glacier
AWS
Storage Gateway
appliance
Application
server
AWS
Storage Gateway
backend
What happens when my application writes data?
iSCSI SSL
AWS Storage Gateway
stores blocks locally, and
asynchronously
compresses and securely
uploads the changed data
Application writes data
to AWS Storage
Gateway using the
iSCSI block protocol
Backend takes
compressed data and
stores in Amazon S3,
Amazon Glacier, or
Amazon EBS, with AES-
256 encryption
321
21 3
Customer premises

8. Amazon EBS
snapshots
Amazon S3
Amazon
Glacier
AWS
Storage Gateway
appliance
Application
server
AWS
Storage Gateway
backend
Customer premises
What happens when my application reads data?
iSCSI SSL
Data not in local storage is
requested from backend
Application reads data
from AWS Storage
Gateway using the
iSCSI block protocol
Backend fetches
compressed data from
Amazon S3, Amazon
Glacier, or Amazon
EBS
4
31 4
AWS Storage Gateway
receives data from backend,
decompresses, stores
locally, and responds to the
application
5
1
AWS Storage Gateway
returns requested data
from local storage
2
2 35

9. AWS Storage Gateway configurations
iSCSI block storage
Gateway-stored volumes
iSCSI virtual tape storage
Low-latency for all your data with point-in-time
backups to AWS
Replacement for on-premises physical tape
infrastructure for backup and archive
Gateway-cached volumes
Gateway-virtual tape library (VTL)
Low-latency for frequently used data with all data
stored in AWS

10. Customer data center
AWS Storage
Gateway VM
Gateway-stored volumes
• Primary data stored on-premises
• Asynchronous backup to AWS
• Point-in-time backups stored as Amazon EBS snapshots
• Up to 12 volumes, up to 16 TB each, for up to 192 TB per gateway
Amazon EBS
snapshots
Application
server
INITIATOR
TARGET
Upload
Buffer
Volume
Storage
AWS
Storage Gateway
service

11. Customer data center
AWS Storage
Gateway VM
Gateway-cached volumes
• Primary data stored in AWS
• Frequently accessed data cached on-premises
• Point-in-time backups stored as Amazon EBS snapshots
• Up to 32 volumes, up to 32 TB each, for up to 1 PB per gateway
Amazon EBS
snapshots
Application
server
INITIATOR
TARGET
Upload
Buffer
Cache
Storage
Volume storage
backed by
Amazon S3
AWS
Storage Gateway
service

12. Gateway-virtual tape library (VTL)
• Virtual tapes stored in AWS
• Frequently accessed data cached on-premises
• Up to 1,500 tapes, up to 2.5 TB each, for up to 150 TB per gateway-VTL
• Unlimited number of tapes in virtual tape shelf (VTS)
Customer data center
VTS storage
backed by
Amazon Glacier
AWS Storage
Gateway VM
Backup
Server
INITIATOR
AWS
Storage Gateway
service
MEDIA
CHANGER
Upload
Buffer
Cache
Storage
Gateway-VTL
storage backed
by Amazon S3
VTS
TAPE
DRIVE

13. Demo setup and activation

14. Monitoring and managing
AWS Storage Gateway

15. Using system resource check on your VM
• Checks virtual system resources available to the gateway
• AWS Storage Gateway won’t start if minimum resources are not
provisioned

16. Using Amazon CloudWatch to monitor performance
• Monitoring and set up alarms for your gateway
• 27 metrics reported for gateways, volumes, and tapes
• Available through AWS console, API, AWS CLI, and SDK

17. Monitoring write performance with Amazon CloudWatch
• Changed data stored in both cache and upload buffer
• Size local storage to match application and network throughput
• Writes are throttled as upload buffer gets close to full
• Overwrites collapsed to expedite upload of latest changed data
WriteBytes
WriteTime
TimeSinceLastRecoveryPoint
UploadBufferPercentUsed
CachePercentDirty
Application
server
AWS Storage Gateway
Learn more: http://docs.aws.amazon.com/storagegateway/latest/userguide/GatewayCachedLocalStorage.html

18. Monitoring read performance with Amazon CloudWatch
• Cache should hold application working set of storage
• Strive for high cache hit rate and high cache utilization
• Writes are stored in cache so can impact read performance
ReadBytes
ReadTime
CloudBytesDownloaded
CloudDownloadLatency
CacheHitPercent
CachePercentUsed
CachePercentDirty
Application
server
AWS Storage Gateway
Learn more: http://docs.aws.amazon.com/storagegateway/latest/userguide/GatewayCachedLocalStorage.html

19. Using AWS CloudTrail to monitor events
AWS Storage Gateway API calls can be logged with AWS CloudTrail

20. Getting notifications for specific events
AWS CloudTrail saves
logs to an Amazon S3
bucket
Amazon S3 bucket notification
configuration publishes event
to AWS Lambda which invokes
function
AWS CloudTrail
Amazon S3
Function
AWS Lambda
Lambda function reads log
and sends Amazon SNS
notification of specific AWS
Storage Gateway events
1
2
3
3
Amazon SNS
TopicLog Bucket
1 2
Learn more: http://docs.aws.amazon.com/lambda/latest/dg/wt-cloudtrail-events-adminuser.html
3

21. Where to learn more

22. Resources
Documentation
http://aws.amazon.com/storagegateway/
Developer Forum
http://forums.aws.amazon.com/
Developer Support
https://aws.amazon.com/premiumsupport/
Try it!
AWS Free Tier
https://aws.amazon.com/free/

23. Remember to complete
your evaluations!

24. Thank you!