1.
Library Workloads on Amazon
Web Services
May 27th, 2015
Sri Elaprolu – Manager, Solutions Architecture
Worldwide Public Sector

2.
AWS Overview
Library Workloads and Use Cases
Getting Data into AWS
Storage Services
Transcoding Service
Content Delivery Service
Q & A
Agenda

3.
What is AWS?

4.
Application Services
Compute Storage Databases
Networking
AWS Global Infrastructure
Deployment & Administration
Amazon Web Services

5.
11 Regions
29 Availability Zones
53 Edge locations
AWS Global Infrastructure
Customer Decides Where Applications and Data Reside

6.
AWS Availability Zone (AZ) View
- Multiple Isolated locations within a Region
- Availability Zone = 1 or more “data center”
- Independent Failure Zone
- Physically separated
- On separate Low Risk Flood Plains
- Discrete UPS
- Onsite backup generation facilities
- Fed from different segments of utility provider
- Redundantly connected to multiple tier-1 ISP’s
- No “Disaster Recovery Datacenter”
- Built for Continuous Availability
- Customer decides Availability Zone for Compute
Availability
Zone A
Availability
Zone B
Availability
Zone C
Sample US Region
~ Data Center

7.
Architected for Government Security Requirements
http://aws.amazon.com/security
Certifications and accreditations for
workloads that matter
AWS CloudTrail and AWS Config - Call
logging and configuration management
for governance & compliance
• Log, review, alarm
on all user actions
• Browse and query
database of current
and previous state
of cloud resources

8.
Experience
Since 2006 supporting
large numbers of
customers across 190
countries
Innovation
Rapid delivery of new
services and features based
on customer feedback
Robust Platform
Number of services and
features, virtually to
support every use case
imaginable
Simple Pricing
Philosophy
48 Price reductions
Expect more reductions
in the future
Global Footprint
11 Regions
29 Availability Zones
53 Edge Locations
Eco system
Thousands of partners
(ISV, SI, consulting)
23 categories; 2100 apps
in Marketplace
AWS Differentiators

9.
Library Workloads

10.
Library Use Cases
Online Public
Access Catalogs
Library Catalogs
Online databases
Institutional
Repositories
Online Archive
Intellectual Output
Digital Asset
Storage
Protect from Loss and
Degradation
Offsite Storage
Redundancy and
Durability
Backups
Offsite
Redundant
Development Space
Disposable Environments
Start and Stop Frequently

11.
Dspace
Open Journal Systems
Open Conference Systems
Thesis and Dissertation Systems
Web Properties – WordPress
DuraCloud Preservation System
• Consortium of higher education institutions in
Texas that has provided shared digital library
services since 2005
• The mission of the Texas Digital Library (TDL) is
to enable each of its member libraries to advance
a program of digital initiatives in support of
research, scholarship, and learning.

12.
Getting Data into AWS

13.
Data Ingestion Options
AWS Direct Connect
Dedicated bandwidth between
your site and AWS
Internet
Transfer data in a secure SSL tunnel over the
public Internet
AWS Import/Export
Physical transfer of media into and
out of AWS

14.
AWS Ingestion Options - Internet
1. Multipart upload
2. Request rate optimization
3. TCP window scaling
4. TCP selective
acknowledgement
AWS has customers that ingest roughly 1 PB per day

15.
AWS Ingestion Options - AWS Direct Connect
• Private connectivity to AWS
– Physical connection – 1 Gbps or 10 Gbps port
• Consistent network performance
• Consider burst models on ingest
• Reduces costs for bandwidth-heavy
outbound workloads
• US Locations
• CoreSite 32 Avenue of the Americas, NY
• CoreSite One Wilshire & 900 North Alameda, LA
• Equinix DC1 – DC6 & DC10 - DC11, Ashburn, VA
• Equinix SV1 & SV5, San Jose, CA
• Equinix SE2 & SE3, Seattle, WA

16.
AWS Ingestion Options - AWS Import/Export
• Rapidly move data into
and out of AWS
• Portable storage device
shipment to AWS
• Supports
– Amazon EBS
– Amazon S3
– Amazon Glacier
• Use cases
– Initial data migration
– Content distribution via portable
devices
– Disaster recovery

17.
Amazon Storage Services

18.
Amazon Simple Storage Service (S3)
Highly scalable object storage
1 byte to 5 TB in size
99.999999999% durability
Amazon Elastic Block Store (EBS)
High-performance block storage device
1 GB to 16 TB in size
Mount as drives to instances with
snapshot/cloning functionalities
Magnetic and General Purpose SSD
Amazon Glacier
Long-term object archive
Extremely low cost per gigabyte
99.999999999% durability
AWS Storage and Archive Options

19.
Amazon Elastic Block Store (EBS)
• High I/O block storage for Amazon
EC2
• Point-in-time snapshots to Amazon S3
• 99.999999999% Durability
• Snapshot software is FREE
• Point-in-time snapshots across
regions

20.
Amazon Simple Storage Service (S3)
• Durable and low cost
• Unlimited number of objects and volume
• Back up to Amazon S3 buckets via
HTTP/HTTPS
– Create scripts using PowerShell,
Perl, Python…
– Numerous solutions for data backup
• Authentication mechanisms ensure data
is kept secure
• Reduced redundancy storage (RRS)
option

21.
Amazon Glacier
• $0.01 per GB/mo, $120 per TB/yr
• 3-5 hour data retrieval latency
• Archives: single file or zipped files
• Vaults: collection of archives
• Infinite archival storage
• 99.999999999% durability
• Immutable, encrypted by default

22.
Object Life Cycle Management Amazon S3 → Amazon Glacier
• Seamlessly move data from Amazon S3 → Amazon Glacier
• 3-5 hour asynchronous retrieval
• Data lifecycle policies
• $0.01 per GB for Amazon Glacier costs

23.
Why AWS for Storage and Archiving?
• Protect digital content from fragility
• Protect digital assets from loss and degradation
• Promote learning
• Share research

24.
TCO: On-Premise Cost Considerations
1. Primary storage hardware (primary / remote site)
2. Storage growth (cost of upgrades)
3. Storage management software and 3rd party tools
4. Professional services
5. Hardware maintenance
6. Software maintenance
7. Backup software
8. Backup hardware (primary / remote site)
9. Offsite tape storage / vault
10. Archive software
11. Archive hardware
12. Power
13. Cooling
14. Space
15. Labor
16. Cost of capital
17. Training
18. Asset depreciation
19. Migration
20. Decommission / remove
21. Recycle
22. …

25.
Storage on AWS
10 TB S3 = $ 3,631.20 per YEAR
5 TB S3 | 5 TB Glacier = $ 2,433.12 per YEAR
10 TB Glacier = $ 1,228.80 per YEAR
Price based on US-EAST-1 region; correct as of May 22nd, 2015

26.
Amazon Elastic
Transcoder

27.
• Managed Transcoding Service built on EC2
– No software to buy or manage
– No need to manage capacity
– Seamless integration with AWS S3 and Amazon CloudFront CDN
• Transcode Content for Any Device
– Select from over 30 transcode presets
– Define up to 50 presets per AWS Account!
• Process jobs in parallel and on demand
Amazon Elastic Transcoder: Overview

28.
• Self-Service Control
• Un-Matched On Demand Capacity
• Industry Leading Reliability
• Lowest Cost Transcoding Service
• Highly Secure
• Global Availability
• Rapidly Releasing New Features
Amazon Elastic Transcoder: Why Customers prefer the service

29.
Pipeline 1 Pipeline 2
Input Bucket 1 Input Bucket 2
Output Bucket
Pipelines, jobs, & outputs
ALL run in parallel
:
:
:
:
:
:
:
Job N (Progressing)
Job N+1 (Complete)
Job N+2 (Progressing)
:
:
:
:
:
:
:
:
:
:
Job M (Progressing)
Job M+1 (Progressing)
Job N+3 (Progressing)
Job N+4 (Submitted)
SNS Topic
Amazon Elastic Transcoder: How it works?

30.
Amazon Content Delivery
Service

31.
• Full Feature Caching Network
• Global Infrastructure
• Tuned for Optimal Performance
• Massively Scalable
• Highly Secure
• Robust Analytics
• Self Service
• Priced to Minimize Cost
Amazon CloudFront: Content Delivery Network

32.
• Media and Entertainment
• Gaming
• Digital Catalogs
• Digital Advertising
• Software Downloads
• Dynamic Websites and Applications
Amazon CloudFront: For Any Market Segment

33.
Video Streaming
On-demand & Live Streaming
RTMP (Flash) and HTTP(S)
Adaptive Bitrate Live Streaming
Microsoft Smooth Streaming
Whole Site Delivery
Static & Dynamic Content
Mobile Detect, CORS Support
Multiple Cache Behaviors
Multiple Origin Servers
Security
Private Content (Signed URLs)
Custom SSL (Dedicated IP & SNI)
Geo Restriction
HTTP to HTTPS Redirect
High Availability
99.9% SLA
Automatic Origin Failover
Custom Error Pages
Serve Stale Content when Origin
unavailable
High Performance
Latency Based Routing
TCP Optimization
Persistent Connections
EDNS Client Subnet
Low TCO
Pay for use
Commit-Based lower pricing
Price Classes
Preferential Pricing for AWS origins
Amazon CloudFront: Popular Features

34.
Dynamic
Static
Video
User
Input
SSL
Amazon CloudFront: Deliver all your site

35.
Thank You!