This session will begin with an introduction to non-relational (NoSQL) databases and compare them with relational (SQL) databases. We will also explain the fundamentals of Amazon DynamoDB, a fully managed NoSQL database service. Learn the fundamentals of DynamoDB and see the new DynamoDB console first-hand as we discuss common use cases and benefits of this high-performance key-value and JSON document store.
2. Agenda
• Brief history of data processing
• Relational (SQL) vs. nonrelational (NoSQL)
• Fully managed features of DynamoDB
• Customer use cases
• Demo – serverless applications
• Pricing and Free Tier
3. Data volume since 2010
• 90% of stored data generated in
last 2 years
• 1 terabyte of data in 2010 equals
6.5 petabytes today
• Linear correlation between data
pressure and technical innovation
• No reason these trends will not
continue over time
7. Relational vs. nonrelational databases
Traditional SQL NoSQL
DB
Primary Secondary
Scale up
DB
DB
DBDB
DB DB
Scale out
8. SQL vs. NoSQL schema design
NoSQL design optimizes for
compute instead of storage
9. Why NoSQL?
Optimized for storage Optimized for compute
Normalized/relational Denormalized/hierarchical
Ad hoc queries Instantiated views
Scale vertically Scale horizontally
Good for OLAP Built for OLTP at scale
SQL NoSQL
15. WRITES
Replicated continuously to 3 AZs
Persisted to disk (custom SSD)
READS
Strongly or eventually consistent
No latency trade-off
Designed to
support 99.99%
of availability
Built for high
durability
High availability and durability
18. MLBAM (MLB Advanced Media) is a full service solutions
provider, operating a powerful content delivery platform.
For the first time, we can
measure things we’ve never
been able to measure
before.
Joe Inzerillo
Executive Vice President and CTO, MLBAM
”
“ • MLBAM can scale to support many games on a
single day.
• Amazon DynamoDB powers queries and supports the
fast data retrieval required.
• MLBAM distributes 25,000 live events annually and
10 million streams daily.
Major League Baseball Fields Big Data,
Excitement with Amazon DynamoDB
19. Redfin is a full-service real estate company with local
agents and online tools to help people buy & sell homes.
We have billions of records
on DynamoDB being
refreshed daily or hourly or
even by seconds.
Yong Huang
Director, Big Data Analytics, Redfin
”
“ • Redfin provides property and agent details and
ratings through its websites and apps.
• With DynamoDB, latency for “similar” properties
improved from 2 seconds to just 12 milliseconds.
• Redfin stores and processes five billion items in
DynamoDB.
Redfin Is Revolutionizing Home Buying and
Selling with Amazon DynamoDB
20. Expedia is a leader in the $1 trillion travel industry, with an
extensive portfolio that includes some of the world’s most
trusted travel brands.
With DynamoDB we were up
and running in a less than
day, and there is no need for
a team to maintain.
Kuldeep Chowhan
Principal Engineer, Expedia
”
“ • Expedia’s real-time analytics application collects data
for its “test & learn” experiments on Expedia sites.
• The analytics application processes ~200 million
messages daily.
• Ease of setup, monitoring, and scaling were key
factors in choosing Amazon DynamoDB.
Expedia’s Real-time Analytics Application Uses
Amazon DynamoDB
21. Nexon is a leading South Korean video game developer
and a pioneer in the world of interactive entertainment.
By using AWS, we
decreased our initial
investment costs, and only
pay for what we use.
Chunghoon Ryu
Department Manager, Nexon
”
“ • Nexon used Amazon DynamoDB as its
primary game database for a new blockbuster
mobile game, HIT
• HIT became the #1 Mobile Game in Korea
within the first day of launch and has > 2M
registered users
• Nexon’s HIT leverages DynamoDB to deliver
steady latency of less than 10ms to deliver a
fantastic mobile gaming experience for
170,000 concurrent players
Nexon Scales Mobile Gaming with Amazon
DynamoDB
22. Ad Tech Gaming MobileIoT Web
Scaling high-velocity use cases with DynamoDB
27. Advanced topics in DynamoDB
• Data modeling
• Indexes - query a table using alternate attributes
• Local Secondary Indexes
• Global Secondary Indexes
• Understanding Partitions
• # of partitions depend on table throughput and size
• Design patterns and best practices
28. To learn more, please attend:
Deep Dive on Amazon DynamoDB
4:45 p.m.– 5:45 p.m.
Rick Houlihan, principal solutions architect
37. • Free Tier
25GB of storage
25 Reads per second
25 Writes per second
• Pricing for additional usage in US East (N. Virginia)
$0.25 per GB per month
Write throughput: $0.0065 per hour for every 10 units of Write Capacity
Read throughput: $0.0065 per hour for every 50 units of Read Capacity
DynamoDB Pricing & Free Tier
42. The Climate Corporation (TCC) scales with Amazon DynamoDB
The Climate Corporation is a San Francisco-based
company that examines weather data to help farmers
optimize their decision-making.
The elasticity of DynamoDB
read/write ops made
DynamoDB the fastest and
most efficient solution to
achieve our high ingest rate.
Mohamed Ahmed
Director of Engineering,
Site Reliability Engineering and Data Analytics
The Climate Corporation
”
“ • Climate is digitizing agriculture, helping
farmers increase their yields and productivity
using scientific and mathematical models on
top of massive amounts of data.
• Weather and satellite imagery is one large
source of data used in TCC’s calculations.
• TCC uses DynamoDB to ingest a burst of
data and satellite images retrieved from third
parties before processing them.
• TCC goes from few read/write operations to
thousands each day to keep up with the
bursts of data written and read from it main
DynamoDB tables.
43. Gaming use case
Nexon delivers unparalleled mobile gaming with DynamoDB
Nexon is a leading South Korean video game developer
and a pioneer in the world of interactive entertainment.
By using AWS, we
decreased our initial
investment costs, and only
pay for what we use.
Chunghoon Ryu
Department Manager, Nexon
”
“ • Nexon used DynamoDB as its primary
game database for a new blockbuster
mobile game, HIT.
• HIT became the #1 mobile game in
Korea within the first day of launch and
has more than 2 million registered
users.
• Nexon’s HIT leverages DynamoDB to
deliver steady latency of less than 10
milliseconds to deliver a fantastic mobile
gaming experience for 170,000
concurrent players.
44. Analytics use case
Expedia’s real-time analytics application uses DynamoDB
Expedia is a leader in the $1 trillion travel industry, with an
extensive portfolio that includes some of the world’s most
trusted travel brands.
With DynamoDB, we were up
and running in a less than
day, and there is no need for
a team to maintain.
Kuldeep Chowhan
Engineering Manager, Expedia
”
“ • Expedia’s real-time analytics
application collects data for its “test
and learn” experiments on Expedia
sites.
• The analytics application processes
~200 million messages daily.
• Ease of setup, monitoring, and
scaling were key factors in
choosing DynamoDB.
45. MediaTek delivers IoT databases to developers with DynamoDB
MediaTek designs and develops silicon wafers for
wireless communications and digital multimedia solutions.
We have been able to have
scalability, consistency, and
availability, which is really
helpful with our customers and
our internal stakeholders.
Marc Naddell
Vice President, MediaTek Labs
”
“ • MediaTek LinkIt One developers
can access their database by using
mobile devices or a desktop
browser.
• MediaTek uses DynamoDB as part
of its highly scalable, consistent,
and available cloud solution.
• MediaTek has expanded globally
and reduced development time by
50 percent with AWS.
46. Mapbox leverages DynamoDB for fast global map delivery
Mapbox provides building blocks that make it easy to
integrate location into any mobile or online application.
Our customers want very fast
maps. DynamoDB saves a
lot of time in the back end.
Ian Ward
Software Engineer, Mapbox
”
“ • Mapbox maps reach over 200
million users worldwide.
• With DynamoDB, Amazon
ElastiCache, and Amazon S3,
Mapbox cut end-user latency in
half.
• Mapbox found AWS was 85 percent
less expensive than alternatives.
• DynamoDB integrates with AWS
CloudTrail and AWS Identity and
Access Management (IAM).
47. Duolingo scales to store over 4 billion items using AWS
Duolingo is a free language learning service where
users help translate the web and rate translations.
Using AWS, we can handle
traffic spikes that expand up
to seven times the amount of
normal traffic.
Severin Hacker
CTO, Duolingo
”
“ • Duolingo stores data about each user to
be able to generate personalized
lessons.
• The MySQL database couldn’t keep up
with Duolingo’s rate of growth.
• By using the scalable database service,
data store capacity increased from 100
million to more than 4 billion items.
• Duolingo has the capacity to scale to
support over 8 million active users.
48. Peak innovates and scales quickly with DynamoDB
Peak is a London-based startup is focused on improving
cognitive skills for its millions of users each month.
The ability to grow from two
to four services or increase
read/write capacity of a
database without spending
more than five minutes on it
from an IT management
perspective is amazing.
Bertrand Lamarque
Director of Engineering, Peak
”
“ • Peak designs cognitive training
games.
• DynamoDB provides fast access to
data and massive scalability.
• Moving faster with fewer resources
is a significant advantage for Peak.
• DynamoDB enables the product
agility needed to keep enhancing
its product.
49. Tigerspike reduces latency, improves uptime with DynamoDB
Tigerspike is one of the foremost providers of personal
media technology consulting.
Since moving to AWS, we’ve
seen latency in some instances
drop by up to 300 milliseconds
and we've achieved 99.95
percent uptime.
Dean Jezard
Chief Technology Officer, Tigerspike
”
“ • Tigerspike won the Deloitte Technology
Fast 50 Australia award for 7
consecutive years.
• An alternative provider would cost up to
75 percent more than AWS.
• The AWS architecture supports
Tigerspike’s existing operating system
and language platforms.
50. DoApp minimizes costs, achieves scale with Amazon DynamoDB
DoApp is a mobile and web app development company
whose apps can be customized for organizations.
AWS provides Fortune 500
infrastructure, reliability, and
scalability at a reasonable
cost to a company with just
12 people.
Ryan Pendergast
Software Developer, DoApp
”
“ • DoApp provides more than 460 mobile
apps.
• Its most popular product is a news app
that can be customized for clients,
including Gannett.
• DoApp operates its own advertising
network to customers around the world.
• Amazon DynamoDB enables high
transaction rates at scale for its
advertising platform.
51. Myriad Group accelerates growth with DynamoDB
Myriad Group is a French-Swiss software company that
builds and markets software for mobile operators.
DynamoDB has allowed us
to grow in a way perhaps we
wouldn’t have been able to
before.
Bruce Jackson
Chief Technology Officer, Myriad Group
”
“ • Myriad Group develops software for
mobile operators.
• Myriad also offers a direct-to-consumer
social content sharing service called
Versy.
• Myriad grew its Versy user base from
38 million users to 170 million users in
about one year.
• Amazon DynamoDB allows Myriad to
focus on the business problem and not
worry about infrastructure.
52. Beatpacking chose DynamoDB over Cassandra and Hbase
The Beatpacking Company is a Seoul-based startup that
provides a free streaming radio service.
I think DynamoDB was the
best choice since we can
provision appropriate
throughput and control
pricing.
Minyoung Jeong
Chief Technology Officer,
The Beatpacking Company
”
“ • Beatpacking provides a free streaming
radio service called Beat that focuses
on Korean K-pop music.
• The Beat service has grown to over 2
million users within 7 months of its
launch.
• Beatpacking typically stores 10,000 to
20,000 events per second, and peaks of
100,000 events per second.
• DynamoDB allows Beatpacking to
easily provision throughput capacity
while the service is running.
53. JustGiving scales event storage for viral giving campaigns with DynamoDB
London-based JustGiving is one of the world’s largest
online social platforms for charitable fundraising.
When you look at concepts
such as #NoMakeupSelfie or
#IceBucketChallenge …
they’re extremely viral. …
To size up for those spikes
would have been impractical.
Richard Atkinson
Chief Information Officer, JustGiving
”
“ • JustGiving is one of the world’s largest
online social platforms for charitable
fundraising.
• The London-based organization’s 24
million users have helped raise $3.5
billion for over 13,000 causes.
• JustGiving uses DynamoDB to store
website clickstream events for its big-
data analytics platform.
• JustGiving can make near–real-time
improvements to its product based on
billions of annual site visits.
54. jobandtalent uses DynamoDB to power its new microservice
Europe-based jobandtalent is a job-matching platform that
uses proprietary algorithms to connect candidates with
openings.
We recently started to use
Amazon DynamoDB with one
of our microservices.
Teo Ruiz
Infrastructure and Platform Architect, jobandtalent
”
“ • More than 5 million users depend on
jobandtalent to connect skills with job
openings.
• jobandtalent uses a real-time
recommendation engine to match 4
million openings and 5 million
candidates.
• With DynamoDB, jobandtalent can
scale to 60,000 requests per minute
during peak traffic periods.
55. Infraware uses DynamoDB to improve Polaris Office for its 10M users
Infraware is a software development company that makes
Polaris Office for desktops and mobile devices.
We use Amazon RDS,
Amazon DynamoDB for
database, and Amazon EMR,
and Amazon Redshift for
data analysis.
Seongtaek Kim
Team Leader, Infraware
”
“ • Infraware delivers the Polaris Office
mobile app to more than 10 million
global users.
• Infraware saves time building
infrastructure by taking advantage of
AWS.
• Infraware now delivers much faster
service.
• Polaris Office is improving with events
stored in DynamoDB.
56. Unalis uses DynamoDB to store game events for its analytics platform
Headquartered in Taiwan, Unalis provides mobile gaming
content, apps, and gaming analytics for developers.
By using our UniCloud analytics
platform, developers can easily
obtain and understand campaign
tracking information as well as
consumption patterns, game
analytics, and product data.
Simpson Chou
Director, Game Development Department, Unalis
”
“ • Unalis provides mobile gaming content,
apps, and gaming analytics for
developers through UniCloud.
• UniCloud captures about 200
megabytes of raw data from users in
different countries daily.
• Tens of thousands of players are online
at peak periods.
• Unalis uses DynamoDB to store large
volumes of gaming data for deeper
analytics.
57. Canary scaled with DynamoDB to process 150M+ videos per day
Canary is a fast-growing New York startup that makes the
Canary in-home security system.
People have checked into our
app from 185 countries and
they’re using our devices in 140
countries, so there’s an
incredible amount of demand
for our services. We can scale
to meet that demand by
running on the AWS cloud.
Chris Rill
Cofounder and Chief Technology Officer, Canary
”
“ • The Canary in-home security system
consists of a wireless device about the
size of a tall coffee mug.
• The Canary system sends a notification
along with video directly to its owner’s
smartphone through an app.
• DynamoDB lets Canary handle 150+
million videos every day.
• Canary can innovate and execute much
faster with AWS.