Amazon Aurora is a MySQL- and PostgreSQL-compatible relational database with the speed, reliability, and availability of commercial databases at one-tenth the cost. This session provides an overview of Aurora, explores recently announced features, such as Serverless, Multi-Master, and Performance Insights, and helps you get started.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s New in Amazon Aurora
Debanjan Saha
GM, Amazon Aurora & RDS
Amazon Web Services
D A T 2 0 4

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Relational Database Service (Amazon RDS)
Choice Value Innovation

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon RDS
Choice of open source and commercial databases
RDS Platform
Open Source Engines Commercial Engines
Advanced monitoring
Routine maintenance
Push-button scaling
Automatic fail-over
Backup & recovery
X-region replication
Isolation & security
Industry compliance
Automated patching
Cloud Native Engine

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Aurora…
Enterprise database at open source price
Delivered as a managed service
Amazon Aurora
Speed and availability of high-end commercial databases
Simplicity and cost-effectiveness of open source databases
Drop-in compatibility with MySQL and PostgreSQL
Simple pay as you go pricing

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Aurora innovations
Re-imagining databases for the cloud
Automate administrative tasks – fully managed service
Scale-out, distributed, multi-tenant design
Service-oriented architecture leveraging AWS services

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scale-out, distributed architecture
Purpose-built log-structured distributed
storage system designed for databases
Storage volume is striped across
hundreds of storage nodes distributed
over 3 different availability zones
Six copies of data, two copies
in each availability zone to protect
against AZ+1 failures
Plan to apply same principles
to other layers of the stack
Shared storage volume
Storage nodes with SSDs
Availability
Zone 1
SQL
Transactions
Caching
Availability
Zone 2
SQL
Transactions
Caching
Availability
Zone 3
SQL
Transactions
Caching

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Leveraging AWS services
Invoke AWS Lambda events
from stored
procedures/triggers
Load data from Amazon
Simple Storage Service
(Amazon S3), store snapshots
and backups in S3
Lambda
function
Amazon
S3
AWS Identity
and Access
Management
Amazon
CloudWatch
Use AWS Identity and Access
Management (IAM) roles to
manage database access
control
Upload systems metrics
and audit logs to CloudWatch

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Automate administrative tasks
Automatic fail-over
Backup & recovery
Isolation & security
Industry compliance
Push-button scaling
Automated patching
Advanced monitoring
Routine maintenance
Takes care of your time-consuming database management
tasks, freeing you to focus on your applications and business
You AWS
Schema design
Query construction
Query optimization

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aurora customer adoption
Fastest growing service in AWS history
Aurora is used by ¾ of the top 100 AWS customers

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Who are moving to Aurora and why?
Customers using
open source engines
• Higher performance – up to 5x
• Better availability and durability
• Reduces cost – up to 60%
• Easy migration; no
application change
Customers using
commercial engines
• One tenth of the cost; no licenses
• Integration with cloud ecosystem
• Comparable performance and
availability
• Migration tooling and services

12. Aurora performance
5x faster than MySQL; 3x faster than PostgreSQL

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Write and read throughput
Aurora MySQL is 5x faster than MySQL
0
50,000
100,000
150,000
200,000
250,000
Series1 Series2 Series3 Series4 Series5
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
800,000
Series1 Series2 Series3 Series4 Series5
Write Throughput Read Throughput
Using Sysbench with 250 tables and 200,000 rows per table on R4.16XL

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Series1
Series1
Series2
Series2
Series3
Series3
0 500 1000 1500 2000 2500 3000 3500 4000
1
2
Runtime (seconds)
pgbench initialization, scale 10000 (150 GiB)
86% reduction in vacuum time
Bulk data load performance
Aurora PostgreSQL loads data 2x faster than PostgreSQL

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Bulk data load performance
Aurora MySQL loads data 2.5x faster than MySQL
Series1
Series1
Series2
Series2
0 100 200 300 400 500 600 700 800
1
2
Runtime (sec.)
10 Sysbench Tables, 10MM rows per each

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
0.00
100.00
200.00
300.00
400.00
500.00
600.00
700.00
1 61 121 181 241 301 361 421 481 541 601 661 721 781 841 901 961 1,021 1,081 1,141 1,201
Responsetime,ms
SYSBENCH RESPONSE TIME (p95), 30 GiB, 1024 CLIENTS
Series1 Series2
Performance variability under load
Aurora PostgreSQL is ~10x more consistent than PostgreSQL

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Sysbench write-only workload with 250 tables and 200,000 initial rows per table
0
500
1,000
1,500
2,000
2,500
0 100 200 300 400 500 600
Time from start of run (sec.)
Write Response Time (ms.) Series1 Series2
Performance variability under load
Aurora MySQL is ~25x more consistent than MySQL

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
0
50
100
150
200
250
1 2 3 4
Max write throughput – up 100%
0
100
200
300
400
500
600
700
800
1 2 3 4
Max read throughput – up 42%
Launched with R3.8xl
32 cores, 256GB memory
Now support R4.16xl
64 cores, 512GB memory
R5.24xl coming soon
96 cores, 768GB memory
Besides many performance optimizations, we are also upgrading HW platform
Performance improvement over time
Aurora MySQL – 2015-2018

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How did we achieve this?
Do less work
• Do fewer IOs
• Minimize network packets
• Cache prior results
• Offload the database engine
Be more efficient
• Process asynchronously
• Reduce latency path
• Use lock-free data structures
• Batch operations together
• Databases are all about I/O
• Network-attached storage is all about packets/second
• High-throughput processing is all about context switches

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aurora I/O profile
MySQL with Replica Amazon Aurora
EBS mirrorEBS mirror
AZ 1 AZ 2
EBS
Amazon Elastic
Block Store (EBS)
Primary
Instance
Replica
Instance
1
2
3
4
5
Amazon
S3
MySQL I/O profile for 30 min Sysbench run
780K transactions
7,388K I/Os per million txns (excludes mirroring, standby)
Average 7.4 I/Os per transaction
AZ 1 AZ 3
Primary
Instance
AZ 2
Replica
Instance
ASYNC 4/6 QUORUM
Distributed writes
Replica
Instance
Amazon
S3
Aurora IO profile for 30 min Sysbench run
27,378K transactions – 35X MORE
0.95 I/Os per transaction (6X amplification) – 7.7X LESS
Binlog Data Double-writeLog From files

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aurora lock management
MySQL lock manager Aurora lock manager
Scan
Delete
Insert
Scan
Scan
Delete
Scan
Insert
Insert
Delete
Insert
Scan
Same locking semantics as MySQL
Concurrent access to lock chains
Multiple scanners in individual lock chains
Lock-free deadlock detection
Needed to support many concurrent sessions, high update throughput

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Asynchronous
key prefetch
 Works for queries using Batched
Key Access (BKA) join algorithm
+ Multi-Range Read (MRR)
Optimization
 Performs a secondary to primary
index lookup during JOIN
evaluation
 Uses background thread
to asynchronously load pages
into memory
Latency improvement factor vs. Batched Key Access (BKA)
join algorithm. Decision support benchmark, R3.8xlarge
14.57
-
2.00
4.00
6.00
8.00
10.00
12.00
14.00
16.00
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Series1
AKP used in queries 2, 5, 8, 9, 11, 13, 17, 18, 19, 20, 21

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Batched scans
Standard MySQL pprocesses rows one-at-
a-time resulting in high overhead due to
• Repeated function calls
• Locking and latching
• Cursor store and restore
• InnoDB to MySQL format conversion
Amazon Aurora scan tuples from
the InnoDB buffer pool in batches for
• Table full scans
• Index full scans
• Index range scans Latency improvement factor vs. Batched Key Access (BKA)
join algorithm. Decision support benchmark, R3.8xlarge
1.78X
-
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1.80
2.00
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Parallel query processing
Aurora storage has thousands of CPUs
• Opportunity to push down and parallelize query
processing
• Moving processing close to data reduces
network traffic and latency
However, there are significant challenges
• Data is not range partitioned – require full scans
• Data may be in-flight
• Read views may not allow viewing most recent
data
• Not all functions can be pushed down
Database Node
Storage nodes
Push down
predicates
Aggregate
results

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Well-known decision support benchmark
0x
20x
40x
60x
80x
100x
120x
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Query response time reduction
 Peak speed up ~120x
 >10x speedup: 8 of 22 queries
We were able to test Aurora’s parallel query feature and the performance gains
were very good. To be specific, We were able to reduce the instance type from
r3.8xlarge to r3.2xlarge. For this use-case, parallel query was a great win for us.
Jyoti Shandil, Cloud Data Architect
Performance results

26. What about availability
“Performance only matters if your database is up”

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
6-way replicated storage
Survives catastrophic failures
• Six copies across three
availability zones
• 4 out 6 write quorum;
3 out of 6 read quorum
• Peer-to-peer replication
for repairs
• Volume striped across
hundreds of storage nodes
SQL
Transaction
AZ 1 AZ 2 AZ 3
Caching
SQL
Transaction
AZ 1 AZ 2 AZ 3
Caching

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Read replica and custom end-point
Master
Read
replica
Read
replica
Read
replica
Shared distributed storage volume
Reader end-point #1 Reader end-point #2
Up to 15 promotable read replicas across multiple availability zones
Re-do log based replication leads to low replica lag – typically < 10ms
Custom reader end-point with configurable failover order

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Instant crash recovery
Traditional database
Have to replay logs since
the last checkpoint
Typically 5 minutes between checkpoints
Single-threaded in MySQL; requires a large
number of disk accesses
Amazon Aurora
Underlying storage replays redo records
on demand as part of a disk read
Parallel, distributed, asynchronous
No replay for startup
Checkpointed Data Redo Log
Crash at T0 requires
a re-application of the
SQL in the redo log since
last checkpoint
T0 T0
Crash at T0 will result in redo logs being
applied to each segment on demand, in
parallel, asynchronously

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Continuous availability with multi-master
Master Master Master Master
Shared distributed storage volume
Application #1 Application #2
Up to 15 promotable read replicas across multiple availability zones
Re-do log based replication leads to low replica lag – typically < 10ms
Custom reader end-point with configurable failover order

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scaling write workload
Optimistic conflict management
There are many “oases” of
consistency in Aurora
Database nodes know transaction
orders from that node
Storage nodes know transactions
orders applied at that node
Only have conflicts when data
changed at both multiple database
nodes AND multiple storage nodes
Much less coordination required
Near linear throughput scaling for
workloads with no or low conflict
Lower commit latency for workloads
with low conflict

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Transactions with no conflict
 Transactions T1 and T2 from BLUE and
ORANGE masters update different
tables (pages) Table 1 and Table 2.
 No logical or physical conflicts – no
coordination required
 Near linear scaling with number of
masters for this type of sharded or
partitioned workloads.
1 1 1 1 11 2 2 2 2 22
PAGE1 PAGE2
MASTER
MASTER
Begin Trx (T1)
Update (Table1)
Commit (T1)
Begin Trx (T2)
Update (Table2)
Commit (T2)
TABLE 1 TABLE 2

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Physical conflict resolution
Arbitration at storage node
1 1 1 2 21
PAGE1 PAGE2
MASTER
MASTER
Begin Trx (T1)
Update (Table1)
Commit (T1)
Begin Trx (T2)
Update (Table1)
Rollback (T2)
TABLE 1 TABLE 2
 Transactions T1 and T2 from BLUE and
ORANGE masters update the same
table (page) Table 1.
 Transaction T1 from BLUE master
achieves quorum – transaction T1 is
committed.
 Transaction T2 from ORANGE master
fails to achieves quorum – transaction
T2 is rolled back.
 Storage nodes act as arbitrator for
conflict resolution
X

34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Logical conflict resolution
Arbitration by designated tiebreaker
1 1 1 2 21 1 2 2 2 21
PAGE1 PAGE2
MASTER MASTER
Begin Trx (T1)
Update (Table1)
Update (Table 2)
Commit (T1)
Begin Trx (T2)
Update (Table2)
Update (Table1)
Rollback (T2)
TABLE 1 TABLE 2
 Transactions T1 and T2 from BLUE and
ORANGE masters update both tables
(pages) Table 1 (page 1)and Table 2
(page 2).
 Transaction T1 from BLUE master
achieves quorum on page 1.
Transaction T2 from ORANGE master
achieves quorum on page 2.
 Designated tiebreaker node resolves
conflict in favor of BLUE master – T2 is
rolled back; T1 is committed.
TIE BREAKER
X

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aurora Multi-Master – scaling and availability
0
10000
20000
30000
40000
50000
60000
1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31
33
35
37
39
41
43
45
47
49
51
53
55
57
59
61
63
65
67
69
71
73
75
77
79
81
83
85
87
89
91
93
95
97
99
101
103
105
107
109
111
113
115
117
119
121
123
125
AggregatedThroughput
Time in minutes
Sysbench workload on 4 R4.XL nodes
Adding a node Adding a node Node going down Node recovering

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Database backtrack
t0 t1 t2
t0 t1
t2
t3 t4
t3
t4
Rewind to t1
Rewind to t3
Invisible Invisible
Backtrack brings the database to a point in time without requiring restore from backups
• Backtracking from an unintentional DML or DDL operation
• Backtrack is not destructive. You can backtrack multiple times to find the right point in time
• Also useful for QA (rewind your DB between test runs)

37. Amazon Aurora is easy to use
Automated storage management, security and compliance,
advanced monitoring, database migration

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Performance Insights
Dashboard showing database load
 Easy – e.g. drag and drop
 Powerful – drill down using zoom in
Identifies source of bottlenecks
 Sort by top SQL
 Slice by host, user, wait events
Adjustable time frame
 Hour, day, week , month
 Up to 2 years of data; 7 days free
Max vCPU
CPU bottleneck
SQL w/ high CPU

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Alexa + Performance Insights
Integration between Alexa &
Performance Insights built by AWS
partner Slalom
Uses the Performance Insights API to
identify bottlenecks in Amazon RDS
Get actionable suggestions such as
on-demand scaling, DBA
notifications, and paging
Go to the Slalom booth (#1438) to
see a live demo and learn more

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Alexa + Performance Insights
Demo

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Security management
Encryption to secure data at rest using
customer managed keys
• AES-256; hardware accelerated
• All blocks on disk and in Amazon S3 are encrypted
• Key management via AWS KMS
Encrypted cross-region replication, snapshot
copy—SSL to secure data in transit
Advanced auditing and logging without
any performance impact
Database activity monitoring
Database
Engine
*NEW*
Customer Master Key(s)
Data key 1
Storage
node
Storage
node
Storage
node
Storage
node
Data key 1 Data key 1 Data key 1

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Industry certifications
Amazon Aurora gives each database
instance IP firewall protection
Aurora offers transparent
encryption at rest and SSL
protection for data in transit
Amazon VPC lets you isolate and
control network configuration and
connect securely to your IT
infrastructure
AWS Identity and Access
Management provides resource-
level permission controls *New* *New* *New*

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CloudWatch
consumer
Database activity monitoring
Search: Look for specific events across log files
Metrics: Measure activity in your Aurora DB cluster
• Continuously monitor activity in your DB clusters by sending these audit logs to CloudWatch logs
• Export to S3 for long term archival; analyze logs using Amazon Athena; visualize logs with Amazon QuickSight
Visualizations: Create activity dashboards
Alarms: Get notified or take actions
Amazon
Aurora
Amazon
CloudWatch
3rd party
consumer
Amazon
Kinesis

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Aurora migration options
Source database From where Recommended option
RDS
EC2, on premises
EC2, on premises, RDS
Console based automated
snapshot ingestion and catch
up via binlog replication.
Binary snapshot ingestion
through S3 and catch up via
binlog replication.
Schema conversion using
SCT and data migration via
DMS.

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s new in AWS DMS and AWS SCT?
Workload qualification
Allows customers to estimate and manage the efforts required to migrate Oracle
and SQL Server workloads to Aurora
Migration playbook
Step-by-step instructions on how to migrate from Oracle and SQL Server to Aurora
Schema conversion
Automation from Oracle and SQL Server to Aurora is over 90%
Native start point
Customers can use engine-native utilities to copy data, such as PG dump and restore,
and replicate the changes using DMS

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aurora Serverless use cases
Infrequently used applications
(e.g., low-volume blog site)
Applications with variable load—
peaks of activity that are hard to
predict (e.g., news site)
Development or test databases not
needed on nights or weekends
Consolidated fleets of multi-tenant
SaaS applications

47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aurora Serverless
Starts up on demand, shuts down
when not in use
Scales up/down automatically
No application impact when scaling
Pay per second, 1 minute minimum
WARM POOL
OF INSTANCES
APPLICATION
DATABASE STORAGE
SCALABLE DB CAPACITY
REQUEST ROUTERS

48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scale up and down with load
1
2
4
8
16
32
64
128
0
500
1000
1500
2000
2500
3000
1
12
23
34
45
56
67
78
89
100
111
122
133
144
155
166
177
188
199
210
221
232
243
254
265
276
287
298
309
320
331
342
353
364
375
386
397
408
419
430
441
452
463
474
485
496
507
518
529
540
551
562
573
584
595
606
617
628
639
650
661
672
683
694
705
716
727
TPS ACU

49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s new in Aurora Serverless
New regions
Seoul, Singapore, Sydney, Mumbai, London, N. California, Paris, Frankfurt, Canada Central
Compliance
FedRAMP, HIPPA, PCI, SOC, ISO & HITRUST
Preview
Support for Aurora PostgreSQL
Preview
Support for REST DATA API

50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon RDS Data API for serverless applications
Millions of
IOT/mobile devices Data API fleet
API
End-point
Amazon Aurora
Serverless
Access through simple web interface
• Public endpoint addressable from anywhere
• No client configuration required
• No persistent connections required
Ideal for Serverless applications (Lambda)
Ideal for light-weight applications (IOT)

51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Related breakouts
Tuesday, November 27
DAT305—Deep Dive on Amazon Aurora with PostgreSQL Compatibility
5:30 PM–6:30 PM | Venetian, Level 3, San Polo 3405
DAT304—Deep Dive on Amazon Aurora with MySQL Compatibility
6:15 PM–7:15 PM | Venetian, Level 4, Marcello 4505
Wednesday, November 28
DAT207—Migrating Databases to the Cloud with AWS Database Migration Service
2:30 PM–3:30 PM | Venetian, Level 2, Titian 2202–T1

52. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Debanjan Saha
deban@amazon.com

53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.