1.
DAT307 - Deep Dive into Amazon ElastiCache
Architecture and Design Patterns
Nate Wiger, Principal Solutions Architect
November 14, 2013
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

2.
Contents
•
•
•
•
•
Caching: What’s all this then?
Amazon ElastiCache
Laziness, impatience, and hubris
From one to a dozen nodes
Memcached vs. Redis showdown

3.
Device Fragmentation
•
•
•
•
•
•
Phones, tablets, PCs, toasters
HTML, apps, JSON APIs
Presentation differs
Data is the same
CDN for static images, videos
Doesn’t help “Welcome Back, Kotter!”

4.
Death By 1000 Queries
•
•
•
•
•
Login, session
New messages, recent posts
Calls to Facebook, Twitter APIs
Your friends love the new Coldplay album!!!
Sudden viral traffic spikes

5.
cache (noun)
a group of things that have been stored in a secret
place because they are illegal or have been stolen

6.
Typical Web 2.0 App
External APIs
ELB
App

7.
Amazon ElastiCache
•
•
•
•
•
Managed cache service
Memcached or Redis
Launch cluster of nodes
Scale up / down
Monitoring + alerts

8.
Memcached
•
•
•
•
•
In-memory
Slab allocator
Multithreaded
No persistence
Gold standard

9.
Fire It Up

10.
Fire It Up

11.
Fire It Up

12.
Wire It Up

13.
Wire It Up

14.
Wire It Up
# Ruby
require ‘dalli’
cache = Dalli::Client([
’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’,
’mycache.z2vq55.0002.usw2.cache.amazonaws.com:11211’
])
cache.set("some_key", "Some value")
value = cache.get("some_key")
cache.set("another_key", 3)
cache.delete("another_key”)

15.
Multiple Cache Nodes
External APIs
ELB
App

16.
Sharding Across Nodes
server_list = [
’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’,
’mycache.z2vq55.0002.usw2.cache.amazonaws.com:11211’
]
server_index = hash(key) % server_list.length
server = server_list[server_index]

17.
Sharding Across Nodes
server_list = [
’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’,
’mycache.z2vq55.0002.usw2.cache.amazonaws.com:11211’
]
server_index = hash(key) % server_list.length
server = server_list[server_index]
BAD

18.
Consistent Hashing

19.
It’s All Been Done Before
•
•
•
•
•
Ruby – Dalli
Python – HashRing / MemcacheRing
Node.js – node-memcached
PHP – libketama or ElastiCache Client
Java – SpyMemcached or ElastiCache Client

20.
So Far
•
•
•
•
Launched a cache cluster
Got the node names
Connected our client
Figured out sharding

21.
What To Cache?
•
•
•
•
•
Everything!
Database records
Full HTML pages
Page fragments
Remote API calls

22.
How To Cache It?
• Lazy population
• Write-through
• Timed refresh

23.
Laziness is a Virtue
# Python
def get_user(user_id):
record = cache.get(user_id)
if record is None:
# Run a DB query
record = db.query("select * from users where id = ?", user_id)
cache.set(user_id, record)
return record
# App code
user = get_user(17)

24.
Ship It
•
•
•
•
•
Most data is never accessed
Ensures cache is filled
Caches fail and scale
But cache miss penalty
Best approach for most data

25.
Foresight is 20-20
# Python
def save_user(user_id, values):
record = db.query("update users ... where id = ?", user_id, values)
cache.set(user_id, record)
return record
# App code
user = save_user(17, {"name": "Nate Dogg"})

26.
Laziness vs. Impatience
•
•
•
•
•
Ensures cache is always current
Write penalty vs. read penalty
But missing data on scale up
Plus excess data / cache churn
Still need lazy fetch too

27.
Combo Move!
def save_user(user_id, values):
record = db.query("update users ... where id = ?", user_id, values)
cache.set(user_id, record, 300) # ttl
return record
def get_user(user_id):
record = cache.get(user_id)
if record is None:
record = db.query("select * from users where id = ?", user_id)
cache.set(user_id, record, 300) # ttl
return record
# App code
save_user(17, {"name": "Nate Diddy"})
user = get_user(17)

28.
Timed Refresh
•
•
•
•
Run job to periodically update cache
Good for Top-N lists
Time-intensive rankings
Trending items

29.
Monitoring
+ Alerts

30.
Monitoring
•
•
•
•
•
Integration with CloudWatch metrics
Setup alarms to send via email
Memory usage
Evictions
Which ElastiCache metrics should I monitor?

31.
Node Discovery
• Setup an Amazon SNS topic for ElastiCache
• Have app listen for events
– ElastiCache:AddCacheNodeComplete
– ElastiCache:RemoveCacheNodeComplete
• Reconfigure connections
• See Event Notifications and Amazon SNS

32.
Programmable Scaling
External APIs
SNS
Add Node
ELB
App

33.
Node Auto-Discovery
# PHP
$server_endpoint = "mycache.z2vq55.cfg.usw2.cache.amazonaws.com";
$server_port
= 11211;
$cache = new Memcached();
$cache->setOption(
Memcached::OPT_CLIENT_MODE, Memcached::DYNAMIC_CLIENT_MODE);
# Set config endpoint as only server
$cache->addServer($server_endpoint, $server_port);
# Lib auto-locates nodes
$cache->set("key", "value");

34.
Redis
•
•
•
•
•
•
Also in-memory
Advanced data types
Atomic operations
Single-threaded
Persistence
Read replicas

35.
Leaderboard with Sorted Sets
ZADD
ZADD
ZADD
ZADD
leaderboard
leaderboard
leaderboard
leaderboard
556
819
105
1312
"Andy"
"Barry"
"Carl"
"Derek"
ZREVRANGE leaderboard 0 -1
1) "Derek"
2) "Barry"
3) "Andy"
4) "Carl"
ZREVRANK "Barry"
2

36.
Follow the Leader
def save_score(user, score):
record = db.query("update users ... where id = ?", user_id, score)
redis.zadd("leaderboard", score, user)
def get_rank(user)
return redis.zrevrank(user) + 1
# App code
save_score("Andy", 556)
save_score("Barry", 819)
save_score("Carl", 105)
save_score("Derek", 1312)
get_rank("Barry")
# 2

37.
Redis Replicas
External APIs
ELB
App
Writes
Reads
Replication Group

38.
Redis Sharding
• Same concept as Memcached
• BUT
• Can't shard
– Lists
– Sets / sorted sets
– Hashes
• Require single in-memory structure

39.
Anti-Pattern: Dedicated Nodes
• Spawn multiple nodes
• Use for different features
– Leaderboard
– Counters
• Can still shard key-value ops

40.
Dedicated Redis Nodes
External APIs
ELB
App
Leaderboard
Counters

41.
Summary
•
•
•
•
•
Caching is good
Good caching is hard
ElastiCache eases deployment
Memcached or Redis
More to come

42.
Please give us your feedback on this
presentation
DAT307 - Nate Wiger
As a thank you, we will select prize
winners daily for completed surveys!