Noah Davis & Luke Melia of Weplay share a series of examples of Redis in the real world. In doing so, they cover a survey of Redis' features, approach, history and philosophy. Most examples are drawn from the Weplay team's experience using Redis to power features on Weplay.com, a social site for youth sports.
4. Tonight’s gameplan
Fly-by intro to Redis
Redis in Practice
View counts Q&A
throughout, please.
Global locks We love being
Presence interrupted.
Social activity feeds
Friend suggestions
Caching
5. Salvatore
Sanfilippo
Original author of Redis.
@antirez on Twitter.
Lives in Italy.
Recently hired by VMWare.
Great project leader.
6. Describing Redis
Remote dictionary server
“advanced, fast, persistent key- value database”
“Data structures server”
“memcached on steroids”
7. In Salvatore’s words
“I see Redis definitely more as a flexible tool
than as a solution specialized to solve a specific
problem: his mixed soul of cache, store, and
messaging server shows this very well.”
- Salvatore Sanfilippo
8. Qualities
Written in C
Few dependencies
Fast
Single-threaded
Lots of clients available
Initial release was March 2009
9. Features
Key-value store where a value is one of:
scalar/string, list, set, sorted sets, hash
Persistence: in-memory, snapshots, append-only log, VM
Replication: master-slave, configureable
Pub-Sub
Expiry
“Transaction”-y
In development: Redis Cluster
10. What Redis ain’t (...yet)
Big data
Seamless scaling
Ad-hoc query tool
The only data store you’ll ever need
11. Who’s using Redis?
VMWare
Grooveshark
Github
Superfeedr
Craigslist
Ravelry
EngineYard
mediaFAIL
The Guardian
Weplay
Forrst
More...
PostRank
25. About sets
0 to N elements Adding a value to a set
does not require you
Unordered
to check if the value
No repeated members exists in the set first
26. Working with Redis sets 1/3
# SADD key, member
# Adds the specified member to the set stored at key
redis = Redis.new
redis.sadd 'my_set', 'foo' # => true
redis.sadd 'my_set', 'bar' # => true
redis.sadd 'my_set', 'bar' # => false
redis.smembers 'my_set' # => ["foo", "bar"]
27. Working with Redis sets 2/3
# SUNION key1 key2 ... keyN
# Returns the members of a set resulting from the union of all
# the sets stored at the specified keys.
# SUNIONSTORE <i>dstkey key1 key2 ... keyN</i></b>
# Works like SUNION but instead of being returned the resulting
# set is stored as dstkey.
redis = Redis.new
redis.sadd 'set_a', 'foo'
redis.sadd 'set_a', 'bar'
redis.sadd 'set_b', 'bar'
redis.sadd 'set_b', 'baz'
redis.sunion 'set_a', 'set_b' # => ["foo", "baz", "bar"]
redis.sunionstore 'set_ab', 'set_a', 'set_b'
redis.smembers 'set_ab' # => ["foo", "baz", "bar"]
28. Working with Redis sets 3/3
# SINTER key1 key2 ... keyN
# Returns the members that are present in all
# sets stored at the specified keys.
redis = Redis.new
redis.sadd 'set_a', 'foo'
redis.sadd 'set_a', 'bar'
redis.sadd 'set_b', 'bar'
redis.sadd 'set_b', 'baz'
redis.sinter 'set_a', 'set_b' # => ["bar"]
31. Implementation 1/2
# Defining the keys
def current_key
key(Time.now.strftime("%M"))
end
def keys_in_last_5_minutes
now = Time.now
times = (0..5).collect {|n| now - n.minutes }
times.collect{ |t| key(t.strftime("%M")) }
end
def key(minute)
"online_users_minute_#{minute}"
end
32. Implementation 2/2
# Tracking an Active User, and calculating who’s online
def track_user_id(id)
key = current_key
redis.sadd(key, id)
end
def online_user_ids
redis.sunion(*keys_in_last_5_minutes)
end
def online_friend_ids(interested_user_id)
redis.sunionstore("online_users", *keys_in_last_5_minutes)
redis.sinter("online_users",
"user:#{interested_user_id}:friend_ids")
end
35. Via SQL, Approach 1/2
Doesn’t scale to more
complex social graphs
e.g. friends who are SELECT activities.*
FROM activities
JOIN friendships f1 ON f1.from_id =
‘hidden’ from your feed activities.actor_id
JOIN friendships f2 ON f2.to_id =
activities.actor_id
May still require multiple WHERE f1.to_id = ? OR f2.from_id = ?
ORDER BY activities.id DESC
queries to grab LIMIT 15
unindexed data
41. via Redis, 2/2
Key Value
Friend: 11
user:11:feed [100,99,97,96]
Friend: 22
user:22:feed [100,99,98]
Activity: 100
Actor 1
user:33:feed [100,99]
Teammate: 33
Key Value
New York
new_york:feed [100,67]
Boston boston:feed [100,99]
53. Suitability for caching
Excellent match for managed denormalization
ex. friendships, teams, teammates
Excellent match where you would benefit from persistence
and/or replication
Historically, not a good match for a “generational cache,” in
which you want to optimize memory use by evicting least-
recently used (LRU) keys
54. As an LRU cache
TTL-support since inception, but with unintuitive behavior
Writing to volatile key replaced it and cleared the TTL
Redis 2.2 changes this behavior and adds key features:
ability to write to, and update expiry of volatile keys
maxmemory [bytes], maxmemory-policy [policy]
policies: volatile-lru, volatile-ttl, volatile-random,
allkeys-lru, allkeys-random
55. Easy to adapt
Namespaced Rack::Session, Rack::Cache, I18n and cache
Redis cache stores for Ruby web frameworks
implementation is under 1,000 LOC
57. Any
questions?
Thanks!
Follow us on Twitter: @noahd1 and @lukemelia
Tell your friends in youth sports about Weplay.com
Editor's Notes
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
http://antirez.com/m/p.php?i=220\nvolatile-lru remove a key among the ones with an expire set, trying to remove keys not recently used.\nvolatile-ttl remove a key among the ones with an expire set, trying to remove keys with short remaining time to live.\nvolatile-random remove a random key among the ones with an expire set.\nallkeys-lru like volatile-lru, but will remove every kind of key, both normal keys or keys with an expire set.\nallkeys-random like volatile-random, but will remove every kind of keys, both normal keys and keys with an expire set.\n\n