Recording: https://www.youtube.com/watch?v=qHkXVY2LpwU External links: https://gist.github.com/itamarhaber/dddc3d4d9c19317b1477 Applications today are required to process massive amounts of data and return responses in real time. Simply storing Big Data is no longer enough; insights must be gleaned and decisions made as soon as data rushes in. In-memory databases like Redis provide the blazing fast speeds required for sub-second application response times. Using a combination of in-memory Redis and disk-based MongoDB can significantly reduce the “digestive” challenge associated with processing high velocity data.

1. Why Your MongoDB
Needs Redis
Ask a Redis Expert™ Webinar
September 3rd, 2015

2. @itamarhaber
A Redis Geek and Cheif Developer Advocate
at
Have you signed for newsletter?
[1] http://bit.ly/RedisWatch

3. The Menu: What MongoDB? What Redis?
A document-oriented
disk-based
database
If you’re here,
you’re probably
already using it 
An in-memory and
optionally persistable
data structures
engine
Redis is
Blazing-Fast™

4. Why broth?
“Both projects are about there is something
wrong if we use an RDBMS for all the kind
of works… in response to the same non-nail
problems, these two tools have taken different
paths”
[2] MongoDB and Redis: a different interpretation of what's
wrong with Relational DBs, June 3rd 2009, @antirez

5. ● “… an [4] open source (BSD licensed), in-memory
data structure store, used as database, cache and
message broker."
● 5 data structures, 180+ commands, in-memory,
persistable to disk, atomic operations, Lua scripting,
replication, high availibility, clustering
and an active & vibrant community
● Nee circa 2009, by [5] antirez
(a.k.a Salvatore Sanfilippo)
● Sponsored by Redis Labs
[3] Redis (REmote Dictionary Server)

6. Why is Redis so Glazing Fast?
Is not unlike asking “Why is a Ferrari so fast?”
Answer: Performance dictates the Design

7. Redis is designed for performance
• [6] The Redis Manifesto: We’re against complexity
• RAM is fast ([7] Latency Numbers Every Programmer
Should Know)
• C is fast ([8] Programming Languages Benchmark)
• (mostly) single-threaded event loops are fast
• Data structures are optimized for performance
• Transparent complexity, time-space tradeoff knobs, …
• Shameless Plug: [9] webinar – How To Achieve 1.5
Million ops/second with Redis

8. A talk about MongoDB performance
[10] WiredTiger iiBench Results
I'm hardly an
expert, but with
MongoDB v3
storage engines
and future work
this could very
well be a moot
point 

9. Hors d'oeuvre: La Quiche
• You can make an excellent
quiche cache with Redis
• Configurable eviction, key expiration and
optional data persistence <- blend and mix
them to your taste
• Manage the cache in app; wrap the logic
around your primary database driver; or grab
something ready from GitHub

10. Souped up Intelligent Cache
Intelligent cache
“understands”
the data it
manages,
whereas for the
dumb ones data
is opaque

11. [11] An introduction to Redis data types
A talk about Redis’ Data Types
Primary
1. String
2. Hash
3. List
4. Set
5. Zorted Set
Secondary
1. Integer
2. Float
3. String bitmap
4. HyperLogLog
5. Coming: Geo hash,
Bloom filter

12. A talk about Redis’ 180+ Commands
[12] Redis Command
Reference
[13] Red is Beautiful: A
Visualization of Redis
Commands

13. Entremets: Counting with Redis
Before moving to the main course, consider the
common need for counting things. Arguably,
any database can do that, but do you want to
load yours with that?
Redis is great for counting
stuff and does it really
fast…

14. - Main Course -
Stop Big Data Indigestion
Before It Starts

15. You probably haven't seen anything
like this before
VolumeMongoDB truly excels when
is comes to volume and
variety of data…
…but data coming in at
extreme velocity poses
a digestive challenge for
for any disk-based database

16. Data ingestion at high velocity
Mobile, online and IoT apps
produce more and more data
with every day that passes.
Simply storing the data as it
comes in doesn't cut it anymore – real time
processing is a must in order distill information
from the data as it rushes in.

17. A talk about more performance
By doing LESS
you can do MORE
(with MongoDB)
Put differently, "chew" your
data with Redis to prevent
data ingestion indigestion

18. Use case A: Google Analytics
• A real time analytics platform provider
• Strongly focuses on users' behavior
• Primary data storage is MongoDB
• Activity is collected immediately or in bulks
• Raw data fed to Hadoop for offline crunching
• Real time metrics and initial information from
the stream is obtained with Redis

19. The tidal flow
Sessions events
Real time analysis
Offline analysis

20. Deep dive topic: sessionizing data
• Stream of events
• A session is a document
• Each has 10s-1000s events
• Events from different users
arrive in order but interleaved
• The result: many small updates
to each session's document
• Peak load: 1.1M ops/sec (Q1 2015)

21. You say potato, I say potato
Hash data type:
HSET session:1
event:1 data
HSET session:1
event:2 data
...
HINCRBY session:1
seq 1
JSON:
{
session: 1,
events: [
{ id: 1,
data: data },
{ id: 2,
data: data },
...

22. Swallowing in Python
import redis
import pymongo
r = redis.Redis()
session = r.hgetall('session:1')
# {'event:1': 'data', 'event:2': 'data', 'seq': '2'}
...
m = pymongo.MongoClient()
db = m.rta
sessionid = db.sessions.insert_one(session)

23. Keeping track of sessions
• Sessions end after a logout or a timeout
• Logout events are trivial to detect
• Timeouts, e.g. 30 minutes of inactivity, are
trickier to manage considering there could
be 10,000s of active sessions
• This is where Redis' key expiry and
keyspace notifications come in very handy 

24. Once you see it, it can't be unseen
Using Redis as a buffer in front
of MongoDB for write-
intensive, hot Big Data is a
useful pattern that makes it
easy to get information in real
time as well as distribute the
load more efficiently.
Ceci n’est pas
une Quiche

25. Use case B: Waze
• An international navigation app/service
• Strongly focuses on public transit
• 10s of millions of users during peak hours
• Primary data storage is MongoDB
• Base data is created in advance
• Real time updates (traffic, vehicles and
passengers) pour into Redis for scheduling
adjustments and notifications

26. Use case C: Tinder
• A dating app/service
• Strongly focuses on spatially-related groups
• Primary data storage is MongoDB
• Data includes user profiles & preferences
• An influx of positional and preferential
("swipes") events is first munched by Redis

27. Use case D: Clash of Clans
• A massive real time game
• Strongly focuses on matched team play
• 1000s of teams with 100s of members
• Primary data storage is MongoDB
• Match progress is sieved through Redis for
real time resources status, leaderboards and
scoring

28. Use case E: Weather.com
• IoT startup
• Focuses on environmental monitoring
• Pilot: real time fire fighting
• Primary data storage is MongoDB
• Sensor data (temperature, humidity, …) is
aggregated in Redis, providing warnings and
alarms in real time

29. Getting started with Redis
• Try it online at [14] http://try.redis.io/
• Build it from the source
• [15] Try Redis Labs Enterprise Cluster
• Run it in a container
• [16] Connect to it from any language
git clone https://github.com/antirez/redis
cd redis
git checkout 3.0.1
make; make test; make install
docker run -d --name redis -p 6379:6379 redis

30. Questions or feedback? Contact me!
Itamar Haber
Chief Developer Advocate
📧 itamar@redislabs.com
@itamarhaber
Follow us on Twitter
@redislabs