Experience feedback on 10 monthes of happy mongodb usage at fotopedia.
You may also checkout: http://www.slideshare.net/octplane/mongodb-vs-mysql-a-devops-point-of-view
3. Fotopedia
• Fotonauts, an American/French company
• Photo — Encyclopedia
• Heavily interconnected system : flickr,
facebook, wikipedia, picassa, twitter…
• MongoDB in production since last october
• main store lives in MySQL… for now
15. Hadoop HBase ?
• as we were already using Hadoop Map/
Reduce for preparation
• bulk load was just emerging at that time,
requiring to code against HBase private APIs,
generate the data in an ad-hoc binary
format, ...
17. Problem, rerephrased
Load ~12GB into any K/V store
in hours, not days
without wasting a week on development
and another week on setup
and several months on tuning
please ?
18. MongoDB attempt
• Transforming the tabular data into a JSON
form : about half an hour or code, 45 minutes
of hadoop parallel processing
• setup mongo server : 15 minutes
• mongoimport : 3 minutes to start it, 90 minutes
to run
• plug RoR app on mongo : minutes
• prototype was done in a day
19. Batch Synchronous
Download
~ 5.7GB gzip
Geo
Redirect
Backlink Ruby on Rails
Related
~12GB, 12M docs
20. Hot swap ?
• Indexing was locking everything.
• Just run two instances of MongoDB.
• One instance is servicing the web app
• One instance is asleep or loading data
• One third instance knows the status of the two
instances.
21. We loved:
• JSON import format
• efficiency of mongoimport
• simple and flexible installation
• just one cumbersome dependency
• easy to start (we use runit)
• easy to have several instances on one box
22. Second contact
• itʼs just all about graphes, anyway.
• wikilinks
• people following people
• related community albums
• and soon, interlanguage links
23.
24. all about graphes...
• ... and itʼs also all about cache.
• The application needs to “feel” faster, letʼs
cache more.
• The application needs to “feel” right, so letʼs
cache less.
• or — big sigh — invalidate.
25.
26. Page fragment caching
photo by Mykl Roventine on Flickr
Nginx SSI
photo by Aires Dos Santos
Varnish HTTP cache
RoR application photo by Leslie Chatfield on Flickr
27. Haiku ?
There are only two hard things
in Computer Science:
cache invalidation and naming things.
Phil Karlton
28. Naming things
• REST have been a strong design principle in
fotopedia since the early days, and the efforts
are paying.
30. Invalidating
• Rest allows us to invalidate by URL prefix.
• When the Paris album changes, we have to
invalidate /en/Paris.*
31. Varnish invalidation
• Varnish built-in regexp based invalidation is
not designed for intensive, fine grained
invalidation.
• We need to invalidate URLs individually.
34. Waw.
• This time we are actually using MongoDB as a
BTree. Impressive.
• The metacache has been running fine for
several months, and we want to go further.
35. Invalidate less
• We need to be more specific as to what we
invalidate.
• Today, if somebody votes on a photo in the
Paris album, we invalidate all /en/Paris prefix,
and most of it is unchanged.
• We will move towards a more clever
metacache.
36. Metacache reloaded
• Pub/Sub metacache
• Have the backend send a specific header to
be caught by the metacache-feeder, conaining
“subscribe” message.
• This header will be a JSON document, to be
pushed to the metacache.
• The purge commands will be mongo search
queries.
38. when somebody votes
{ url:/en/Paris.*, observe:photos }
when the summary changes
{ url:/en/Paris.*, observe:summary }
when the a new link is created
{ url:/en/Paris.*, observe:links }
{url:/en/Paris, observe:[summary,links]}
{url:/en/Paris/fragment/left_col, observe: [cover]}
{url:/en/Paris/photos.json?skip=0&number=20,
observe:[photos]}
{url:/en/Paris/photos.json?skip=0&number=20,
observe:[photos]}
39. Other uses cases
• Timeline activities storage: just one more BTree
usage.
• Moderation workflow data: tiny dataset, but more
complex queries, map/reduce.
• Suspended experimentation around log
collection and analysis
40. Current situation
• Mysql: main data store
• CouchDB: old timelines (+ chef)
• MongoDB: metacache, wikipedia, moderation,
new timelines
• Redis: raw data cache for counters, recent
activity (+ resque)
41. What about the main store ?
• albums are good fit for documents
• votes and score may be more tricky
• recent introduction of resque
42. In short
• Simple, fast.
• Hackable: in a language most can read.
• Clear roadmap.
• Very helpful and efficient team.
• Designed with application developer needs in
mind.