SlideShare a Scribd company logo
1 of 29
Download to read offline
2013
MongoDB Hacks of
Frustration
Foursquare Hacks for a Better Mongo
MongoNYC
June 21, 2013
Leo Kim
Software Engineer, Foursquare
2013
Agenda
• About Foursquare
• Vital stats
• Our hacks/tools
• Questions
2013
2013
What is Foursquare?
Foursquare helps you
explore the world
around you.
Meet up with friends,
discover new places,
and save money
using your phone.
2013
Big stats
35,000,000+ people
4,000,000,000+ check-ins
55,000,000+ points of interest
1,300,000+ merchants
2013
2013
Moar stats
●
5MM-6MM checkins a day
●
~4K-5K qps against our API, ~150K-300K qps against
Mongo
●
11 clusters, 8 sharded + 3 replica sets
– Mongo 2.2.{3,4}
●
~4TB of data and indexes, all of it kept in memory
– 24-core Intel, 192GB RAM, 4 120GB SSD drives
●
Extensive use of sharding + replica set
ReadPreference.secondary reads
2013
2013
Mongo has scaled with us
We have been using Mongo for three years.
Mongo has enabled us to scale our service
along with our growing user base. It has
given us the flexibility and agility to innovate
in an exciting space where there is still much
to figure out.
2013
2013
Still, some things to deal with
●
Monitoring
– MMS is good, but always could use more stats to
narrow down on pain points.
●
General maintenance
– Constant struggle with data fragmentation
●
Sharding
– No load-based balancing (SERVER-2472)
– Overhead of all-shards queries
– Bugs can leave duplicate records on the wrong
shards
2013
2013
Monitoring hack: “ODash”
2013
2013
Monitoring hack: “Mongolyzer”
2013
2013
Monitoring hack: “Telemetry”
2013
2013
Data size and fragmentation
• Problem: Even with bare metal and SSDs, fragmentation
can degrade performance by increasing data size beyond
available memory.
– Can also be an issue with autobalancing as chunk
moves induce increased paging and further degrade
I/O
• We have enough replicas (~400) where we need to do
this regularly
2013
2013
Alerts!
2013
2013
Hack: “Mackinac”
• (Mostly) automated repair script
• “Kill file” mongod
– Drain queries gracefully from mongod
– About kill files:
http://engineering.foursquare.com/2012/06/20/stability-in-the-midst-of-chaos/
• Stops mongod
• Resyncs from primary
– We considered running compact() but it doesn't
reclaim disk space. May revisit this though.
2013
2013
Hack: “Mackinac”
2013
2013
Hack: Shard key checker
• Checks for shard key usage in the app
• Loads shard keys from mongo config servers,
matches keys against a given {query, document}
• e.g.
db.users({ _id : 12345 }) // shard key match!
db.users({ twitter: “joe@some.com” }) // shard key
miss!
• Why use this?
2013
2013
Detect all-shards queries!
• Problem: All-shards queries
– Not all our queries use shard keys (unfortunately)
– Use up connections, network traffic, overhead in
query processing on mongod + mongos
– Gets worse with more shards
– What happens if one of the replicas is not
responding?
●
Solution: Measure by intercepting queries with
shard key checker and count misses
2013
2013
All-shards queries
[2013-06-09 19:24:27,706] WARN c.f.boot.RogueShardKeyChecker - Possible all-
shards query: db.venues.find({ "del" : { "$ne" : true} , "mayor" : xxxx , "closed" :
{ "$ne" : true}}).sort({ "mayor_count" : -1})
[2013-06-09 19:24:28,296] WARN c.f.boot.RogueShardKeyChecker - Possible all-
shards query: db.tips.find({ "uid" : xxxx}, { "_id" : 1})
[2013-06-09 19:24:28,326] WARN c.f.boot.RogueShardKeyChecker - Possible all-
shards query: db.photos.find({ "uid" : xxxx})
[2013-06-09 19:24:28,696] WARN c.f.boot.RogueShardKeyChecker - Possible all-
shards query: db.comments2.find({ "c.u" : xxxx})
[2013-06-09 19:24:32,246] WARN c.f.boot.RogueShardKeyChecker - Possible all-
shards query: db.user_venue_aggregations2.find({ "_id" : { "$gte" : { "u" : xxxx , "v" :
{ "$oid" : "000000000000000000000000"}} , "$lte" : { "u" : xxxx , "v" : { "$oid" :
"000000000000000000000000"}}}})
2013
2013
All-shards queries
2013
2013
Find hot chunks!
• Problem: Autobalancer balances by data size, but not
load.
– Checkins shard key → {u : 1}
– Imagine a group of users who check in a bunch.
– Imagine the balancer putting all those users on the
same shard, or even the same chunk.
• Solution: Intercept queries with shard key checker and
bucket hits by chunk
2013
2013
Hack: Hot chunk detector
• Need to do a little more to make this work with the
shard key checker
– Create trees of chunk ranges per-collection
– Match shard keys from queries to chunk ranges,
accumulate counts
2013
2013
Hack: Hot chunk detector
2013
2013
Deeper hack: Hot chunk mover
• A standalone process reading from JSON endpoint on
hot chunk detector
– Identifies the hottest chunk
– Attempts to split it (if necessary)
– Move the chunk to the “coldest” shard
• Subject to same problems as regular chunk moves, can
disrupt latencies
• Currently using a hit ratio of p9999/p50 to identify hot
chunk
• Work in progress
2013
2013
Sample hot chunk detector json
2013
2013
Fix data integrity!
• Problem: Application doesn't always clean up after itself
properly.
– Duplicate documents can exist on multiple shards
• Solution: Compare the document shard key from the
host replica against the canonical metadata in shard
key checker (i.e. where the document should “live”)
2013
2013
Hack: “Chunksanity”
• Simple algorithm:
– Connects to each shard
– Iterates through each document in each
collection
– Verifies that the document is correctly placed
according to chunk data in mongo config server
– Deletes any incorrectly placed documents
• Heavyweight process, run only periodically on
specific suspicious collections
2013
2013
Sample Chunksanity logging
[2013-05-29 18:23:36,128] [main] INFO c.f.m.chunksanity.MongoChunkSanity -
Logging misplaced docs to localhost/production_misplaced_docs
[2013-05-29 18:23:36,301] [main] INFO c.f.m.chunksanity.MongoChunkSanity -
Verifying chunks on users at office-canary-2/10.101.1.175:26190
[2013-05-29 18:23:37,971] [ForkJoinPool-1-worker-1] INFO
c.f.m.chunksanity.MongoChunkSanity - Looking at collection foursquare.users on
shard shard0007 using filter: { } and shardKey { "_id" : 1.0}
[2013-05-29 18:24:06,892] [ForkJoinPool-1-worker-2] INFO
c.f.m.chunksanity.MongoChunkSanity - Done with foursquare.users on shard users20
0/xxxx misplaced in 28911 ms
2013
20132013
Questions?
leo@foursquare.com
foursquare.com/jobs
2013
MongoDB Hacks of Frustration
MongoDB Hacks of Frustration

More Related Content

What's hot

Back to Basics 2017: Introduction to Sharding
Back to Basics 2017: Introduction to ShardingBack to Basics 2017: Introduction to Sharding
Back to Basics 2017: Introduction to ShardingMongoDB
 
5 Pitfalls to Avoid with MongoDB
5 Pitfalls to Avoid with MongoDB5 Pitfalls to Avoid with MongoDB
5 Pitfalls to Avoid with MongoDBTim Callaghan
 
Introduction to Sharding
Introduction to ShardingIntroduction to Sharding
Introduction to ShardingMongoDB
 
Creating social features at BranchOut using MongoDB
Creating social features at BranchOut using MongoDBCreating social features at BranchOut using MongoDB
Creating social features at BranchOut using MongoDBLewis Lin 🦊
 
Webinar: Performance Tuning + Optimization
Webinar: Performance Tuning + OptimizationWebinar: Performance Tuning + Optimization
Webinar: Performance Tuning + OptimizationMongoDB
 
Back to Basics Spanish Webinar 3 - Introducción a los replica sets
Back to Basics Spanish Webinar 3 - Introducción a los replica setsBack to Basics Spanish Webinar 3 - Introducción a los replica sets
Back to Basics Spanish Webinar 3 - Introducción a los replica setsMongoDB
 
mongoDB Performance
mongoDB PerformancemongoDB Performance
mongoDB PerformanceMoshe Kaplan
 
MongoDB for Time Series Data: Sharding
MongoDB for Time Series Data: ShardingMongoDB for Time Series Data: Sharding
MongoDB for Time Series Data: ShardingMongoDB
 
Password hashing, salting, bycrpt
Password hashing, salting, bycrptPassword hashing, salting, bycrpt
Password hashing, salting, bycrptAhmad karawash
 
Sharding
ShardingSharding
ShardingMongoDB
 
MongoDB Auto-Sharding at Mongo Seattle
MongoDB Auto-Sharding at Mongo SeattleMongoDB Auto-Sharding at Mongo Seattle
MongoDB Auto-Sharding at Mongo SeattleMongoDB
 
Webinar: Best Practices for Getting Started with MongoDB
Webinar: Best Practices for Getting Started with MongoDBWebinar: Best Practices for Getting Started with MongoDB
Webinar: Best Practices for Getting Started with MongoDBMongoDB
 
Sharding
ShardingSharding
ShardingMongoDB
 
Advanced Sharding Features in MongoDB 2.4
Advanced Sharding Features in MongoDB 2.4 Advanced Sharding Features in MongoDB 2.4
Advanced Sharding Features in MongoDB 2.4 MongoDB
 
Proper passwordhashing
Proper passwordhashingProper passwordhashing
Proper passwordhashingfangjiafu
 
Putting the Go in MongoDB: How We Rebuilt The MongoDB Tools in Go
Putting the Go in MongoDB: How We Rebuilt The MongoDB Tools in GoPutting the Go in MongoDB: How We Rebuilt The MongoDB Tools in Go
Putting the Go in MongoDB: How We Rebuilt The MongoDB Tools in GoMongoDB
 
Microservices Tracing with Spring Cloud and Zipkin
Microservices Tracing with Spring Cloud and ZipkinMicroservices Tracing with Spring Cloud and Zipkin
Microservices Tracing with Spring Cloud and ZipkinMarcin Grzejszczak
 
Schema Design Best Practices with Buzz Moschetti
Schema Design Best Practices with Buzz MoschettiSchema Design Best Practices with Buzz Moschetti
Schema Design Best Practices with Buzz MoschettiMongoDB
 
Mongo Sharding: Case Study
Mongo Sharding: Case StudyMongo Sharding: Case Study
Mongo Sharding: Case StudyWill Button
 
Шардинг в MongoDB, Henrik Ingo (MongoDB)
Шардинг в MongoDB, Henrik Ingo (MongoDB)Шардинг в MongoDB, Henrik Ingo (MongoDB)
Шардинг в MongoDB, Henrik Ingo (MongoDB)Ontico
 

What's hot (20)

Back to Basics 2017: Introduction to Sharding
Back to Basics 2017: Introduction to ShardingBack to Basics 2017: Introduction to Sharding
Back to Basics 2017: Introduction to Sharding
 
5 Pitfalls to Avoid with MongoDB
5 Pitfalls to Avoid with MongoDB5 Pitfalls to Avoid with MongoDB
5 Pitfalls to Avoid with MongoDB
 
Introduction to Sharding
Introduction to ShardingIntroduction to Sharding
Introduction to Sharding
 
Creating social features at BranchOut using MongoDB
Creating social features at BranchOut using MongoDBCreating social features at BranchOut using MongoDB
Creating social features at BranchOut using MongoDB
 
Webinar: Performance Tuning + Optimization
Webinar: Performance Tuning + OptimizationWebinar: Performance Tuning + Optimization
Webinar: Performance Tuning + Optimization
 
Back to Basics Spanish Webinar 3 - Introducción a los replica sets
Back to Basics Spanish Webinar 3 - Introducción a los replica setsBack to Basics Spanish Webinar 3 - Introducción a los replica sets
Back to Basics Spanish Webinar 3 - Introducción a los replica sets
 
mongoDB Performance
mongoDB PerformancemongoDB Performance
mongoDB Performance
 
MongoDB for Time Series Data: Sharding
MongoDB for Time Series Data: ShardingMongoDB for Time Series Data: Sharding
MongoDB for Time Series Data: Sharding
 
Password hashing, salting, bycrpt
Password hashing, salting, bycrptPassword hashing, salting, bycrpt
Password hashing, salting, bycrpt
 
Sharding
ShardingSharding
Sharding
 
MongoDB Auto-Sharding at Mongo Seattle
MongoDB Auto-Sharding at Mongo SeattleMongoDB Auto-Sharding at Mongo Seattle
MongoDB Auto-Sharding at Mongo Seattle
 
Webinar: Best Practices for Getting Started with MongoDB
Webinar: Best Practices for Getting Started with MongoDBWebinar: Best Practices for Getting Started with MongoDB
Webinar: Best Practices for Getting Started with MongoDB
 
Sharding
ShardingSharding
Sharding
 
Advanced Sharding Features in MongoDB 2.4
Advanced Sharding Features in MongoDB 2.4 Advanced Sharding Features in MongoDB 2.4
Advanced Sharding Features in MongoDB 2.4
 
Proper passwordhashing
Proper passwordhashingProper passwordhashing
Proper passwordhashing
 
Putting the Go in MongoDB: How We Rebuilt The MongoDB Tools in Go
Putting the Go in MongoDB: How We Rebuilt The MongoDB Tools in GoPutting the Go in MongoDB: How We Rebuilt The MongoDB Tools in Go
Putting the Go in MongoDB: How We Rebuilt The MongoDB Tools in Go
 
Microservices Tracing with Spring Cloud and Zipkin
Microservices Tracing with Spring Cloud and ZipkinMicroservices Tracing with Spring Cloud and Zipkin
Microservices Tracing with Spring Cloud and Zipkin
 
Schema Design Best Practices with Buzz Moschetti
Schema Design Best Practices with Buzz MoschettiSchema Design Best Practices with Buzz Moschetti
Schema Design Best Practices with Buzz Moschetti
 
Mongo Sharding: Case Study
Mongo Sharding: Case StudyMongo Sharding: Case Study
Mongo Sharding: Case Study
 
Шардинг в MongoDB, Henrik Ingo (MongoDB)
Шардинг в MongoDB, Henrik Ingo (MongoDB)Шардинг в MongoDB, Henrik Ingo (MongoDB)
Шардинг в MongoDB, Henrik Ingo (MongoDB)
 

Viewers also liked

Get your Spatial on with MongoDB in the Cloud
Get your Spatial on with MongoDB in the CloudGet your Spatial on with MongoDB in the Cloud
Get your Spatial on with MongoDB in the CloudMongoDB
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDBJean-Luc David
 
Webinar: MongoDB Persistence with Java and Morphia
Webinar: MongoDB Persistence with Java and MorphiaWebinar: MongoDB Persistence with Java and Morphia
Webinar: MongoDB Persistence with Java and MorphiaMongoDB
 
Partner Recruitment Webinar: "Join the Most Productive Ecosystem in Big Data ...
Partner Recruitment Webinar: "Join the Most Productive Ecosystem in Big Data ...Partner Recruitment Webinar: "Join the Most Productive Ecosystem in Big Data ...
Partner Recruitment Webinar: "Join the Most Productive Ecosystem in Big Data ...MongoDB
 
Concurrency Patterns with MongoDB
Concurrency Patterns with MongoDBConcurrency Patterns with MongoDB
Concurrency Patterns with MongoDBYann Cluchey
 
An introduction to MongoDB
An introduction to MongoDBAn introduction to MongoDB
An introduction to MongoDBCésar Trigo
 
Webinar: 10-Step Guide to Creating a Single View of your Business
Webinar: 10-Step Guide to Creating a Single View of your BusinessWebinar: 10-Step Guide to Creating a Single View of your Business
Webinar: 10-Step Guide to Creating a Single View of your BusinessMongoDB
 

Viewers also liked (8)

Get your Spatial on with MongoDB in the Cloud
Get your Spatial on with MongoDB in the CloudGet your Spatial on with MongoDB in the Cloud
Get your Spatial on with MongoDB in the Cloud
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
 
Webinar: MongoDB Persistence with Java and Morphia
Webinar: MongoDB Persistence with Java and MorphiaWebinar: MongoDB Persistence with Java and Morphia
Webinar: MongoDB Persistence with Java and Morphia
 
Partner Recruitment Webinar: "Join the Most Productive Ecosystem in Big Data ...
Partner Recruitment Webinar: "Join the Most Productive Ecosystem in Big Data ...Partner Recruitment Webinar: "Join the Most Productive Ecosystem in Big Data ...
Partner Recruitment Webinar: "Join the Most Productive Ecosystem in Big Data ...
 
Concurrency Patterns with MongoDB
Concurrency Patterns with MongoDBConcurrency Patterns with MongoDB
Concurrency Patterns with MongoDB
 
JSON-LD and MongoDB
JSON-LD and MongoDBJSON-LD and MongoDB
JSON-LD and MongoDB
 
An introduction to MongoDB
An introduction to MongoDBAn introduction to MongoDB
An introduction to MongoDB
 
Webinar: 10-Step Guide to Creating a Single View of your Business
Webinar: 10-Step Guide to Creating a Single View of your BusinessWebinar: 10-Step Guide to Creating a Single View of your Business
Webinar: 10-Step Guide to Creating a Single View of your Business
 

Similar to MongoDB Hacks of Frustration

Sharding in MongoDB Days 2013
Sharding in MongoDB Days 2013Sharding in MongoDB Days 2013
Sharding in MongoDB Days 2013Randall Hunt
 
Silicon Valley Code Camp 2016 - MongoDB in production
Silicon Valley Code Camp 2016 - MongoDB in productionSilicon Valley Code Camp 2016 - MongoDB in production
Silicon Valley Code Camp 2016 - MongoDB in productionDaniel Coupal
 
MongoDB Tokyo - Monitoring and Queueing
MongoDB Tokyo - Monitoring and QueueingMongoDB Tokyo - Monitoring and Queueing
MongoDB Tokyo - Monitoring and QueueingBoxed Ice
 
Hellenic MongoDB user group - Introduction to sharding
Hellenic MongoDB user group - Introduction to shardingHellenic MongoDB user group - Introduction to sharding
Hellenic MongoDB user group - Introduction to shardingcsoulios
 
MongoDB Sharding
MongoDB ShardingMongoDB Sharding
MongoDB ShardingRob Walters
 
MongoDB at Gilt Groupe
MongoDB at Gilt GroupeMongoDB at Gilt Groupe
MongoDB at Gilt GroupeMongoDB
 
Webinar: Serie Operazioni per la vostra applicazione - Sessione 6 - Installar...
Webinar: Serie Operazioni per la vostra applicazione - Sessione 6 - Installar...Webinar: Serie Operazioni per la vostra applicazione - Sessione 6 - Installar...
Webinar: Serie Operazioni per la vostra applicazione - Sessione 6 - Installar...MongoDB
 
AWS re:Invent 2016| GAM302 | Sony PlayStation: Breaking the Bandwidth Barrier...
AWS re:Invent 2016| GAM302 | Sony PlayStation: Breaking the Bandwidth Barrier...AWS re:Invent 2016| GAM302 | Sony PlayStation: Breaking the Bandwidth Barrier...
AWS re:Invent 2016| GAM302 | Sony PlayStation: Breaking the Bandwidth Barrier...Amazon Web Services
 
Introduction to Sharding
Introduction to ShardingIntroduction to Sharding
Introduction to ShardingMongoDB
 
Webinar: Scaling MongoDB
Webinar: Scaling MongoDBWebinar: Scaling MongoDB
Webinar: Scaling MongoDBMongoDB
 
Augmenting Mongo DB with Treasure Data
Augmenting Mongo DB with Treasure DataAugmenting Mongo DB with Treasure Data
Augmenting Mongo DB with Treasure DataTreasure Data, Inc.
 
Augmenting Mongo DB with treasure data
Augmenting Mongo DB with treasure dataAugmenting Mongo DB with treasure data
Augmenting Mongo DB with treasure dataTreasure Data, Inc.
 
MongoDB Best Practices
MongoDB Best PracticesMongoDB Best Practices
MongoDB Best PracticesLewis Lin 🦊
 
Back tobasicswebinar part6-rev.
Back tobasicswebinar part6-rev.Back tobasicswebinar part6-rev.
Back tobasicswebinar part6-rev.MongoDB
 
Benchmarking at Parse
Benchmarking at ParseBenchmarking at Parse
Benchmarking at ParseTravis Redman
 
Advanced Benchmarking at Parse
Advanced Benchmarking at ParseAdvanced Benchmarking at Parse
Advanced Benchmarking at ParseMongoDB
 
Mongodb at-gilt-groupe-seattle-2012-09-14-final
Mongodb at-gilt-groupe-seattle-2012-09-14-finalMongodb at-gilt-groupe-seattle-2012-09-14-final
Mongodb at-gilt-groupe-seattle-2012-09-14-finalMongoDB
 
Introduction to MongoDB sharding
Introduction to MongoDB shardingIntroduction to MongoDB sharding
Introduction to MongoDB shardingJordi Soucheiron
 
Silicon Valley Code Camp 2015 - Advanced MongoDB - The Sequel
Silicon Valley Code Camp 2015 - Advanced MongoDB - The SequelSilicon Valley Code Camp 2015 - Advanced MongoDB - The Sequel
Silicon Valley Code Camp 2015 - Advanced MongoDB - The SequelDaniel Coupal
 
MongoDB: How We Did It – Reanimating Identity at AOL
MongoDB: How We Did It – Reanimating Identity at AOLMongoDB: How We Did It – Reanimating Identity at AOL
MongoDB: How We Did It – Reanimating Identity at AOLMongoDB
 

Similar to MongoDB Hacks of Frustration (20)

Sharding in MongoDB Days 2013
Sharding in MongoDB Days 2013Sharding in MongoDB Days 2013
Sharding in MongoDB Days 2013
 
Silicon Valley Code Camp 2016 - MongoDB in production
Silicon Valley Code Camp 2016 - MongoDB in productionSilicon Valley Code Camp 2016 - MongoDB in production
Silicon Valley Code Camp 2016 - MongoDB in production
 
MongoDB Tokyo - Monitoring and Queueing
MongoDB Tokyo - Monitoring and QueueingMongoDB Tokyo - Monitoring and Queueing
MongoDB Tokyo - Monitoring and Queueing
 
Hellenic MongoDB user group - Introduction to sharding
Hellenic MongoDB user group - Introduction to shardingHellenic MongoDB user group - Introduction to sharding
Hellenic MongoDB user group - Introduction to sharding
 
MongoDB Sharding
MongoDB ShardingMongoDB Sharding
MongoDB Sharding
 
MongoDB at Gilt Groupe
MongoDB at Gilt GroupeMongoDB at Gilt Groupe
MongoDB at Gilt Groupe
 
Webinar: Serie Operazioni per la vostra applicazione - Sessione 6 - Installar...
Webinar: Serie Operazioni per la vostra applicazione - Sessione 6 - Installar...Webinar: Serie Operazioni per la vostra applicazione - Sessione 6 - Installar...
Webinar: Serie Operazioni per la vostra applicazione - Sessione 6 - Installar...
 
AWS re:Invent 2016| GAM302 | Sony PlayStation: Breaking the Bandwidth Barrier...
AWS re:Invent 2016| GAM302 | Sony PlayStation: Breaking the Bandwidth Barrier...AWS re:Invent 2016| GAM302 | Sony PlayStation: Breaking the Bandwidth Barrier...
AWS re:Invent 2016| GAM302 | Sony PlayStation: Breaking the Bandwidth Barrier...
 
Introduction to Sharding
Introduction to ShardingIntroduction to Sharding
Introduction to Sharding
 
Webinar: Scaling MongoDB
Webinar: Scaling MongoDBWebinar: Scaling MongoDB
Webinar: Scaling MongoDB
 
Augmenting Mongo DB with Treasure Data
Augmenting Mongo DB with Treasure DataAugmenting Mongo DB with Treasure Data
Augmenting Mongo DB with Treasure Data
 
Augmenting Mongo DB with treasure data
Augmenting Mongo DB with treasure dataAugmenting Mongo DB with treasure data
Augmenting Mongo DB with treasure data
 
MongoDB Best Practices
MongoDB Best PracticesMongoDB Best Practices
MongoDB Best Practices
 
Back tobasicswebinar part6-rev.
Back tobasicswebinar part6-rev.Back tobasicswebinar part6-rev.
Back tobasicswebinar part6-rev.
 
Benchmarking at Parse
Benchmarking at ParseBenchmarking at Parse
Benchmarking at Parse
 
Advanced Benchmarking at Parse
Advanced Benchmarking at ParseAdvanced Benchmarking at Parse
Advanced Benchmarking at Parse
 
Mongodb at-gilt-groupe-seattle-2012-09-14-final
Mongodb at-gilt-groupe-seattle-2012-09-14-finalMongodb at-gilt-groupe-seattle-2012-09-14-final
Mongodb at-gilt-groupe-seattle-2012-09-14-final
 
Introduction to MongoDB sharding
Introduction to MongoDB shardingIntroduction to MongoDB sharding
Introduction to MongoDB sharding
 
Silicon Valley Code Camp 2015 - Advanced MongoDB - The Sequel
Silicon Valley Code Camp 2015 - Advanced MongoDB - The SequelSilicon Valley Code Camp 2015 - Advanced MongoDB - The Sequel
Silicon Valley Code Camp 2015 - Advanced MongoDB - The Sequel
 
MongoDB: How We Did It – Reanimating Identity at AOL
MongoDB: How We Did It – Reanimating Identity at AOLMongoDB: How We Did It – Reanimating Identity at AOL
MongoDB: How We Did It – Reanimating Identity at AOL
 

More from MongoDB

MongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
MongoDB SoCal 2020: Migrate Anything* to MongoDB AtlasMongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
MongoDB SoCal 2020: Migrate Anything* to MongoDB AtlasMongoDB
 
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!MongoDB
 
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...MongoDB
 
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDBMongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDBMongoDB
 
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...MongoDB
 
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series DataMongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series DataMongoDB
 
MongoDB SoCal 2020: MongoDB Atlas Jump Start
 MongoDB SoCal 2020: MongoDB Atlas Jump Start MongoDB SoCal 2020: MongoDB Atlas Jump Start
MongoDB SoCal 2020: MongoDB Atlas Jump StartMongoDB
 
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]MongoDB
 
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2MongoDB
 
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...MongoDB
 
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!MongoDB
 
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your MindsetMongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your MindsetMongoDB
 
MongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
MongoDB .local San Francisco 2020: MongoDB Atlas JumpstartMongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
MongoDB .local San Francisco 2020: MongoDB Atlas JumpstartMongoDB
 
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...MongoDB
 
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++MongoDB
 
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...MongoDB
 
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep DiveMongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep DiveMongoDB
 
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & GolangMongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & GolangMongoDB
 
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...MongoDB
 
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...MongoDB
 

More from MongoDB (20)

MongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
MongoDB SoCal 2020: Migrate Anything* to MongoDB AtlasMongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
MongoDB SoCal 2020: Migrate Anything* to MongoDB Atlas
 
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
MongoDB SoCal 2020: Go on a Data Safari with MongoDB Charts!
 
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
 
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDBMongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
MongoDB SoCal 2020: A Complete Methodology of Data Modeling for MongoDB
 
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
MongoDB SoCal 2020: From Pharmacist to Analyst: Leveraging MongoDB for Real-T...
 
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series DataMongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
MongoDB SoCal 2020: Best Practices for Working with IoT and Time-series Data
 
MongoDB SoCal 2020: MongoDB Atlas Jump Start
 MongoDB SoCal 2020: MongoDB Atlas Jump Start MongoDB SoCal 2020: MongoDB Atlas Jump Start
MongoDB SoCal 2020: MongoDB Atlas Jump Start
 
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
MongoDB .local San Francisco 2020: Powering the new age data demands [Infosys]
 
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
MongoDB .local San Francisco 2020: Using Client Side Encryption in MongoDB 4.2
 
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
MongoDB .local San Francisco 2020: Using MongoDB Services in Kubernetes: any ...
 
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
MongoDB .local San Francisco 2020: Go on a Data Safari with MongoDB Charts!
 
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your MindsetMongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
MongoDB .local San Francisco 2020: From SQL to NoSQL -- Changing Your Mindset
 
MongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
MongoDB .local San Francisco 2020: MongoDB Atlas JumpstartMongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
MongoDB .local San Francisco 2020: MongoDB Atlas Jumpstart
 
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
MongoDB .local San Francisco 2020: Tips and Tricks++ for Querying and Indexin...
 
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
MongoDB .local San Francisco 2020: Aggregation Pipeline Power++
 
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
MongoDB .local San Francisco 2020: A Complete Methodology of Data Modeling fo...
 
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep DiveMongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB .local San Francisco 2020: MongoDB Atlas Data Lake Technical Deep Dive
 
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & GolangMongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
MongoDB .local San Francisco 2020: Developing Alexa Skills with MongoDB & Golang
 
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
MongoDB .local Paris 2020: Realm : l'ingrédient secret pour de meilleures app...
 
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
MongoDB .local Paris 2020: Upply @MongoDB : Upply : Quand le Machine Learning...
 

Recently uploaded

Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 

Recently uploaded (20)

Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 

MongoDB Hacks of Frustration

  • 1. 2013 MongoDB Hacks of Frustration Foursquare Hacks for a Better Mongo MongoNYC June 21, 2013 Leo Kim Software Engineer, Foursquare
  • 2. 2013 Agenda • About Foursquare • Vital stats • Our hacks/tools • Questions 2013
  • 3. 2013 What is Foursquare? Foursquare helps you explore the world around you. Meet up with friends, discover new places, and save money using your phone.
  • 4. 2013 Big stats 35,000,000+ people 4,000,000,000+ check-ins 55,000,000+ points of interest 1,300,000+ merchants 2013
  • 5. 2013 Moar stats ● 5MM-6MM checkins a day ● ~4K-5K qps against our API, ~150K-300K qps against Mongo ● 11 clusters, 8 sharded + 3 replica sets – Mongo 2.2.{3,4} ● ~4TB of data and indexes, all of it kept in memory – 24-core Intel, 192GB RAM, 4 120GB SSD drives ● Extensive use of sharding + replica set ReadPreference.secondary reads 2013
  • 6. 2013 Mongo has scaled with us We have been using Mongo for three years. Mongo has enabled us to scale our service along with our growing user base. It has given us the flexibility and agility to innovate in an exciting space where there is still much to figure out. 2013
  • 7. 2013 Still, some things to deal with ● Monitoring – MMS is good, but always could use more stats to narrow down on pain points. ● General maintenance – Constant struggle with data fragmentation ● Sharding – No load-based balancing (SERVER-2472) – Overhead of all-shards queries – Bugs can leave duplicate records on the wrong shards 2013
  • 11. 2013 Data size and fragmentation • Problem: Even with bare metal and SSDs, fragmentation can degrade performance by increasing data size beyond available memory. – Can also be an issue with autobalancing as chunk moves induce increased paging and further degrade I/O • We have enough replicas (~400) where we need to do this regularly 2013
  • 13. 2013 Hack: “Mackinac” • (Mostly) automated repair script • “Kill file” mongod – Drain queries gracefully from mongod – About kill files: http://engineering.foursquare.com/2012/06/20/stability-in-the-midst-of-chaos/ • Stops mongod • Resyncs from primary – We considered running compact() but it doesn't reclaim disk space. May revisit this though. 2013
  • 15. 2013 Hack: Shard key checker • Checks for shard key usage in the app • Loads shard keys from mongo config servers, matches keys against a given {query, document} • e.g. db.users({ _id : 12345 }) // shard key match! db.users({ twitter: “joe@some.com” }) // shard key miss! • Why use this? 2013
  • 16. 2013 Detect all-shards queries! • Problem: All-shards queries – Not all our queries use shard keys (unfortunately) – Use up connections, network traffic, overhead in query processing on mongod + mongos – Gets worse with more shards – What happens if one of the replicas is not responding? ● Solution: Measure by intercepting queries with shard key checker and count misses 2013
  • 17. 2013 All-shards queries [2013-06-09 19:24:27,706] WARN c.f.boot.RogueShardKeyChecker - Possible all- shards query: db.venues.find({ "del" : { "$ne" : true} , "mayor" : xxxx , "closed" : { "$ne" : true}}).sort({ "mayor_count" : -1}) [2013-06-09 19:24:28,296] WARN c.f.boot.RogueShardKeyChecker - Possible all- shards query: db.tips.find({ "uid" : xxxx}, { "_id" : 1}) [2013-06-09 19:24:28,326] WARN c.f.boot.RogueShardKeyChecker - Possible all- shards query: db.photos.find({ "uid" : xxxx}) [2013-06-09 19:24:28,696] WARN c.f.boot.RogueShardKeyChecker - Possible all- shards query: db.comments2.find({ "c.u" : xxxx}) [2013-06-09 19:24:32,246] WARN c.f.boot.RogueShardKeyChecker - Possible all- shards query: db.user_venue_aggregations2.find({ "_id" : { "$gte" : { "u" : xxxx , "v" : { "$oid" : "000000000000000000000000"}} , "$lte" : { "u" : xxxx , "v" : { "$oid" : "000000000000000000000000"}}}}) 2013
  • 19. 2013 Find hot chunks! • Problem: Autobalancer balances by data size, but not load. – Checkins shard key → {u : 1} – Imagine a group of users who check in a bunch. – Imagine the balancer putting all those users on the same shard, or even the same chunk. • Solution: Intercept queries with shard key checker and bucket hits by chunk 2013
  • 20. 2013 Hack: Hot chunk detector • Need to do a little more to make this work with the shard key checker – Create trees of chunk ranges per-collection – Match shard keys from queries to chunk ranges, accumulate counts 2013
  • 21. 2013 Hack: Hot chunk detector 2013
  • 22. 2013 Deeper hack: Hot chunk mover • A standalone process reading from JSON endpoint on hot chunk detector – Identifies the hottest chunk – Attempts to split it (if necessary) – Move the chunk to the “coldest” shard • Subject to same problems as regular chunk moves, can disrupt latencies • Currently using a hit ratio of p9999/p50 to identify hot chunk • Work in progress 2013
  • 23. 2013 Sample hot chunk detector json 2013
  • 24. 2013 Fix data integrity! • Problem: Application doesn't always clean up after itself properly. – Duplicate documents can exist on multiple shards • Solution: Compare the document shard key from the host replica against the canonical metadata in shard key checker (i.e. where the document should “live”) 2013
  • 25. 2013 Hack: “Chunksanity” • Simple algorithm: – Connects to each shard – Iterates through each document in each collection – Verifies that the document is correctly placed according to chunk data in mongo config server – Deletes any incorrectly placed documents • Heavyweight process, run only periodically on specific suspicious collections 2013
  • 26. 2013 Sample Chunksanity logging [2013-05-29 18:23:36,128] [main] INFO c.f.m.chunksanity.MongoChunkSanity - Logging misplaced docs to localhost/production_misplaced_docs [2013-05-29 18:23:36,301] [main] INFO c.f.m.chunksanity.MongoChunkSanity - Verifying chunks on users at office-canary-2/10.101.1.175:26190 [2013-05-29 18:23:37,971] [ForkJoinPool-1-worker-1] INFO c.f.m.chunksanity.MongoChunkSanity - Looking at collection foursquare.users on shard shard0007 using filter: { } and shardKey { "_id" : 1.0} [2013-05-29 18:24:06,892] [ForkJoinPool-1-worker-2] INFO c.f.m.chunksanity.MongoChunkSanity - Done with foursquare.users on shard users20 0/xxxx misplaced in 28911 ms 2013