Concurrency Control in MongoDB 3.0
•Download as PPTX, PDF•
12 likes•10,592 views
- MongoDB's concurrency control uses multiple-granularity locking at the instance, database, and collection level. This allows finer-grained locking than previous approaches. - The storage engine handles concurrency control at lower levels like the document level, using either MVCC or locking depending on the engine. WiredTiger uses MVCC while MMAPv1 uses locking at the collection level. - Intents signal the intention to access lower levels without acquiring locks upfront, improving concurrency compared to directly acquiring locks. The lock manager enforces the locking protocol and ensures consistency.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Concurrency Control in MongoDB 3.0
Similar to Concurrency Control in MongoDB 3.0 (20)
More from MongoDB
More from MongoDB (20)
Concurrency Control in MongoDB 3.0
- 1. Concurrency Control in MongoDB 3.0 Kaloian Manassiev Senior Engineer at MongoDB kaloianm@mongodb.com @kaloianm
- 2. 2 Audience • Operations engineers • Application developers • Third-party storage engine developers • Anybody who is curious
- 3. 3 What is concurrency control?
- 4. 4 What is concurrency control? • Locking • Data consistency $100 + $100 = $200 (not $100 ) • MVCC
- 7. 7 What you will learn • Top level concurrency control –Instance, database and collection • How top level cooperates with the storage engine
- 8. 8 What you will NOT learn • WiredTiger internals –Separate session on this
- 9. 9 Talk outline • MongoDB concurrency control • Multiple-granularity locking • WiredTiger vs MMAP V1 • Questions
- 10. 10 MongoDB 2.0 (and before) THE TOP LOCK R W R OK NO W NO NO
- 11. 11 MongoDB 2.2 Lock per database db1.coll.insert({a:1}) db4.coll.find({c:5})
- 12. 12 MongoDB 2.2 THE TOP LOCK + intents
- 15. 15 Why intents? • Get rid of the global lock? • Use read (shared) lock?
- 20. 20 TOP db.fsyncLock({lock:1}) S S S S S db1.coll.find({a:1}) IS S
- 21. 21 Intents • “Intention” to access one or more children of an item –Compatible with other intents –Need to acquire lock further down –No overhead
- 22. 22 Locks • “Lock” an item for particular access –Item being locked –All items below it
- 23. 23 Intents compatibility IS IX S X IS OK OK OK NO IX OK OK NO NO S OK NO OK NO X NO NO NO NO
- 26. 26 Lock Manager • Ensures the locking protocol is obeyed • Very low overhead • Testable
- 27. 27 Lock Manager • Improved fairness • Support for locking policies • Lock statistics gathering
- 28. 28 TOP db.coll1.update({a:1}) db.coll1.update({a:4}) IX IX IX IX IX IX X X
- 30. 30 Storage Engine API TOP Storage Engine API
- 31. 31 TOP db.coll1.update({a:1}) db.coll1.update({a:4}) IX IX IX IX IX IX WiredTiger MVCC
- 32. 32 TOP db.coll1.update({a:1}) db.coll1.drop() IX IX IX X IX WiredTiger MVCC
- 33. 33 How does MMAP V1 work? • Supports collection concurrency • Takes lock on the collection –Instead of intent
- 34. 34 TOP db.coll1.update({a:1}) db.coll2.update({a:4}) IX IX IX IX X X MMAP V1
- 35. 35 TOP db.coll1.update({a:1}) db.coll1.find({a:4}) IX IS IX IS X S MMAP V1
- 36. 36 WiredTiger • Intents: Global, Database, Collection • Document-level concurrency left to the storage engine
- 37. 37 MMAP V1 • Intents: Global, Database • Locks: Collection
- 38. 38 Conclusion • Storage engines have direct control over concurrency –Completely decoupled from top-level locking –Enabled document-level control
- 39. 39 Conclusion • Retrofitted MMAP V1 to use multi- granularity locks –Support for collection-level locking –Got rid of the global lock usage for journaling
- 40. QUESTIONS?
- 41. EXTRA SLIDES
- 42. 42
- 43. 43
- 44. 44
- 45. 45
- 46. 46
- 47. 47
- 48. 48
- 49. 49
Editor's Notes
- Thank for coming to MongoDB World Introduce myself and what team I work on Say about what I did before MongoDB (Microsoft SQL Server and then AWS) Say what I worked on in MongoDB 3.0 Explain what I will be talking about
- Explain for whom is this talk intended and why is it useful for that particular audience. Leave the questions for the end. I will also be available in the Ask the Experts area.
- What is concurrency control? PAUSE Locks Consistent data (ensuring the correct amount of money is in a bank account) High throughput
- In database systems, concurrency control is all of those things, but also more. Before I tell you what more means, let’s do a quick recap of what’s the logical composition of mongodb.
- You all know what are the logical parts of the database, but let’s recap.
- In MongoDB 3.0: Call out what is the Top part – responsibility of MongoDB Call out what is the Bottom part (actual documents) – responsibility of the storage engine Top part and bottom part talk through the storage engine API PAUSE
- Now that I have highlighted the parts I’ll be talking about…
- Let’s get started.
- CHANGE THE PACE OF TALKING Single “top” lock Protected both the data and the metadata
- Allowed two operations for two different databases to run in parallel (this is for 2.2, 2.4 and 2.6) Synchronized access to the same database Question: How is the instance protected?
- Before I formally define what intents are, let me give a couple of examples of how they work.
- WAIT A FEW SECONDS BEFOR SHOWING THE FIRST BULLET
- Let’s look at this hypothetical scenario. It kind of looks like it works.
- Now, consider an operation which wants to have the entire server in read-only mode. PAUSE BEFORE WALKING THROUGH THE ANIMATION QUESTION: Has it made the server read-only? PAUSE
- Now let’s see how intents help make this correct.
- At some point the insert operation will complete and the S lock will be granted. PAUSE BEFORE CONTINUING
- For completeness, this is the compatibility matrix between locks and intents. PAUSE
- CHANGE PACE OF TALKING This was the world up until MongoDB version 2.6. Now, in 3.0 we wanted to add document-level locking. QUESTION AND PAUSE BEFORE CLICKING: Can you guess what is the first thing we did?
- This started to look like multi-granularity locking, which is a very well studied and tested concept from database systems. Multiple granularity locking requires locks to be acquired in top to bottom fashion.
- EXPLAIN THE BULLETS PAUSE A LITTLE BIT: Now let me show you how multi-granularity locking works in the document world. CHANGE OF PACE
- EXPLAIN THE BULLETS PAUSE A LITTLE BIT: Now let me show you how multi-granularity locking works in the document world. CHANGE OF PACE
- PAUSE A LITTLE BIT AND WALK THROUGH A STORY STORY: Turned out adapting a data structure which was never intended to be concurrent is extremely difficult. Documents are not as independent as we thought (indexes).
- Instead we did something even better. We scrapped this document-level locking…
- … and implemented a storage engine API. PAUSE A BIT TO LET THAT SINK IN
- Let’s see an example.
- Many things can go wrong - such as WT accessing a null pointer.
- This showed how top-level concurrency control works with WiredTiger (and any other storage engine, which supports document-level locking). How does MMAP V1 work? Wait a few seconds.
- Operation 1 and Operation 2 both access completely different parts of the storage engine. QUESTION AND PAUSE: Protection against the collection being dropped?
- AT THE END – protection against the database being dropped remains exactly the same.
- Separated responsibilities for concurrency control between MongoDB and the pluggable storage engine.
- Now I’ll be happy to take questions.
- This slide zooms in on what the Locker API is and what is the state of a particular locker object at runtime. This locker is for an operation, which does some kind of write action (insert/update/delete) for a storage engine, which supports document-level locking. The locker has intent on the GLOBAL resource, intent on the ‘sales’ database resource and intent on the ‘orders’ collection resource. If there were another thread doing a read for example, it would locally have similar state, but instead of IX, it would have IS instead. So these two threads never need to synchronize at the catalog level and all the concurrency control is left to the storage engine instead.
- The lock manager, since it uses named resources and we do not know the names of these resources in advance, is essentially a chained hash table with an entry for each resource containing the resource’s type and name like shown in this diagram. Like any hash table, it contains a fixed (non-growing array) of lock buckets and each bucket is either empty or contains a linked-list of something we call LockHead. There is a single LockBuckets array for the entire instance and there is one LockHead per each lockable resource.
- ----- Meeting Notes (5/21/15 15:35) ----- REMOVE THESE SLIDES ABOUT IMPLEMENTATION
- Let’s recap how memory mapped files work.