What you need to know before you deploy your next MongoDB implementation

1© 2017 Rogue Wave Software, Inc. All Rights Reserved. 1
What You Need to Know
Before You Deploy Your Next
MongoDB Implementation

Presenter
Bill Crowell
Enterprise Architect
Open Source Support
Rogue Wave Software

Who am I?
• Enterprise Architect in the RogueWave Open Source Software group
• 22+ years experience encompassing EDI, insurance, retail, entertainment, banking, and
health care sectors (Fortune 500 companies)
• Worked in various software roles related to full stack development including:
– User interface (Java Server Faces, JavaScript, Spring MVC, Node.js, Angular,
some PHP)
– Middleware (MQ/JMS, WebServices/SOA, and REST)
– Databases (NoSQL and RDBMS)
– Big Data (Apache Hadoop and Spark)
– Security (RBAC and SSO)
– DevOps (Jenkins CI with Docker and SCM)
– Infrastructure
– Testing (performance)
– Training and mentorship
• Worked with proprietary and OSS projects
• Contributed to OSS SSO-project Central Authentication Service
• Primary focus is helping others to applying open source in the enterprise.

Why should I be in this session?
• Your RDBMS or current database system is not meeting your needs.
• You are considering deploying a NoSQL database.
• You are thinking about deploying MongoDB.
• You are currently developing or are in production with MongoDB.
• You have deployed MongoDB and need help!
This presentation is real use cases that are common causes of pain for
MongoDB deployments.
This is where I see customers commonly struggle with Mongo.

What is MongoDB?
• Developed by in 2007 (based in New York).
• Released as open source in 2009 under the GNU Affero General Public
License (AGPL) and Apache License (language drivers)
• Community free edition has been downloaded 30 million times 1
• NoSQL Database: Uses collections of documents instead of rows in a table.
• Written in C++, C and JavaScript
• Dynamic schema design: Provides flexibility and changes usually have
minimal impact on code. Be agile.
1 https://techcrunch.com/2017/09/21/database-provider-mongodb-has-filed-to-go-public/

A bit of history first…
• changes name to MongoDB to align closer to the product name in
August 2013.
• June 2016, MongoDB Atlas is released on the cloud initially on Amazon Web
Services.
• June 2017, MongoDB Atlas includes and . MongoDB
Stitch (Beta) is launched which sits on top of MongoDB and aggregates 3rd-
party REST services like:
• Filed to go public on September 21st, 20171
• Last Thursday, MongoDB stock was traded on the . The stock
skyrocketed 33% on the first day of trading ($34/share). @ $30.50 today.
• The company is worth an estimated $1.6 billion dollars.
1 https://www.sec.gov/Archives/edgar/data/1441816/000104746917006014/a2233365zs-1.htm

Who uses MongoDB?
Source: https://www.mongodb.com/who-uses-mongodb
About 4,300 paid licenses1
1 https://www.sec.gov/Archives/edgar/data/1441816/000104746917006396/a2233556zs-1a.htm
a

MongoDB Use Cases
High-volume data where structure can change…
• Real-time Analytics: Collect data from censors
“ uses it with special screwdriver drills that can measure the torque of the
screws as they install them in airplanes. By tracking torque, they can make airplanes
safer…”1
• Product Catalogs: Categories and inventory
• Reporting: Website traffic logs
• Stock Trading
• Click-stream Ad Campaigns
• Social Media
• Data Analysis: Call records and data mining
• Demographics and Biometrics 2
• Content Management: News, comment fields, photos
• Metadata and Asset Management: Type ahead searches
• Report Aggregation: Merge data from desperate systems into one record.
https://www.mongodb.com/use-cases
1 http://www.businessinsider.com/people-told-the-mongodb-founders-they-were-completely-crazy-2017-10
2 https://techcrunch.com/2013/12/06/inside-indias-aadhar-the-worlds-biggest-biometrics-database/

How is Data Stored in MongoDB?
• JSON is stored in Mongo as binary-encoded serialization format called
BSON (which is transparent to the developer).
• BSON adds extra info to documents to allow for easier traversal.
• BSON provides additional data types not part of JSON spec (Date and
BinData types) and ordered fields: http://bsonspec.org
{
"conference": "ZendCon 2017”
}
x28x00x00x00
x02
conferencex00
x06x00x00x00ZendCon 2017x00
x00
// total document size
// 0x02 = type String
// field name (including collection name)
// field value
// 0x00 = type EOO ('end of object')
ad

Working with BSON in PHP
http://php.net/manual/en/book.bson.php

MongoDB Versions
Free: Open source: https://github.com/mongodb/mongo
• Command-line shell, database and config server, query router, basic troubleshooting
tools, MongoDB Monitoring Service (MMS)
Paid: GUIs and Cloud-based Management (Atlas)
• On-Premise
– Professional: Ops Manager (monitoring, query optimization, automate
configuration
– Enterprise Advanced: Ops Manager, additional storage engines (encrypted and
in-memory), Compass, advanced security (LDAP, RBAC, TLS), auditing
• Cloud-based Service: Atlas
– Free: 512MB Elastic Block Storage (EBS), shared RAM, 3-node replica set,
monitoring and alerts, encryption
– Essential: Starts at 8₵/hour, elastic scaling, snapshot backups (1st GB free, then
$2.50/GB/month), performance panel, enhanced monitoring and alerts, uptime
SLA.
– Professional: Compass, proactive issue detection, schema/database design
support, enhanced support, 2-hour support SLA

MongoDB Total Cost of Ownership
A Total Cost of Ownership Comparison of MongoDB & Oracle1
http://s3.amazonaws.com/info-mongodb-com/TCO_MongoDB_vs._Oracle.pdf
Oracle software maintenance and support costs (as analogous as possible to
MongoDB configurations):
• Oracle Database Enterprise Edition ($47,500 per core) plus Oracle RAC pricing
($23,000 per core), for a total of $70,500 per core. Discounts of 0% for small
deployments to 80% for large.
• Demands 50% of one DBA’s time (small). Requiring 1.5 full-time DBAs (large)
• “We assume a conservative 50% discount on the list price for the smaller and
larger projects. Additionally, we apply a further 50% discount on top of that to
account for Oracle's core processor licensing factor. Amounts to $17,625 per core
for both projects.”
MongoDB software maintenance and support costs:
• Smaller projects: $11,990/server/year
• Larger projects: $10,800/server/year (%10 discount)
• Demands 25% of one DBA’s time (small) and 75% of one DBA’s time (large).
• Assuming 22% of license costs for Oracle.
1 Thanks to Richard Sherrard (Director Product Management in Product Management) for the link!
a
Assumes:
10% of hardware maintenance and support costs of 10% for both.
Hardware maintenance and support costs of 10% of the
hardware purchase price for both MongoDB and
Oracle.

Development versus Deployment
A Tale of Two Roads
The most common issue is not with application development, but with
deploying the application to production and not understanding the
application’s requirements and misconfiguring the software or under
sizing the infrastructure.
Usually this comes at a time when changing the software configuration (and
hardware if on-premise) is very difficult. Code (usually) is simpler to change.

MongoDB Components
• MongoDB Driver: Talks to mongos/mongod
• mongos: Query router for sharded clusters; routing proxy
process.
• mongod: Database including primary/secondary nodes
• mongod: Config server which stores metadata
• mongo: Interactive JavaScript shell (type-ahead)
Zend Server
Driver
Application
Query Router
(mongos)
Data Center 1
(3-node replica set)
Primary (mongod)
Config Servers
Data Center 1 Config Server 1
Secondary
(mongod)
Secondary
(mongod)
Data Center 2
Primary (mongod)
Secondary
(mongod)
Secondary
(mongod)
a

Why would I use MongoDB?
Example: RDBMS One-To-Many Relationship
SELECT * FROM USERS INNER JOIN EMAIL_ADDRESS ON USERS.USER_ID =
EMAIL_ADDRESS.USER_ID;
Much thought must be taken to not change the schema later on. This is hard to do
especially if the tables are shared by other applications.
Field Value
user_id 1
username wcrowell
firstname William
lastname Crowell
Field Value
id 10
user_id 1
email william.crowell@abc.com
Field Value
id 11
user_id 1
email wcrowell@xyz.com
Users Table
Email Address Table

Why would I use MongoDB?
Example: MongoDB One-To-Many Relationship
{
"_id": 1,
"username": "wcrowell",
"firstname": "William",
"lastname": "Crowell",
"email": [
"william.crowell@abc.com",
"wcrowell@xyz.com"
]
}
Document-Based Data Model
• Nested fields allowing for a richer data model and requiring less joins than tables.
• Database changes are easily made: Lack of schema can make your data model more fluid.
• You can collapse a multi-table RDBMS model into a single MongoDB collection using arrays
and nested documents.
• Many-to-many relationships can be modeled as arrays in MongoDB.

MongoDB Levels of Granularity
• A database contains a collection of documents. Top-level named grouping in the system.
• A collection is a group of documents similar to a table.
• A document is similar to a row in a table and is the simplest unit of data.
• A chunk is a group of documents clustered by values on a field. (more on this later)
Important limits: https://docs.mongodb.com/manual/reference/limits/
• 16MB limit on documents.
• Maximum document nesting depth is 100.
Database: drinks
Collection: beers
Chunk: All documents
with field ‘beer’ from a - c
Document:
{”beer”: “Bud”}
Collection: wines
Document:
{”beer”: “Blue Moon”}
Document:
{”beer”: “Bass”}
with field ‘beer’ from d - g
Document:
{”beer”: “Corona”}
Document:
{”beer”: “Dogfish”}
Document:
{”beer”: “Guiness”}
with field ‘wine’ from a - c
Document:
{”wine”: “Blush”}
Document:
{”wine”: “Chardonnay”}
Document:
{”wine”: “Champagne”}
with field ‘wine’ from d - g
Document:
{”wine”: “Dolcetto”}
Document:
{”wine”: “Eiswein”}
Document:
{”wine”: “Frascati”}
What manages how documents and collections are stored?
a

Storage Engines
What is a storage engine?
“A storage engine is the part of a database that is responsible for managing
how data is stored, both in memory and on disk. Many databases support
multiple storage engines, where different engines perform better for specific
workloads. For example, one storage engine might offer better performance
for read-heavy workloads, and another might support a higher throughput for
write operations.”1
Pluggable storage engines are a key feature in many open source projects
(middleware and database) which allows the user to tailor the software to
their application’s needs.
1 https://docs.mongodb.com/manual/faq/storage/#what-is-a-storage-engine

Storage Engines and Locking
Locking in Mongo has come a long way. A single write operation on a
document used to lock the entire Mongo instance. This meant every
database was locked for a single write operation on a document. Very
inefficient. For a Big Data application that is a big deal.
Database locking can directly impact performance. The finer grained a lock
is the better in terms of contention and performance.
All databases, regardless of RDBMS or NoSQL, implement some type of
locking to ensure consistency. The locking mechanism in pluggable storage
engines available with Mongo can differ between implementations.

Storage Engines and Locking
Global: The entire Mongo instance is locked until the lock is released. This includes all
databases, collections, and documents. These locks are very expensive.
Database: Only the database and all collections owned by that database are locked.
Database locks are still expensive.
Collection: Only the documents in the collection (table) are locked. Better.
Document: Individual documents can be locked instead of locking the entire collection
which improves performance for write-heavy applications. Only the WiredTiger storage
engine implements document-level locking.
There are operations in Mongo that can cause locking at each level1.
There are 3 (really 4) storage engines. Each is tailored to different workloads.
1 https://docs.mongodb.com/manual/faq/concurrency/

Storage Engines: MMAPv1 and In-Memory
MMAPv11: Storage engine for memory mapped files. Used in pre-3.x Mongo.
As of version 3.2, is no longer the default. Great for high volume inserts,
reads, and updating existing documents. Writes to the disk every 60 seconds
(customizable) and uses on-disk journal to maintain durability. Uses all free
memory on the machine for the cache and yields to other processes that need
memory. Swaps to disk as needed.
• Uses lots of disk space
• Implements collection-level locking
• Maximum of 32TB (using a 64-byte key)
In-Memory2: Enterprise license only. Does not use any disk. If Mongo is
shutdown, then the data is lost. High-performance. Real-time analytics.
• 50% of RAM – 1GB
1 https://docs.mongodb.com/manual/core/mmapv1/
2 https://docs.mongodb.com/manual/core/inmemory/

Storage Engines: WiredTiger
WiredTiger1 (default): Features:
• Introduced in March 2015 with MongoDB 3.0. More CPU-intensive.
• WiredTiger uses multi-version concurrency control (MVCC) to perform
write locks.
• Locks can be done on a global/instance-level, database, collection, or
document level. Promises 7-10x better write performance.
• Provides on-disk data compression (index and documents). Up to 80%
less storage (snappy or zlib compression)
• Version 3.4+: 50% of RAM – 1GB or 256MB
• Version 3.2: 60% of RAM – 1GB or 1GB
• Encryption at Rest: Enterprise license only. HIPPA-compliant. AES256-
CBC (default). 3.2+ only.
1 https://docs.mongodb.com/manual/core/wiredtiger/

MongoDB PHP Drivers
PHP MongoDB Driver Homepage
https://docs.mongodb.com/ecosystem/drivers/php/
http://php.net/manual/en/set.mongodb.php
1) MongoDB Driver for PHP from PHP Extension Community Library (PECL)
https://pecl.php.net/package/mongodb
• Thin (bare-bones) limited functionality driver
• Currently maintained (version 1.3.1 released October 16th, 2017)
2) MongoDB PHP Library
https://docs.mongodb.com/php-library/current/
• Wrapper for the lower-level PHP driver above
• Recommended fully-featured driver
• Requires PHP 5.4+, libbson, and libmongoc and OpenSSL
• Documentation: https://docs.mongodb.com/php-library/current/

MongoDB PHP Drivers
MongoDB Compatibility1
The following compatibility table specifies the recommended version(s) of the
MongoDB PHP driver for use with a specific version of MongoDB.
1 https://docs.mongodb.com/ecosystem/drivers/driver-compatibility-reference/#php-driver-compatibility
PHP Driver MongDB 2.4 MongoDB 2.6 MongoDB 3.0 MongoDB 3.2 MongonDB 3.4
PHPLIB 1.1 +
mongodb-1.2*
Yes Yes Yes Yes Yes
PHPLIB 1.0 +
mongodb-1.1*
Yes Yes Yes Yes
mongodb-1.1* Yes Yes Yes Yes
mongodb-1.0* Yes Yes Yes
mongo-1.6** Yes Yes Yes
mongo-1.5** Yes Yes
mongo-1.4** Yes Yes
mongo-1.3** Yes
*New driver
**Legacy driver

MongoDB PHP Drivers
PHP Language Compatibility1
The following compatibility table specifies the recommended version(s) of the MongoDB PHP driver
for use with a specific version of PHP/Zend2.
1 https://docs.mongodb.com/ecosystem/drivers/driver-compatibility-reference/#reference-compatibility-language-php
2 https://framework.zend.com/blog/2017-06-06-zf-php-7-1.html
2 https://zend18.zendesk.com/hc/en-us/articles/217058968-PHP-Versions-and-APIs
3 Clark Everetts
PHP Driver PHP 5.6
Zend 8.0
PHP 5.6
Zend 8.5LTS
PHP 7.0.15
Zend 9.0.2
PHP 7.1.3/7
Zend 9.1
HHVM 3.12 HHVM 3.15
mongodb-1.2* Yes Yes Yes Yes Yes Yes
mongodb-1.1* Yes Yes Yes Yes Yes
mongodb-1.0* Yes Yes Yes
mongo-1.6** Yes Yes
mongo-1.3-1.5** Yes Yes
*New driver
**Legacy driver
Note: Support for PHP 5.6 is available via Zend Server 8.5, not 8.03.
HHVM - HipHop Virtual Machine (Facebook)

Demo: PHPLIB with MongoDB
http://php.net/manual/en/mongodb.tutorial.library.php
Install Composer (package manager).
Install the library: composer require mongodb/mongodb
Creates a bootstrap for dependency classes: vendor/autoload.php
Entire library: https://docs.mongodb.com/php-library/current/
Libraries used:
PHP 7.1.10
Apache 2.4.25
MongoDB PHPLIB Extension 1.2.9
http://localhost/phpinfo.php
2 files: list.php and create.php
No schema is defined (e.g.
collections/documents).
d

Hardware: Memory
The most important part of a Mongo deployment is RAM and not CPU. Mongo uses much less
CPU than a RDBMS. Insufficient RAM is the most common performance issue. RAM will contain
your indexes and working set.
The working set for an application should be able to fit comfortably in memory. This is the amount
includes:
• Data or collections: Containers for like documents stored in extents. Number of pages accessed
per second by active users on the system.
• Indexes on the collections.
Also consider the following:
• The period of time the data and indexes need to be retained.
• Connection pooling (1MB per active thread).
• Account for fragmentation.
• Operations on the data including sorting and aggregation (map reduce).
This does not mean all of the documents and indexes in the database have to fit within RAM. Only
documents and collections (a majority) your application accesses are part of the working set.

Hardware: Memory
1For example, you have a year’s worth of data, and assume each month is
1GB of data totaling 12GB. For every month of data you have 1GB of
indexes totaling 12GB. If your application is accessing 12 month’s worth of
data, then your working set is: 12GB of data and 12GB of indexes = 24GB.
If you had 8GB RAM and your application started accessing 6 month’s
worth of data (6GB data + 6GB indexes), then your working set would start
to exceed the available RAM and poorly perform as time progresses (as
more data is accessed).
You want to prevent Mongo from paging in and paging out documents in your
working set.
1 https://stackoverflow.com/questions/6453584/what-does-it-mean-to-fit-working-set-into-ram-for-mongodb

Hardware: Memory
How much RAM do I need for my deployment?
It depends on:
• An application’s typical use cases/access patterns. Every application is
different. Requires understanding your application.
• How well the queries are indexed. If queries are doing full collection
scans, then more memory will be used.
Always allow room to grow.
Best practice: Create automated test scripts (e.g. PHPUnit1) focusing on
repeatability and keep them updated as the application changes. Monitor the
application using the approaches mentioned here. Take time to do this as it
will pay dividends in the long run. Sell this to your management team that
you need to do this for the application’s success.
1 https://zendframework.github.io/zend-test/phpunit/

Hardware: Paging
You do not get a notification when Mongo is paging. So, how do I know if I am paging?
Run the following from a mongo prompt:
db.serverStatus().extra_info
{
"page_faults" : 6137
}
Example, client ran a perf test with 8GB of RAM:
{
"page_faults" : 353
}
After increasing to 16GB of RAM:
{
"page_faults" : 76
}

Hardware: Disks
If Mongo must access disk, then prefer RAID10 or SSD-based VM over HDD
(which will cost more).
“Solid state drives (SSDs) can outperform spinning hard disks (HDDs) by
100 times or more for random workloads” 1 and are increasingly more
affordable.
Maximizing MongoDB Performance on AWS
https://www.mongodb.com/blog/post/maximizing-mongodb-performance-on-
aws
1 https://docs.mongodb.com/manual/core/write-performance/#storage-performance

Security
Disabling SELinux
“Problems have been reported when using MongoDB with SELinux enabled.
To avoid issues, disable SELinux when possible.”1
There are methods on how to work around this without disabling SELinux.
Try audit2why on the /var/log/audit/audit.log to view the violations, and build
custom policies with audit2allow2.
1 https://docs.mongodb.com/manual/tutorial/install-mongodb-on-red-hat/
1 https://docs.mongodb.com/manual/administration/production-notes/#recommended-configuration
2 https://serverfault.com/questions/770227/selinux-setup-for-mongodb

Security
Do enable MongoDB security!
Tutorial: Enable Authentication
https://docs.mongodb.com/manual/tutorial/enable-authentication/
Major security alert as 40,000 MongoDB databases left unsecured (February
2015)
https://www.techworm.net/2015/02/major-security-alert-40000-mongodb-
databases-left-unsecured.html
This was discovered by 3 students in Germany by checking TCP port 27017.
You can easily go out on (search engine for IoT) and do a search on
MongoDB:
shodan download --limit -1 mongodb "product:MongoDB”

Security
It's Still the Data, Stupid! (December 2015 by John Matherly)
https://blog.shodan.io/its-still-the-data-stupid/
“At the moment, there are at least 35,000 publicly available, unauthenticated
instances of MongoDB running on the Internet... There's a total of 684.4 TB of
data exposed on the Internet via publicly accessible MongoDB instances that
don't have any form of authentication.”
MongoDB ransacking starts again: Hackers ransom 26,000 unsecured
instances (September 5th, 2017 by Liam Tung)
http://www.zdnet.com/article/mongodb-ransacking-starts-again-hackers-ransom-
26000-unsecured-instances/
“Three groups of hackers have wiped around 26,000 MongoDB databases over
the weekend and demanded victims to pay about ~$650 (495 pounds) to have
them restored.”

Operating System Tuning: THP
Disable Transparent Huge Pages (THP) for Linux:
https://docs.mongodb.com/manual/tutorial/transparent-huge-pages/
Why?
When an application is accessing memory, you want memory to be accessed
contiguously. Databases tend to have sparse rather than contiguous memory
access patterns.
Set readahead to 0 Regardless of Storage (e.g. SSD, HDD)
https://docs.mongodb.com/manual/administration/production-notes/#readahead
Why?
“Setting a higher readahead benefits sequential I/O operations1.” MongoDB
accesses the disk in random patterns. Increasing this value can degrade
performance.

Operating System Tuning: File System Types
Use XFS and not EXT4 or NFS
https://docs.mongodb.com/manual/administration/production-notes/#kernel-and-file-systems
Why?
XFS is arguably better for concurrent writes. It is best to run mongoperf on your system to check.
“With the WiredTiger storage engine, use of XFS is strongly recommended to avoid performance issues
that may occur when using EXT4 with WiredTiger1.”
“Avoid using NFS drives for your dbPath. Using NFS drives can result in degraded and unstable
performance1.”
XFS vs EXT4 – Comparing MongoDB Performance on AWS EC2
https://scalegrid.io/blog/xfs-vs-ext4-comparing-mongodb-performance-on-aws-ec2/
“In performance terms, XFS is indeed a force multiplier when paired with high speed disks that it can take
real advantage from. For low to mid-end systems, it doesn’t seem to be able to do much to improve your
performance.”
Windows Only: Do not use FAT file system. Use NTFS instead.
1 https://docs.mongodb.com/manual/administration/production-checklist-operations/#filesystem

Operating System Tuning: Disk Space
Disk Space
Have enough disk space for the size of your data, indexes, and log files plus
plenty of room for expansion.
Running db.stats() (proactively) and looking at the following fields
should give an idea:
• avgObjSize: Average size of files allocated for this database.
• dataSize: Size of BSON objects in the database.
• storageSize: Total space allocated for collection extents. Extra space
reserved for collection growth and unallocated deleted space.

Operating System Tuning: NUMA
Disable NUMA (Non-Uniform Access Memory)
https://docs.mongodb.com/manual/administration/production-notes/#mongodb-and-numa-hardware
What is NUMA?
It is used to increase processor speed on a multi-core system without increasing load on the processor bus. This is where
you would have 2 memory pools where each core has some degree of proximity to each memory pool on the bus.
Why disable NUMA?
It can cause memory to paged in and out unnecessarily. Mongo will complain if it expects NUMA is enabled. Accessing
local memory is faster than remote.
How is NUMA disabled?
On Windows, it can be configured in the BIOS.
On Linux:
echo 0 | sudo tee /proc/sys/vm/zone_reclaim_mode
Or:
sudo sysctl -w vm.zone_reclaim_mode=0
Then any Mongo application (e.g. mongod, mongos, mongo) must be started with numactl:
numactl --interleave=all /usr/bin/mongod --quiet -f /etc/mongod.conf run
Note: This may not be necessary when bound to a single NUMA node:
See: https://jira.mongodb.org/browse/SERVER-25984
8GB
RAM
CP
U 0
CP
U 2
CP
U 4
CP
U 6
CP
U 1
CP
U 3
CP
U 5
CP
U 7
8GB
RAM
bus
a

Operating System Tuning: atime
Disabling Last Accessed Time
Disable last accessed time (atime) in the file system table (/etc/fstab)
entries for volumes containing Mongo database files.
Why disable last accessed time?
This can provide a significant performance improvement unless you have an
application that relies on atime.
How is atime disabled on Linux?
/dev/mapper/datavg-datalv /apps xfs defaults,noatime 0 0
/dev/mapper/appvg-appsloglv /apps/logs xfs
defaults,noatime 0 0

Tools: Finding the Bottleneck
iostat1: Number of accesses over time to the disk.
Example: iostat –xmt 1
• %util: This is the most useful field for a quick check, it indicates what percent
of the time the device/drive is in use.
• avgrq-sz: Average request size. Smaller number for this value reflect more
random IO operations.
vmstat2: How much data being used and fitting into memory and page faults.
mongostat3: Provides a quick overview of the status of a currently
running mongod or mongos instance. Similar to vmstat.
Profiling4: mongod --profile <level:0> --slowms <milliseconds: 100>
1 https://docs.mongodb.com/manual/administration/production-notes/#iostat
2 https://docs.mongodb.com/manual/faq/diagnostics/#how-do-i-read-memory-statistics-in-the-unix-top-command
3 https://docs.mongodb.com/manual/reference/program/mongostat/
4 https://docs.mongodb.com/manual/reference/program/mongod/#bin.mongod
http://edgystuff.tumblr.com/post/81219256714/tips-to-check-and-improve-your-storage-io

Tools: Finding the Bottleneck
mongoperf1: Checks disk I/O performance independently of MongoDB.
mongoperf can overstate performance problems in ext-X filesystems
https://jira.mongodb.org/browse/SERVER-13417
“People who use mongoperf to compare XFS and ext-X might get results that overstate the benefits of
XFS…The workaround is to let mongoperf use multiple files. That would also make the mongoperf load
more realistic given mongodb will use many files.”
Set the file size, # of threads, read/write operations in a .conf file:
{ nThreads:1024, fileSizeMB:1000, mmf:false, r:true, w:true, syncDelay:60 }
mongoperf < ./mongoperf.conf
Tips to check and improve your storage IO performance with MongoDB
http://edgystuff.tumblr.com/post/81219256714/tips-to-check-and-improve-your-storage-io
mongotop2: Tracks time spent reading/writing per namespace (database/collection).
MongoDB Monitoring Service (MMS): Graphical user display for monitoring, backup, and deployment.
http://api.mongodb.com/mms/
1 https://docs.mongodb.com/manual/reference/program/mongoperf/
2 https://docs.mongodb.com/manual/reference/program/mongotop/

Indexes
What makes a good index?
• The query optimizer chooses the most efficient query plan for the available indexes.
• An index an a unique ID value field is very selective. When multiple indexes are
involved, Mongo evaluates the indexes and uses the more highly selective index.
• Will the query planner use the index? If the fields named in the query are part of the
index, then yes.
• The index should be selective to narrow down the results for a given key.
• Index on a boolean field only are usually not selected because only two possible values
(true or false) will not narrow down the selection.
• Mongo keeps statistics on index hits to see if a key match points to a few or many
documents.
• Mongo can select the wrong index to use meaning that the other index would perform
better.
– Mongo caches the selection and may remember a non-optimal choice.
– Statistically an index may look good, but another might perform better.

MongoDB Indexes
Indexes are implemented as a B-tree data structure1.
Each collection has an index on _id automatically.
MongoDB allows 64 indexes per collection2.
Run explain on your application’s main queries to determine if they are using indexes.
See if a column is indexed;
db.beers.find( { "beer": "Stickee Monkee" } ).explain()
If queryPlanner.winningPlan.stage equals:
• “IDHACK”: Uses special ID index strategy to retrieve the documents for this query.
• “COLLSCAN”: This is a collection scan which means the query had to visit every document
in the collection. For large databases, MongoDB would have to page all of the
documents into memory which is very slow.
• “IXSCAN”: Index scan.
• “FETCH”: Document retrieval (IXSCAN index hit).
1 https://docs.mongodb.com/manual/indexes/#create-an-index
2 https://docs.mongodb.com/manual/reference/limits/#Number-of-Indexes-per-Collection
d

MongoDB Indexes
Get execution statistics:
db.<collection>.find({_id:ObjectId("595411070a797e7aaeff2733")}).exp
lain(‘executionStats’)
Look at the following:
• executionStats.totalDocsExamined (previously named nscannedObjects). If
this number is high, then an index was not used.
• executionStats.totalKeysExamined: The number of index entries scanned. If it is
zero, then an index was not used in the query.
• executionStats.executionStages.nReturned: # of results the query returned.
Reference:
Explain results:
https://docs.mongodb.com/manual/reference/explain-results/
Get indexes:
db.<collection>.getIndexes()
d

Indexes
Create an index:
db.<collection>.createIndex({<field>: <1 for ascending, 2
for descending>})
• Creates an index and collection if it does not exist.
• Direction can be 1 for ascending, -1 for descending
“If a write operation modifies an indexed field, MongoDB updates all
indexes that have the modified field as a key1.”
d
1 https://docs.mongodb.com/manual/faq/indexes/#how-do-write-operations-affect-indexes

2 Types of Scaling
The traditional way to scale a RDBMS is to add more hardware. Eventually,
a price or scale limit is hit making this approach unfeasible.
• Vertical: Adding CPU, faster disks, additional memory on a bare-metal
machine. Simple to implement.
• Horizontal: Pooling resources to distribute the load and data across
multiple machines. Cloud solutions allow us to scale up or down
depending on need. One way to accomplish this is through sharding.
*It is essential to understand your application’s requirements regarding
latency and throughput, the volume and type of data, and the period of
time the data is kept.

Replication and Replica Sets
• Replication
– Protects your data but is not a backup.
– Provides redundancy by synchronizing across nodes.
– Disaster recovery: Automatic failover when your primary node fails.
– Used to scale reads.
– Goal: Always have one full copy of your data at all times.
• What is a replica set?
– Replication where a configured group of nodes automatically
synchronize their data and fail over when a node is no longer
available.
– A recommended minimum of 3 nodes

Replica Set: What does it look like?
Driver
Application
Data Center 1
Secondary
Primary
Secondary
• Typical setup: A primary (writable) and 2
secondary (read-only nodes make up a 3-
node replica set.
• Near 100% uptime is critical: A loss of 1 node
does not take down the entire replica set.
• Primary and secondary are copies of each
other.
• Writes can only be done on the primary.
• Reads can be performed on the secondary.
• A priority can be set that favors a node to
become a primary in case of failover1.
• If there are 2 data centers with 2 nodes in
one data center, then use one of the data
centers for DR.
• Not scalable: Primary is largest common
denominator (memory, drive space)
Asynchronous
Replication
Data Center 2
(DR)
Secondary
Secondary
1 https://docs.mongodb.com/manual/tutorial/adjust-replica-set-member-priority/
2 https://docs.mongodb.com/manual/reference/replica-configuration/#rsconf.members[n].priority
a
Ok, that’s great. How do we scale horizontally?

Sharding
• What is sharding?
– A shard contains a subset of the sharded data. Shards are
deployed as a replica set1.
– Distributes the load by partitioning documents into smaller
manageable pieces on less powerful machines so one machine does
not have to store everything.
– Partitioning is abstract to the application.
– Used to scale writes. Most operations are inserts and updates for
large-volume applications.
– Adds overhead and complexity: Moving data off overloaded shards
takes time and resources.
– Balancing and redistribution of the data across the shards is done
automatically.
1 https://docs.mongodb.com/manual/core/sharded-cluster-components/

When to use sharding?
• Geo-locality: Support geographically distributed deployments of optimal
user experience for customers in many locations.
– Lower network latency.
– Great for mobile applications.
• Scalability: The working set growth is unbounded and exceeds the
available RAM on the largest node/server. When not enough resources
exist on a single machine and there is a lot of I/O, configuring sharding and
spreading the load across several machines may reduce the load on an
individual server.
• Hardware Optimization: Enhancing performance without the cost.
• Low Recovery Times: Mitigate the impact on from server failure.

Sharding
• When not to use sharding?
– New deployments: Prototype first with a large amount of data. Use
a replica set first, and then move to sharding later on if required.
– A prototype (or performance test) allows for a more accurate
estimate on the hardware.
– If you determine sharding is needed within the first 6 to 12 months
after deployment of a project, then plan for a sharded cluster.
– An application has a tendency to access data not local.
– Infrastructure limitations: Sharding requires several servers.
– The application has one shard.

Should I use sharding in this use
case?
• I own a trucking company that owns 1,000 18-wheelers delivering steel
beams to construction companies. The U.S. Department of Transportation
enacted the Transportation Recall Enhancement, Accountability, and
Documentation Act (TREAD) which required Tire Pressure Monitoring
System (TPMS) be installed on each truck to automatically measure tire
pressure for each tire in 1-minute increments using sensors while each
truck is moving. The data is uploaded to the home office where it is
required to be kept for 3 years.
a

Sharding Deployments
• A sharded cluster contains:
– Shards store the application data. In a sharded cluster, only the mongos
routers or system administrators should be connecting directly to the
shards.
– mongos query routers cache the cluster metadata and use it to route
operations to the correct shard or shards.
– Config servers persistently store metadata about the cluster, including
which shard has what subset of the data.
• Guidelines
– Each member of a replica set, whether it’s a complete replica or an
arbiter, needs to live on a distinct machine.
– Replica set arbiters are lightweight enough to share a machine with
another process. Arbiters can be placed on a mongos query router.
– Config servers can optionally share a machine. The only hard
requirement is that all config servers in the config cluster reside on
distinct machines.

Sharding Caveats
“My application performed fine with a replica set, and now my application
performs terribly with a sharded cluster! Why?”
Initial load of data is not split and as a result written to one shard. As data
is written more chunks are created. Chunks are is a group of documents
clustered by values on a field (e.g. list of beers between letters a – d). As
more chunks are created and certain thresholds are met, Mongo moves the
data to other shards/servers to balance out. This process called a
migration.
What does this look like?

Sharding Migrations
Driver
Application
Query Router
(mongos)
Query Router
(mongos)
…
Chunk
64MB
Shard 1 Shard 3Shard 2 Shard 4
• A balancer process (automatically)
monitors the number of chunks on each
shard.
• The first data load all writes go to one
shard/server.
• As data is loaded the chunks are created
and segmented as the chunks grow.
Chunk
64MB
Chunk
64MB
Chunk
64MB
Chunk
64MB
Chunk
64MB
Chunk
64MB
Chunk
64MB
Chunk
64MB
Chunk
64MB
• When a shard fills to capacity, the
balancer issues a migration.
• Migrations take time because data is
being moved from server to server.
• The goal: Equal # of chunks per shard.
Chunk
64MB
Chunk
64MB
a

Sharding Caveats
• The data can be pre-split to mitigate this issue (using either a command1 or
script2). The chunks will be created and balanced ahead of time. When data
is written then the data will be distributed according to your sharding strategy.
• Be proactive! Your application’s performance can degrade. A good rule of
thumb is to plan to add a new shard at least several weeks before the indexes
and working set on your existing shards reach 90% of RAM.
• Fixed-size collections (or capped collections) cannot be sharded.
• Use care with mapReduce and aggregation functions with sharding as they
tend to globally lock the database:
Remove unnecessary global lock during "replace" out action
https://jira.mongodb.org/browse/SERVER-13552
“During map-reduce operation there are unnecessary global lock used and
should be removed.”
1 https://docs.mongodb.com/manual/reference/command/shardCollection/
1 https://docs.mongodb.com/manual/tutorial/migrate-chunks-in-sharded-cluster/
2 https://docs.mongodb.com/manual/tutorial/create-chunks-in-sharded-cluster/

Deployment Strategy
Zend Server
Driver
Application
Query Router
(mongos)
Data Center 1
Secondary
(shard 2)
Primary (shard 1)
Secondary
(shard 3)
Zend Server
Driver
Application
Query Router
(mongos)
Zend Server
Driver
Application
Query Router
(mongos)
Query Router
(mongos)
Config Servers
Data Center 2 Data Center 3
Primary (shard 2)
Query Router
(mongos)
Primary (shard 3)
Query Router
(mongos)
Secondary
(shard 1)
Secondary
(shard 3)
Secondary
(shard 1)
Secondary
(shard 2)
• Query routers can be placed at the application
server or shard primary. There are typically
more mongos servers than application servers
(1-to-1 mongos/mongod).
• Config servers are a replica set.
• Setup provides Redundancy and HA.
• Shards spread across data centers.
a

Shard Keys
• A shard key is set by the developer/data modeler which describes a
range of values in how a data set should be partitioned. It determines
how a collection of documents are spread over partitions.
• Data is segregated into chunks by the shard key. The chunks are
distributed across shards residing across multiple servers.
• Data modelers can struggle with defining a good shard key which can
cause problems with performance later on. You cannot change a shard
key after sharding a collection. Choose the shard key wisely. You
would need to dump and restore the data to repartition the data for a new
shard key. Not trivial.
Customers commonly pick the wrong shard key and get stuck.

Good Shard Keys
Good shard keys exhibit 5 properties:
• Cardinality: For set A = { “East”, “Central”, “West” } the set has 3 elements
giving it a cardinality of 3. This means I can have only 3 chunks the
balancer can create. This can limit horizontal scaling in the cluster.
Although, high cardinality does not mean the data will be distributed evenly
across the cluster.
• Write Distribution: Writes should be distributed evenly (as possible)
across the cluster.
• Query isolation: Majority of queries for the same collection result in
targeting one (or a few) particular shard(s).
• Reliability: Do some or all documents get affected after one shard goes
down?
• Index locality: Locality is how the indexes are accessed. Do we have to
page in the entire index each time to query?

Bad Shard Keys
• Data center: This would be a good shard tag, but does not have good
cardinality. If we had 3 data centers, then the data can be split up into 3
chunks. There will be large chunks that cannot be split resulting in one shard
with more data than the others.
• Customer IDs: May not be as well distributed if some customers are larger
than others (cardinality). Depends on use case.
• Timestamp: Bad write distribution because writes could be targeted more to
one chunk than the others. Example: My website activity is greatest between
the hours of 10am and 4pm.
• Hashed timestamp: An MD5 hash of a timestamp provides random
distribution, but would cause the queries to be scattered across the cluster.

Ranged Sharding
MongoDB provides 3 types of sharding strategies:
• Range1/User-defined: Shard key defines a range of values are segmented
into sequential chunks. Documents with “close” shard key values are likely to
be in the same chunk or shard. For example: A primary key for a collection of
documents is an auto-increment integer. 0 – 10 go to shard 1, 11 – 20 go to
shard 2, 21 – 30 go to shard 3, and so on. Composite keys are supported with
this strategy.
Deploy Sharded Cluster using Ranged Sharding
1 https://docs.mongodb.com/manual/tutorial/deploy-sharded-cluster-ranged-sharding/
Shard 1
{ x: minKey }
Shard 2 Shard 3 Shard 4
{ x: maxKey }{ x: 11} { x: 21} { x: 31}
Chunk 1
1
2
Chunk 2
7
8
Chunk 1
11
17
Chunk 1
21
Chunk 2
29
30
Chunk 1
35
38

Hashed Sharding
• Hashed1: Subset of range sharding. MD5 hash is applied on the key to
ensure data is spread randomly within MD5 range of values. Composite
keys are not supported.
Deploy Sharded Cluster using Hashed Sharding
1 https://docs.mongodb.com/manual/tutorial/deploy-sharded-cluster-hashed-sharding/
Shard 1 Shard 2 Shard 3 Shard 4
{ x: maxKey }{ x: d3d9446802...}
Chunk 1
c4ca4238a0b923820dcc509a
6f75849b
MD5 Hash Function
1
{ x: minKey }
Chunk 1
d3d9446802a44259755d38e
6d163e820
c20ad4d76fe97759aa27a0c
99bff6710
Chunk 1
8e296a067a37563370ded05
f5a3bf3ec
Chunk 1
Chunk 2
{ x: 34173cb38f07f...} { x: c0c7c76d30bd3...}
a

U.S.
U.S.
U.S.
U.S.
U.S.
Tag-Aware Shard Keys
• Subset of shards are tagged and assigned to a sub-range of the shard-key. This creates
zones where each zone with one or more shards. Most relevant data should reside on
shards geographically closest to the application servers.
AMER
Secondary
Primary
Secondary
EMEA
Secondary
Primary
Secondary
APAC
Secondary
Primary
Secondary
United States
Secondary
Primary
Secondary
ZONE:
U.S.
Canada
Mexico
England
Germany
y
France
India
China
Japan
TAGS:
U.S.
U.S.
U.S.
U.S.
EMEA/APAC
Secondary
Primary
Secondary
a

Which type of shard key should I use?
Use Cases
• Scalability: Range or hash
• Geo-locality: Tag-aware
• Hardware Optimization: Tag-aware
• Low Recovery Times: Range or hash
Combining tag-aware with range or hash is acceptable.
A good shard key usually has the following (in this order):
• A random component like a universally unique identifier (UUID).
– Note: Shard key sizes are limited to 512 bytes.
WiredTigerIndex::insert: key too large to index,
failing 1060 { : new Date(1500393090062)…
• An increasing sequence like a timestamp.

Links
MongoDB Documentation
https://docs.mongodb.com/
MongoDB Limits and Thresholds
https://docs.mongodb.com/manual/reference/limits/
MongoDB Production Checklist
https://docs.mongodb.com/manual/administration/production-checklist-
operations/
Who uses MongoDB?
https://www.mongodb.com/who-uses-mongodb
MongoDB IRC Chat
irc://irc.freenode.net/#mongodb

Links
API Documentation for MongoDB Drivers
https://api.mongodb.com/
MongoDB Integration and Tools
https://docs.mongodb.com/ecosystem/tools/
MongoDB Drivers (All)
https://docs.mongodb.com/ecosystem/drivers/
mongo Keyboard Shortcuts
https://docs.mongodb.com/manual/reference/program/mongo/#keyboard-
shortcuts
Back Up a Sharded Cluster with Database Dumps
https://docs.mongodb.com/manual/tutorial/backup-sharded-cluster-with-
database-dumps/

Links
MongoDB in Action Manning Forum
http://manning-sandbox.com/forum.jspa?forumID=677
MongoDB Videos
https://www.mongodb.com/presentations/
MongoDB University
https://university.mongodb.com/
MongoDB Events
https://www.mongodb.com/events/
Community Support Forum
http://groups.google.com/group/mongodb-user

Links
ServerFault
https://serverfault.com/questions/tagged/mongodb
Stack Overflow
https://stackoverflow.com/questions/tagged/mongodb

Q/A
Questions?

Thank you!
Be inspired to do something BIG!

Slides Removed Due to Time Constraints

Security: Roles
• Built-In Roles
https://docs.mongodb.com/manual/reference/built-in-roles/
– userAdminAnyDatabase: Complete access
– backup: Ability to run mongodump
• User-Defined Roles

Data Archival
Backing up and restoring MongoDB1
• Best for replica set/sharded cluster
– MongoDB Cloud Manager (Enterprise Advanced only)
– Ops Manager: On-premise (Enterprise Advanced only)
• mongodump/mongorestore
– Not the best solution for larger databases.
– Does impact mongod performance while running.
• File system snapshot
– Snapshots: Not specific to MongoDB
• More efficient but volume must support snapshots (Logical Volume Manager on Linux)
• Must have journaling enabled and journal must be on the same logical volume.
Journaling is enabled by default on 64-bit builds (--journal).
• Sharded cluster backup is more difficult: “disable the balancer and capture a snapshot
from every shard as well as a config server at approximately the same moment in time”
– cp or rsync
Remember to test your backup and restore process no matter what strategy you choose.
1 https://docs.mongodb.com/manual/core/backups/

Data Archival
Journaling
• Journaling is enabled by default.
• Prevents data corruption.
• Every write is flushed to the journal every 100 milliseconds.
• Journaling can be disabled to increase performance; however, replication
should be enabled to ensure durability.

Data Archival
mongodump/mongorestore1
• Excludes the local database in its output.
• Requires the backup role if security is enabled.
• mongod must be running since storage engines have different file layouts.
• For sharding clusters, mongos (sharding router) must be running.
• Suggestion: Create a specific operating system user to run the backup and
only give access to the backups to the backup user.
1 https://docs.mongodb.com/manual/tutorial/backup-and-restore-tools/

Data Archival
Examples:
Export and import all databases (excluding local and admin)1:
mongodump
mongorestore <location of dump>
Drop all documents before inserting:
mongorestore --drop <location of dump>
Export a database:
mongodump --db <database>
Export a specific database and collection:
mongodump --db <database> --collection <collection>
Restore a specific database and collection:
mongodump --drop --db <database> --collection <collection> <location of .bson file of
collection>
Export a specific database and collection with security enabled:
mongodump --db <database> --collection <collection> --username <username> --password
<password>
1 https://docs.mongodb.com/manual/tutorial/backup-and-restore-tools/

Data Archival
• Protecting the backups
– Backups are stored in BSON/binary format.
– Binary does not mean encrypted.
– bsondump can be used to read backups:
<$MONGO_ROOT>/bin/bsondump

Indexes
Query Selectors
$in and $all can take advantage of indexes. $ne and $nin cannot unless used with another operator.
Cannot use index and uses a collection scan instead:
{timeframe: {$nin: ['morning', 'afternoon']}}
Can use index: {timeframe: 'evening'}
JavaScript Query Operators
This statement cannot use an index and is singled-threaded:
db.reviews.find({'$where': "this.helpful_votes > 3"})
Regular Expressions
MongoDB is compiled with PCRE (Perl Compatible Regular Expressions)
Unless a prefix-style query is used then a regular expression query cannot use an index.
Example of a prefix-style query “starts with”: db.users.find({'last_name': /^Sm/})
Using case-insensitive flag against an index field nullifies the use of an index because indexes are case-sensitive.
Use the new text search capability or use an external search engine instead.
Another option is to store the searched field as all lower case and search against that field making it an indexed case-
insensitive search.
The modulo operator $mod will not use an index.

Tools
mtools: https://www.mongodb.com/blog/post/introducing-mtools
A set of Python scripts to parse and filter MongoDB log files.
• mloginfo: Log file analysis. This is the most useful tool within mtools.
mloginfo mongod_prod.log –queries
• mlogfilter: Filtering tool used to pipe output to other tools. Helps
narrow down a search.
• mplotqueries: Creates a graph. Has some bugs.
mplotqueries mongod_prod.log
• mlogvis: Web-based version of mplotqueries.

Hardware: Faults and Memory
2 types of page faults:
• Soft fault: Moves memory pages from one list to another (e.g. OS file
cache).
• Hard fault: Mongo accesses the disk to store or retrieve data. This is
what we are trying to prevent.

Operating System Tuning: Disk Space
Check ulimit settings:
https://docs.mongodb.com/manual/reference/ulimit/
Recommended settings:
• -f (file size): unlimited
• -t (cpu time): unlimited
• -v (virtual memory): unlimited
• -n (open files): 64000
• -m (memory size): unlimited
• -u (processes/threads): 64000

Indexes
Index are created immediately when declared.
Indexes by default are created in the foreground which is a locking activity
on the collection.
To create an index in the background1:
db.<collection>.ensureIndex({…}, { background: true })
• Background index builds allow reads and writes while the index is being
built.
• Background index builds take longer and indexes can be larger than if they
were built in the foreground.
1 https://docs.mongodb.com/manual/reference/method/db.collection.createIndex/

Indexes
How do we remove old documents automatically?
Time-To-Live (TTL) Indexes
Create a TTL index on a collection to expire documents after 30 days:
db.<collection>.ensureIndex({ time: 1}, {expireAfterSeconds:
60*60*24*30})
• Only one TTL index is allowed per collection.
• The field the TTL index is declared on must be a date type and cannot be the _id field.
• expireAfterSeconds specifies how old the document can be before it is removed.
Do not be disappointed if a document is not deleted/expired when you expect.
• An internal job runs every 60 seconds and queries for documents whose date is older than that
interval relative to the current date and time.
– This means that documents could live 60 seconds longer than the expiration date.
• If you have a large amount of documents that need to expire when you first create a TTL index,
then this can cause a bottleneck because deletion is a write activity. Write activities require I/O
and locks which require time. It may make sense to delete documents manually in batches and
then implement a TTL index. This has less impact.

Indexes
Forcing an index to be rebuilt:
An index rebuild can be triggered by the following:
db.<collection>.reIndex()
Or by issuing a repair which will cause all indexes to be dropped and recreated:
db.repairDatabase()
Caveat: Global write lock! 1
Compaction:
Compact defragments documents (or releases unused disk space) and rebuilds indexes.
Compact is a blocking operation, and run this only during a maintenance window on a
primary or an offline secondary in a replica set. To compact a collection:
db.runCommand( { compact: ‘<collection’ } )
Note: Compaction does run differently depending on storage engines.
d

Production Replica Set Configurations
• 2 Replicas and 1 Arbiter
– Arbiter runs on an application server.
– Both replicas get their own machine.
• Arbiters
– Replica sets vote on a new primary if the old primary goes down.
– Lightweight mongod process that break ties if the nodes cannot
agree on a new primary.
– Use an arbiter if there are an equal number of nodes between
data centers.
– Does not replicate data.
Ok, that’s great. How do we scale horizontally?

Indexes
Compound Indexes
• By having more than one key, it increases selectivity. Therefore, multiple queries can
benefit.
• Determine if the index really speeds up a query by look at explain().
• Minimize index size and count: Having less indexes can be a performance gain which
requires less space and time to build the index. Adding an index for every possible query
is not a good idea. This is a balancing act.
• Maximize selectivity.
Limitations2
• Index key value sizes are limited to 1024 bytes. Do not index too many fields. Do a
Object.bsonsize( { … } ) on the key values to help determine the size.
• Index key names are limited to 128 characters.
• Compound indexes can contain up to 31 fields.
• A single collection can have no more than 64 indexes. If you are hitting this limitation,
then you are over-indexing and reevaluate your data model.2 https://docs.mongodb.com/manual/reference/limits/#indexes

Sharding Deployments
Driver
Application
Query Router
(mongos)
Query Router
(mongos)
…
Data Center 1
Secondary
Primary
Secondary
Data Center 2
Secondary
Primary
Secondary
Data Center 3
Secondary
Primary
Secondary
Data Center N
Secondary
Primary
Secondary
Shard 1 Shard 3Shard 2
…
Shard N
Principals
• A node and server are one and the same.
• A primary (writable) and 2 secondary (read-only
nodes make up a 3-node replicate set.
• Primary and secondary are copies of each other.
• Shards reside across multiple servers.
• Load and data is distributed across each shard
facilitating scaling.
a

Two types of replication: Same replication mechanism. A primary node receives writes
while secondary nodes read and apply writes asynchronously.
• Master-slave: Deprecated.
– Uses a manual mechanism to promote a secondary to primary.
– oplog is only stored on the master (more on this later).
– Supports more than 50 nodes.
• Replica Sets:
– If the primary goes down, then a secondary is promoted to primary and receives
writes.
– Transparent recovery.
– More sophisticated deployment topologies.
– Supports up to 50 nodes.
– Recommended for production.
– Rollbacks: A write is not considered promoted unless it was written to a majority of
member nodes (50%). If the majority cannot accept the write, then all nodes
become read-only.
– A node replays a journal to determine what writes to apply after a failure.

• Why not use replication?
– Hardware load issues:
• Working set does not fit in RAM.
• Disk speed.
• More than half the operations are writes.
– Secondary nodes must always have an updated copy of data.
• This can be fixed, but it introduces a latency back to the
application.
• How does replication work?
– oplog: Capped collection in the local database containing records of
write data. Steps to reproduce the write are stored each time the primary
is written to. Each entry has a BSON timestamp to track writes.
– Heartbeat: Monitors health and controls failover by pinging every 2
seconds (default).

What you need to know before you deploy your next MongoDB implementation

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (18)

More from Rogue Wave Software

More from Rogue Wave Software (20)

Recently uploaded

Recently uploaded (20)

What you need to know before you deploy your next MongoDB implementation

Editor's Notes