Running MongoDB on AWS
•Download as PPTX, PDF•
14 likes•5,695 views
This presentation covers best practices for running MongoDB on AWS. We also discuss how to utilize the automation features of MMS to spin up new clusters in minutes on AWS.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (17)
Similar to Running MongoDB on AWS
Similar to Running MongoDB on AWS (20)
More from MongoDB
More from MongoDB (20)
Recently uploaded
Recently uploaded (20)
Running MongoDB on AWS
- 1. Running MongoDB onAWS Sandeep Parikh Senior Solutions Architect MongoDB, Inc.
- 2. Agenda Background Deployment Automation Management Integrations Resources
- 3. MongoDB • Flexible document data model • Rich ad-hoc queries and in-place updates • Real-time aggregation • Geospatial support • Text search • Built-in support for – Redundancy and High Availability – Auto-partitioning and scale out
- 4. Amazon WebServices • Complete cloud infrastructure – Compute – Storage – Database – Analytics – Processing – Deployment – Containers • Multitude of configuration options • Pricing flexibility – On-demand, Spot instance, Reserved instance
- 5. Instance Selection • General Purpose • Compute-optimized • GPU • Memory-optimized • Storage-optimized • Micro
- 6. Instance Selection • General Purpose (M3) • Compute-optimized (C3) • GPU (compute resources not needed) • Memory-optimized (R3) • Storage-optimized (I2, HS1) • Micro (bursty, no sustained CPU)
- 7. Instance Characteristics • Distinctions – CPU, memory, storage, networking • Networking – EBS-optimized, enhanced networking, placement groups • Availability – Varies by region
- 8. Storage Configurations S3 Blob storage Static content EBS Magnetic SSD, burst IOPS OS root volume PIOPS EBS SSD-backed, predictable performance Cost scales up with size and IOPS Instance Store SSD-backed Blazing, ephemeral Included in instance cost
- 9. Storage Configurations S3 Blob storage Static content EBS Magnetic SSD, burst IOPS OS root volume PIOPS EBS SSD-backed, predictable performance Cost scales up with size and IOPS Instance Store SSD-backed Blazing, ephemeral Included in instance cost
- 10. Storage Configurations • PIOPS EBS or Instance Store are best choices • Instance Store offers best $/IOP – Storage is ephemeral – Must be used with MongoDB Replica Sets • Can mix/match in a single deployment – E.g. some Secondary nodes on EBS – …But you’ll need several EBS volumes to maintain reasonable IOPS parity
- 11. Instance Configuration • Use EXT4 or XFS along with appropriate attributes • Tune block device read-ahead • Tune TCP keep alive • Disable NUMA • Disable zone-reclaim mode • Increase ulimits for processes and open files
- 13. High Availability Across Zones MongoDB Primary MongoDB Secondary MongoDB Secondary Zone 1 Zone 2
- 14. High Availability Across Regions MongoDB Primary MongoDB Secondary MongoDB Secondary Region 1 Region 2
- 21. MongoDB Management Service • MMS is a web-based tool that starts you from the beginning of your MongoDB deployment lifecycle • Use MMS to build and maintain your deployment and to manage its lifecycle (monitoring and backup)
- 22. MMSChanges • Before, MMS was used to monitor and backup • But MMS was “late to the party” – mistakes or misconfigurations had been applied to the initial deployment • Monitoring was helpful but not in setting users down the right path • Upgrade/maintenance tasks were non-trivial and very involved
- 23. Automation
- 24. Automation
- 25. Automation Provision instances in AWS Deploy any version of MongoDB Add replicas or shards Update configuration at any time Push a button to upgrade MongoDB
- 26. Automation
- 27. Automation
- 28. Monitoring
- 29. Monitoring
- 30. Monitoring Charting MongoDB-specific metrics and measurements View complete cluster topology and metrics for each component Create custom dashboards for key metrics and nodes Alerting Create alerts for just about any metric value change Target some or all hosts Customizable notifications including SMS, HipChat, PagerDuty Proactive Support Our engineers monitor your deployment and make suggestions Offered to Subscription Customers
- 31. Backup
- 32. Backup Customizable snapshot policy Point-in-time recovery for replica sets Consistent sharded cluster snapshots Low overhead, securely transferred Continuous, incremental backups
- 33. Backup Mongodump File system MMS Backup Initial complexity Medium High Low Confidence in Backups Medium Medium High Point in time recovery of replica set Sort of ☺ No Yes System Overhead High Can be low Low Scalable No With work Yes Consistent Snapshot of Sharded System Difficult Difficult Yes
- 34. Integrations Compute Storage Persistent IPs DNS Hadoop Data Warehouse Stream Processing App Deployment Orchestration Database App Services Caching
- 35. Integrations Compute Storage Persistent IPs DNS Hadoop Data Warehouse Stream Processing App Deployment Orchestration Database App Services Caching
- 36. Elastic MapReduce • Background – Quickly deploy and run Hadoop in AWS – Tuned distributions to run on top of EC2 – Provision deployments with any number of nodes – Supports spot and reserved pricing to minimize cost • MongoDB – MongoDB Connector for Hadoop – https://github.com/mongodb/mongo-hadoop – Bi-directional access – MapReduce, Hive, Pig, Streaming, Spark – MongoDB deployments or BSON backup files
- 37. CloudWatch • Monitoring for AWS resources • Supports custom metrics • Use AWS CLI to pipe MongoDB metrics aws cloudwatch put-metric-data --metric-name ResidentMemory --namespace MongoDB --timestamp 2014-01-01T00:00:00Z --value 32 --unit Gigabytes
- 38. Redshift • Fully managed petabyte scale data warehouse as a service • MongoDB not natively supported as an input data source • Use Data Pipeline and EMR to move data http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/what-is-datapipeline.html
- 39. Elastic Beanstalk • Deploy and manage applications • Handles provisioning, scaling, load balancing • Built on EC2, S3, SNS, Auto Scaling • Customize and configure software that your app needs • Install packages, create files • Execute commands • Control system services App Server App Server App Server Security Group Elastic Load Balancer Auto Scaling Group mongosmongosmongos MongoDB
- 40. Route53 • Highly available and scalable DNS service • Hostnames can be assigned to – EC2 instances, ELB instances, S3 buckets • DNS load balancing with weighted round robin • Supports hostnames for non-AWS infrastructure • Use hostnames for all MongoDB components • With replica sets, hostnames can ease machine replacement • With sharded clusters, hostnames can simplify config server maintenance • Or use Automation!
- 41. Questions? • MongoDB – http://www.mongodb.org • MongoDB Documentation – http://docs.mongodb.org • MongoDB Management Service – http://mms.mongodb.com