Trading up: Adding Flexibility and Scalability to Bouygues Telecom with MongoDB
•Download as PPTX, PDF•
0 likes•2,839 views
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (9)
Similar to Trading up: Adding Flexibility and Scalability to Bouygues Telecom with MongoDB
Similar to Trading up: Adding Flexibility and Scalability to Bouygues Telecom with MongoDB (20)
More from MongoDB
More from MongoDB (20)
Recently uploaded
Recently uploaded (20)
Trading up: Adding Flexibility and Scalability to Bouygues Telecom with MongoDB
- 1. Adding Flexibility and Scalability to Bouygues Telecom with MongoDB Pierre-Alban DEWITTE @__pad__ Bouygues Telecom MongoDB World – 06/24/2014
- 2. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Bouygues Telecom 13 M Customers
- 3. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Who is speaking ? @__pad__
- 4. Thanks !
- 5. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Agenda Adding Flexibility and Scalability to Bouygues Telecom with MongoDB 1.Existing problems 2.Design choices 3.Build 4.A few steps back
- 6. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Agenda 1.Existing problems
- 7. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Existing problems Consumer Customers Existing Sytem Compagny Customers LDAP access Micropayment system Identity and Oauth management system Network Connexion system ETL
- 8. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Existing problems Two years delay Schema customization very hard to design and implement Performance issues
- 9. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Existing problems
- 10. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Agenda 1.Existing problems 2.Design choices A.MongoDB B.Tomcat C.Storm
- 11. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - MongoDB Schema less data integration High availability Sharding Less expensive data storage allowing duplication
- 12. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - MongoDB Team with development and architecture skills Schema design experience Capability to train new actors
- 13. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld POC between Tomcat and NodeJS We choose our battle, MongoDB ! Design choices - Tomcat
- 14. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld
- 15. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - Storm Distributed real-time computation system
- 16. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - Storm Consumer Customer MongoDB database Compagny Customers REST Micropayment system Identity and Oauth management system Network Connexion system API STORM
- 17. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Agenda 1.Existing problems 2.Design choices 3.Build A.Storm topology B.Schema design and refactor C.DevOps
- 18. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Storm topology Customer files are pushed every night Read file line by line 1 … DetectFile Spout ProcessLine Bolt 2 line
- 19. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Storm topology
- 20. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Storm topology Emitting tuple for each set of functional collections ProcessLine Bolt 4 Oauth UpdaterBolt Identity UpdaterBoltTransform a line into a document & check delta 3
- 21. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Storm topology
- 22. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Storm topology
- 23. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - Storm Message failure handle ack(Object msgId) failed(Object msgId) Take care of system stability
- 24. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - Storm We scaled and easily !
- 25. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld
- 26. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Conception User -Name -Compagny -Billcycle -Payment Mode Acces Point -Name -Compagny -Billcycle -Payment Mode 1 1..n Conception
- 27. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Development
- 28. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Qualification
- 29. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Development
- 30. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor « That works » Qualification Conception
- 31. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor StressTest
- 32. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor ConceptionConception
- 33. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor ConceptionConception
- 34. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Development
- 35. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Development Qualification Conception StressTest
- 36. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor PERFECT ! Development Conception StressTes
- 37. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - DevOps Tooling first
- 38. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - DevOps Moving the line between Dev and Ops
- 39. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - DevOps Internal training by development team Redaction of a play-book dedicated to ops Pre production tests played by both team Early write of stress tests, played during qualification tests
- 40. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld 6 months later
- 41. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Agenda 1.Existing problems 2.Design choices 3.Iterations during build 4.A few steps back
- 42. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld A few steps back Apache Tomcat Apache STORM Flexibility and Scale with a 100% Open Source Stack !
- 43. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld A few steps back 1.Choose your battle 2.Use the right tool for the right job 3.Be prepare to fail and to rework 4.To move organization, training is a key
- 44. Thank you for listening Questions ? Further talks @__pad__
- 45. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Photo credits 100 m start : http://fr.wikipedia.org/wiki/100_m%C3%A8tres_%28athl%C3%A9tisme%29#mediaviewer/Fichier :Mens_100m_finals_British_Champs_and_Olympic_Trials.jpg by Paul Foot from Birmingham, UK Question mark block : https://www.flickr.com/photos/jarbo/9379813470 by Jared Cherup Pelleteuse orange : http://fr.123rf.com/photo_3669950_un-grand-pelleteuse-orange-stationne-a-un-chantier-de- construction.html by Stephen Mcsweeny Punaise : https://www.flickr.com/photos/24362608@N05/3501112978/sizes/l/in/photolist- f5R8g4-6kiVfR-6koahu-6ko7US-6ko8Qh-d9NUKU-nDi2eb-ffg13f-8h56wx-cgAcib-hchCtD-decZ4p- 6kocrL-6kj2J6-5doxaw-gbTPmK-nCFBBq-672Snf-2sCzvw-e1zAjM-8voRbc-c9Rh1m-amFFGt- 2iZNBq-cFSnFU-81Ckk4-2XHZAy-9Ggpfw-m8FfKz-8hW6r5-4xSuxC-awXsp4-5aVn7g-fbXVHZ- 9GAPLX-9qr7uU-fcddgA-cP1EN7-np2q92-6V8oBT-cenSsb-cyzaEd-8h5a2F-6V2b3F-ch1RME-fJ52s3- 9XthVN-kBs5mW-6qSPaF-5cA9Qb/ par dractrain94 Une longue vue qui louche : https://www.flickr.com/photos/la_bretagne_a_paris/3847733265/ par Yann Caradec NASA-Apollo8-Dec24-Earthrise : http://www.hq.nasa.gov/office/pao/History/alsj/a410/AS8-14- 2383HR.jpg par NASA / Bill Anders
Editor's Notes
- Team like to try new things, why not used node ? Go quick
- Scalable database, Scalable front-end but provisioning part will not scale for the customer intergation part if we keep doing our traditional parallelized batch
- GO QUICK Storm well know now for real time processing but you can also use this for a batch It is scalable, fault-tolerant, guarantees your data will be processed, and is easy to set up and operate. Suits for real-time data acquisition treatment but also for batch
- This is the new design of our system. POC tests where what we expected. We fell confident enougth to start the building part
- Focus on three key parts of our project project