-
Be the first to like this
Upcoming SlideShare
PyBay 2018: Production-Ready Python Applications
- 1. Production-Ready Python applications Michael Kehoe Staff Site Reliability Engineer
- 2. Today’s agenda 1 Introduction 2 Tenets of Readiness 3 Building Production-Ready Python Applications 4 Recap
- 3. Introduction
- 4. Michael Kehoe $ /USR/BIN/WHOAMI • Staff Site Reliability Engineer @ LinkedIn • Production-SRE Team • Funny accent = Australian + 4 years American • Worked on: • Networks • Micro-services • Traffic Engineering • Databases
- 5. Production-SRE Team @ LinkedIn $ /USR/BIN/WHOAMI • Disaster Recovery - Planning & Automation • Incident Response – Process & Automation • Visibility Engineering – Making use of operational data • Reliability Principles – Defining best practice & automating it
- 6. Production-Ready Python Applications • This talk is a high-level overview of what it takes to build a production-ready service • Each of the topics could be its own separate talk • Focus on standard open-source options • Number of options not mentioned
- 7. What makes an application Production-Ready?
- 8. O’Reilly 2017 Susan J. Fowler Production-Ready Microservices
- 9. “A production-ready application or service is one that can be trusted to serve production traffic…” S U S A N J . F O W L E R
- 10. “… We trust it to behave reasonably, we trust it to perform reliably, we trust it to get the job done and to do its job well with very little downtime.” S U S A N J . F O W L E R
- 11. Tenets of readiness
- 12. Tenets of Readine ss 1 Stability 2 Reliability 3 Scalability 4 Performance 5 Fault Tolerance 6 Disaster Recovery 7 Monitoring 8 Documentation
- 13. Building Production-Ready Python applications
- 14. Stability
- 15. Tenets of Readiness STABILITY • Stable development cycle • Code Linting • Code Review • Central Repository • Build system • See Fabio Fleitas’s talk from Saturday
- 16. Tenets of Readiness STABILITY • Stable deployment cycle • Canary/ Staging environment • Reliable Deployment via: • Docker/ Kubernetes • Heroku • CD tools
- 17. Reliability
- 18. Tenets of Readiness RELIABILITY • Dependency Management • Onboarding + Deprecation procedures • See Documentation section • Routing + Discovery • Etcd • PyDiscover • Consul-service-discovery
- 19. Scalability
- 20. Tenets of Readiness SCALABILITY • Understanding growth-scales • Qualitative vs quantitative growth scale • Resource awareness • Dependency scaling • What services/ databases need to scale
- 21. Performance
- 22. Tenets of Readiness PERFORMANCE • Constant performance evaluation • Understand how to benchmark application • Traffic management • Understand traffic pattern performance • Capacity Planning • Have the right metrics
- 23. Fault Tolerance
- 24. Tenets of Readiness FAULT TOLERANCE • Avoiding Single Points of Failure (SPOF) • Use multiple instances behind a load balancer • Catch exceptions (meaningfully)
- 25. Tenets of Readiness FAULT TOLERANCE • Resiliency Engineering • Add testing to verify that non-standard behavior is handled correctly • Run chaos experiments: • DNS/ Network failures • Consume disk space/ IO • Consume CPU
- 26. Disaster Recovery
- 27. Tenets of Readiness DISASTER RECOVERY • Understand common failures • Have an application-specific disaster- recovery plan in place • Have a general incident management plan
- 28. Tenets of Readiness DISASTER RECOVERY: DAEMONS
- 29. Tenets of Readiness DISASTER RECOVERY: FLASK APP
- 30. Monitoring
- 31. Tenets of Readiness MONITORING • Logging • Tracing • Metrics Dashboards/ Alerts
- 32. Tenets of Readiness MONITORING: LOGGING https://docs.python.org/2/library/logging.handlers.html#sysloghandler
- 33. Tenets of Readiness MONITORING: TRACING • Multiple (free/ open-source) Options • Opentracing • Jaeger • Zipkin (various community libraries)
- 34. Tenets of Readiness MONITORING: METRICS • Multiple (free/ open-source) Options • Statsd • Jaeger • prometheus
- 35. Monitoring: Metrics
- 36. Documentation
- 37. In-Code Documentation What you should be covering • Function/ API & Class docstrings • Code usage documentation • Project documentation • Onboarding • Contribution • Testing
- 38. Tenets of Readiness DOCUMENTATION How to do it? • Sphinx • JavaDoc • Doxygen • Etc….
- 39. Documentation RESTRUCTURED TEXT EXAMPLE https://github.com/requests/requests/blob/master/requests/api.py
- 40. Documentation RESTRUCTURED TEXT EXAMPLE http://docs.python-requests.org/en/master/api/
- 41. Documentation RESTRUCTURED TEXT EXAMPLE https://github.com/requests/requests/blob/master/requests/api.py
- 42. Documentation RESTRUCTURED TEXT EXAMPLE http://docs.python-requests.org/en/master/api/
- 43. Documentation https://oncall.tools/docs/
- 44. Documentation • Relevant PEP’s: • PEP -257: Docstring Conventions • PEP-287: reStructured Docstring Format (Official Python Documentation Standard) • Further info: • https://pythonhosted.org/an_example_ pypi_project/sphinx.html • https://realpython.com/documenting- python-code
- 45. Recap
- 46. Tenets of Readine ss 1 Stability 2 Reliability 3 Scalability 4 Performance 5 Fault Tolerance 6 Disaster Recovery 7 Monitoring 8 Documentation
- 47. Further Resources • Find me at: • michael-kehoe.io • @matrixtek • linkedin.com/in/michaelkkehoe • Slides will be available in multiple locations shortly
Login to see the comments