Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Performance Metrics Driven CI/CD - Introduction to Continuous Innovation and Optimization

1,113 views

Published on

Deck used for my talk at the 2016 Spring User Conference in Toronto. Deck was followed up by a walkthrough of a Jenkins workflow that deployed to Cloud Foundry based on jmeter test results

Published in: Software
  • I’ve personally never heard of companies who can produce a paper for you until word got around among my college groupmates. My professor asked me to write a research paper based on a field I have no idea about. My research skills are also very poor. So, I thought I’d give it a try. I chose a writer who matched my writing style and fulfilled every requirement I proposed. I turned my paper in and I actually got a good grade. I highly recommend ⇒ www.HelpWriting.net ⇐
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I’ve personally never heard of companies who can produce a paper for you until word got around among my college groupmates. My professor asked me to write a research paper based on a field I have no idea about. My research skills are also very poor. So, I thought I’d give it a try. I chose a writer who matched my writing style and fulfilled every requirement I proposed. I turned my paper in and I actually got a good grade. I highly recommend ⇒ www.WritePaper.info ⇐
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • A professional Paper writing services can alleviate your stress in writing a successful paper and take the pressure off you to hand it in on time. Check out, please ⇒ www.HelpWriting.net ⇐
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hello there! Get Your Professional Job-Winning Resume Here! http://bit.ly/topresum
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Performance Metrics Driven CI/CD - Introduction to Continuous Innovation and Optimization

  1. 1. Performance metrics driven CI/CD Michael Villiger – Developer Advocate michael.villiger@dynatrace.com @mikevilliger Introduction to CIO (Continuous Innovation & Optimization
  2. 2. So how do we change? Confidential, Dynatrace, LLC
  3. 3. Confidential, Dynatrace LLC “24 Features in a Box” “Very late Feedback” “Ship the whole box” “Frustration”
  4. 4. Confidential, Dynatrace LLC “1 Feature at a Time” “Optimize before Deploy”“Immediate Customer Feedback” Continuous Innovation and Optimization
  5. 5. Confidential, Dynatrace LLC Waterfall  Agile Continuous Integration / Delivery Test Automation & Automated Deployment Monolith  Microservices DevOps & Continuous Deployment on New Stack A/B Testing, Feature Flags and Metrics-Driven Feedback
  6. 6. Source: https://www.threadless.com/@alanis
  7. 7. 700 deployments / YEAR 10 + deployments / DAY 50 – 60 deployments / DAY Every 11.6 SECONDS Unicorns: Deliver value at the speed of business
  8. 8. DevOps @ Target presented at Velocity, DOES and more … http://apmblog.dynatrace.com/2016/07/07/measure-frequent-successful-software-releases/ “We increased from monthly to 80 deployments per week … only 10 incidents per month … … over 96% successful! ….”
  9. 9. “We Deliver High Quality Software, Faster and Automated using New Stack” “Shift-Left Performance to Reduce Lead Time”, Adam Auerbach, Sr. Dir DevOps https://github.com/capitalone/Hygieia & https://www.spreaker.com/user/pureperformance “… deploy some of our most critical production workloads on the AWS platform …”, Rob Alexander, CIO
  10. 10. 2 major releases/year customers deploy & operate on-prem 26 major releases/year 170 prod deployments/day self-service online sales SaaS & Managed 2011 2016
  11. 11. Not every Sprint ends without bruises!
  12. 12. Richard Dominguez Developer in Operations Prep Sportswear “In 2013 business demanded to go from monthly to daily deployments” “80% failed!”
  13. 13. Understanding Code Complexity • 4 Millions Lines of Monolith Code • Partially coded and commented in Russian From Monolith to Microservice • Initial devs no longer with company • What to extract without breaking it? Shift Left Quality & Performance • No automated testing in the pipeline • Bad builds just made it into production Cross Application Impacts • Shared Infrastructure between Apps • No consolidated monitoring strategy
  14. 14. Scaling an Online Sports Club Search Service 2015201420xx Response Time 2016+ 1) 2-Man Project 2) Limited Success 3) Start Expansion 4) Performance Slows Growth Users 5) Potential Decline?
  15. 15. Early 2015: Monolith Under Pressure Can‘t scale vertically endlessly! May: 2.68s 94.09% CPU Bound April: 0.52s
  16. 16. From Monolith to Services in a Hybrid-Cloud Move Front End to Cloud Scale Backend in Containers!
  17. 17. Go live – 7:00 a.m.
  18. 18. Go live – 12:00 p.m.
  19. 19. What Went Wrong?
  20. 20. 26.7s Load Time 5kB Payload 33! Service Calls 99kB - 3kB for each call! 171!Total SQL Count Architecture Violation Direct access to DB from frontend service Single search query end-to-end
  21. 21. It‘s not about blind automation of pushing more bad code on new stacks through a pipeline
  22. 22. Understanding Code Complexity • Existing 10 year old code & 3rd party • Skills: Not everyone is a perf expert or born architect From Monolith to Microservice • Service usage in the End-to-End Scenarios? • Will it scale? Or is it just a new monolith? Understand Deployment Complexity • When moving to Cloud/Virtual: Costs, Latency … • Old & new patterns, e.g: N+1 Query, Data Understand Your End Users • What they like and what they DONT like! • Its priority list & input for other teams, e.g: testing
  23. 23. The fixed end-to-end use case “Re-architect” vs. “Migrate” to “New Stack” 2.5s (vs 26.7) 5kB Payload 1! (vs 33!) Service Call 5kB (vs 99) Payload! 3!(vs 177) Total SQL Count
  24. 24. Dev&Test: Check-In Better Code Performance: Production Ready Checks! Validate Monitoring Ops/Biz: Provide Usage and Resource Feedback for next Sprints Test / CI: Stop Bad Builds Early Build & Deliver Apps like the Unicorns! With a Metrics-Driven Pipeline!
  25. 25. Build 17 testNewsAlert OK testSearch OK Build # Use Case Stat #API Calls # SQL Payload CPU 1 5 2kb 70ms 1 35 5kb 120ms Use Case Tests and Monitors Service & App Metrics Build 26 testNewsAlert OK testSearch OK Build 25 testNewsAlert OK testSearch OK 1 4 1kb 60ms 34 171 104kb 550ms Ops #ServInst Usage RT 1 0.5% 7.2s 1 63% 5.2s 1 4 1kb 60ms 2 3 10kb 150ms 1 0.6% 3.2s 5 75% 2.5s Build 35 testNewsAlert - testSearch OK - - - - 2 3 10kb 150ms - - - 8 80% 2.0s Continuous Innovation and Optimization Re-architecture into “Services” + Performance Fixes Scenario: Monolithic App with 2 Key Features
  26. 26. #SQL, #Threads, Bytes Sent, # Connections WPO Metrics, Objects Allocated, ...
  27. 27. https://github.com/Dynatrace/Dynatrace-Test-Automation-Samples https://dynatrace.github.io/ufo/ Fail the build!
  28. 28. Integrate into your Build Server
  29. 29. Integrate into your Build Server
  30. 30. 1.Performance validation: 1.response time corridor 2.# of SQL calls (N+1!) 3.# of service calls (N+1!) 4.# of exceptions (Frameworks) 2.Architectural validation 1.Making sure services are calling the right thing and not skipping layers Performance metrics in your pipeline
  31. 31. Continuous Innovation and Optimization Dev: Only Check in Quality Code Test: Stop Bad Features/Builds early Arch: Optimize scalability, architecture and performance DevOps: Validate production readiness and monitoring Biz/Ops: Ensure Happy End Users and Healthy Environment Biz/App: Innovate through User Analytics and Ops Feedback Ops: Monitor ALL of your apps powered by Dynatrace
  32. 32. SLA Monitoring: Are we up or down? Availability dropped to 0%
  33. 33. UEM: Conversion to User Experience New Deployment + Mkt Push Increase # of unhappy users! Decline in Conversion Rate Overall increase of Users! Spikes in FRUSTRATED Users!
  34. 34. Resource Budgets & Cost per Feature 4x $$$ to IaaS
  35. 35. #4: Performance -> Behavior
  36. 36. 1.Not just for back-end developers! 1.# of domains 2.# of images 3.Resource size 4.Page size Performance metrics in your pipeline
  37. 37. Performance management for the digital customer age

×