Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Our DevOps Journey: 6 Month Waterfalls to 1 Hour Code Deploys

584 views

Published on

Our DevOps Journey
Transforming 6 Month Waterfalls to 1 Hour Code Deploys

https://info.dynatrace.com/17q3_wc_from_agile_to_cloudy_devops_na_registration.html

In the 2nd part of our webinar series, Anita Engleder, DevOps Lead at Dynatrace reviews and dissects lessons learned during the transformational journey moving Dynatrace from an on-prem culture to one that is cloud native. She will lend her perspective as a key member of the team that executed on the original vision: to “implement a new cloud native offering and deploy a new feature release every 2 weeks. Additionally, be able to support a 1-hour lead time from Code Change to Production”.

On November 17th at 1pm/10am PT Anita will present the challenges she and her team faced transforming 6 Months Waterfall to 1 Hour Code Deploys.

In this webinar Anita will discuss:

How to enable a complete cultural shift across multiple teams, in terms of thought process AND execution

What the specific role of her DevOps team is and how it played into the transformation

The role of Feature teams and why continuous feedback is critical for them

How to successfully influence key stakeholders for complete alignment

Today Anita’s team runs 170 production changes every day, running across several AWS Data Centers as well as On-Premise – something that would have been thought impossible only a few years prior.

Published in: Technology
  • I have always found it hard to meet the requirements of being a student. Ever since my years of high school, I really have no idea what professors are looking for to give good grades. After some google searching, I found this service ⇒ www.HelpWriting.net ⇐ who helped me write my research paper.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • My personal experience with research paper writing services was highly positive. I sent a request to ⇒ www.HelpWriting.net ⇐ and found a writer within a few minutes. Because I had to move house and I literally didn’t have any time to sit on a computer for many hours every evening. Thankfully, the writer I chose followed my instructions to the letter. I know we can all write essays ourselves. For those in the same situation I was in, I recommend ⇒ www.HelpWriting.net ⇐.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Get the best essay, research papers or dissertations. from ⇒ www.HelpWriting.net ⇐ A team of professional authors with huge experience will give u a result that will overcome your expectations.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Our DevOps Journey: 6 Month Waterfalls to 1 Hour Code Deploys

  1. 1. Our DevOps Journey Transforming 6 Month Waterfalls to 1 Hour Code Deploy http://dynatrace.com/trial Anita Engleder DevOps Manager
  2. 2. From the DevOps Webinar with Gene & Mark Mark Tomlinson Performance Sherpa @mark_on_task Andi Grabner Performance Advocate @grabnerandi Gene Kim, CTO Researcher and Author @RealGeneKim
  3. 3. @RealGeneKim 24 “Features in a Box” Ship the whole box! Very late feedback  @grabnerandi
  4. 4. @RealGeneKim Continuous Innovation and Optimization 1 “Feature at a Time” Optimize Before DeployImmediate Customer Feedback @grabnerandi
  5. 5. From 0 to DevOps in 80 days Lessons learnt from shifting an on-prem to a cloud culture Bernd Greifeneder, CTO http://dynatrace.com/trial Webinar: http://ow.ly/cEYo305kFEy Podcast: http://bit.ly/pureperf
  6. 6. 2 major releases/year customers deploy & operate on-prem 26 major releases/year 170 prod deployments/day self-service online sales SaaS & Managed 2011 2016
  7. 7. 7 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015 believe in the mission impossible 6months major/minor release + intermediate fix-packs + weeks to months rollout delay sprint releases (continuous-delivery) 1h : code to production
  8. 8. Dynatrace Transformation by the numbers 23x 170 more releases Deployments / Day 31000 60hUnit+Int Tests / hour UI Tests per Build More Quality ~200 340code commits / day Stories per sprint More Agile 93% Production bugs found by Dev @grabnerandi More Stability 450 99.998%Global EC2 Instances Global Availability
  9. 9. DevOps Stories & Practices Learned •The Role of the DevOps Team •The Role of the Feature Team •The Journey to DevOps •The DevOps Tool Chain @grabnerandi
  10. 10. Typical Role of DevOps Teams mainly responsible for … … monitoring Production and creating tickets for engineers … building deployment automation
  11. 11. NOT mainly responsible for … Role of Dynatrace DevOps Team … monitoring Production and creating tickets for engineers … building deployment automation
  12. 12. owning and promoting tooling that gives engineering teams … Role of Dynatrace DevOps Team … VISIBILITY & CONTINUOUS FEEDBACK from commit to deployment and into all deployments … CONTROL to deploy features/fixes … OWNERSHIP of their own source code for all deployments
  13. 13. acting as Engineers Role of Dynatrace DevOps Team Dynatrace Managed/SaaS Orchestration Layer DynatracePipeline Visualization Deployment Timeline Log Overview using Dynatrace Log APIJIRA Integrations & Product Managers
  14. 14. confidential Role of Dynatrace Feature Teams
  15. 15. ManualCICDFcontinuous integration continuous delivery & deployment and feedback
  16. 16. Role of Dynatrace Feature Teams DevOps = Engineers start thinking like an Ops before Commit CDF develop features & provide fixes build & adjust test coverage implement go-to- production/ migration plan add/use proper feedback channel consume feedback improve & harden CI
  17. 17. confidential DevOps Our Journey to CDF & NoOps
  18. 18. Our Journey to CDF & NoOps 1 3 2 4 5 6 7 Infrastructure/Platform as a Service API call instead fixing hardware Architects are responsible for production too Same monitoring in every stage! Visibility into every stage/deployment! Same Deployment Automation in every stage! No manual touches! Direct Feedback to Engineers! The real experts are primary consumers Control into engineer’s hand wherever it makes sense. Who pushes the button “deploy”?Provide fast lane to production for Hotfixes! 1 Hour Code Deploy! 3 4 5 7
  19. 19. 3 Same Deployment Automation in every stage! No manual touches! Dynatrace Managed Orchestration Layer Dynatrace SaaS Orchestration Layer Orchestration Layer Product owned by a Feature Team having it’s own CDF Pipeline
  20. 20. 4 Same monitoring in every stage! Visibility into every stage/deployment! Dev Stage Daily Production StageAcceptance Stage Bi-Daily
  21. 21. 4 Same monitoring in every stage! Visibility means also easy access! https://www.dynatrace.com/meet-davis/
  22. 22. 4 Same monitoring in every stage! Visibility means also easy access! https://www.dynatrace.com/meet-davis/
  23. 23. 5 Direct Feedback to Engineers! Engineers typically like Logs. But how to get the whole context?
  24. 24. Thank you! Food for thoughts: • How is your trust level to your engineering team? • Do you encourage failure and learn from it? • Where do you currently have bottlenecks from Dev to Ops? • Can technology help? Can process change help? • Do you have the right monitoring strategy / feedback loop? http://dynatrace.com/trial
  25. 25. http://dynatrace.com/trial Thank you!
  26. 26. 7 Provide fast lane to production for Hotfixes! 1 Hour Code Deploy How can I achieve that in case my trunk/master is not yet at any time production ready?
  27. 27. Dev Stage Daily Continuous Delivery & Feedback Production Stage Bi-Weekly & On Demand Acceptance Stage Bi-Daily Deploy Develop & Fixing Acceptance & Performance & Load Tests & Monitor Deploy Fixing Acceptance & Performance & Load Tests & Monitor Deploy Hotfixing Monitor Release Every 2 weeks version is pushed to next stage = 2 weeks release cycle Unit+Integration testing & Build Unit+Integration testing & Build Unit+Integration testing & Build Week 1 Week 2 Week 3 Week 4
  28. 28. CDF Tooling Chain - Dev Stage Acceptance Tests Selenium/Appium Automated Functional Tests on UI Level Automated E2E Tests on UI Level Browser Compatibility Testing (Chrome, IE, FF, Edge, Safari) 24/7 Deployment Checks Jira Manual Regression Testing AWS, VMware, VirtualBox Test Environments Develop Gradle + Artifactory build/deployment automation SVN + Git Version Control Eclipse / IntelliJ IDEA Quickbuild + Jenkins Continuous Integration Unit & Integration Tests (fully automated) Unit Tests +Integration Tests Quickbuild, Eclipse/IntelliJ IDEA for local execution before commit Memory/CPU Overhead tests Dynatrace AppMon Code Quality and Coverage Checks Sonar Memory leak detection Bullseye, Valgrind Virus Scan Kaspersky Open Source License compliance checks Blackduck Facts/Numbers 28.000 Unit Tests + 3.000 integration test executions / hour Deployment (fully automated) Quickbuild Automated daily deployment of Trunk Builds using Ansible and Puppet or calling Dynatrace Ruxit CloudControl, that uses AWS Cloud Formation (whole infrastructure as code!) Selenium/Appium Automated Customer-Like Deployment Deployment Checks on UI Level Performance & Load Tests Cluster Workload Simulator Eclipse Mem Analyzer (MAT) + Eclipse Thread Dump Analyzer Java Flightrecorder Monitoring Dynatrace Real-User, Service and Infrastructure monitoring WebChecks, AWS Monitoring, Log Analytics & Monitoring Continuous Feedback Tools built by Devs & DevOps OpsGenie and common HipChat Room to escalate detected Problems directly to Development Security 24/7 OSSEC Host intrusion monitoring Deploy Develop & Fixing Unit+Integration testing & Build Acceptance & Performance & Load Tests & Monitor
  29. 29. CDF Tooling Chain - Acceptance Stage Deploy Fixing Unit+Integration testing & Build Acceptance & Performance & Load Tests & Monitor Facts/Numbers (Dev + Acceptance Stage) ~700 Automated UI Tests ~60 hours UI Test execution per Build  ~20 parallel running test sets executed on ~30 execution machines up to ~5 hours per test set ~15 different OS (Windows, Linux) Unit & Integration Tests (fully automated) Unit Tests +Integration Tests Quickbuild, Eclipse/IntelliJ IDEA for local execution before commit Virus Scan Kaspersky Deployment (fully automated) Quickbuild Automated deployment of Sprint Builds using Ansible and Puppet or calling Dynatrace Ruxit CloudControl, that uses AWS Cloud Formation (whole infrastructure as code!) Selenium/Appium Automated Customer-Like Deployment Deployment Checks on UI Level Acceptance Tests Selenium/Appium Automated Functional Tests on UI Level Automated E2E Tests on UI Level Browser Compatibility Testing (Chrome, IE, FF, Edge, Safari) 24/7 Deployment Checks Jira Manual Regression Testing AWS, VMware, VirtualBox Test Environments Fixing Gradle + Artifactory build/deployment automation SVN + Git Version Control Eclipse / IntelliJ IDEA Quickbuild + Jenkins Continuous Integration Performance & Load Tests Cluster Workload Simulator Eclipse Mem Analyzer (MAT) + Thread Dump Analyzer Java Flightrecorder Monitoring Dynatrace Real-User, Service and Infrastructure monitoring WebChecks, AWS Monitoring, Log Analytics & Monitoring Continuous Feedback Tools built by Devs & DevOps OpsGenie and common HipChat Room to escalate detected Problems directly to Development Security 24/7 OSSEC Host intrusion monitoring
  30. 30. CDF Tooling Chain - Production Stage Deploy Hotfixing Unit+Integration testing & Build Monitor Fixing Gradle + Artifactory build/deployment automation SVN + Git Version Control Eclipse / IntelliJ IDEA Quickbuild + Jenkins Continuous Integration Monitoring Dynatrace Real-User, Service and Infrastructure monitoring WebChecks, AWS Monitoring, Log Analytics & Monitoring Continuous Feedback Tools built by Devs & DevOps OpsGenie and common HipChat Room to escalate detected Problems directly to Development Security 24/7 OSSEC Host intrusion monitoring Monthly and on demand Vulnerability Scans (KPMG Linz) Closed Bug Bounty Program at HackerOne Unit & Integration Tests (fully automated) Unit Tests +Integration Tests Quickbuild, Eclipse/IntelliJ IDEA for local execution before commit Virus Scan Kaspersky Deployment (fully automated) Quickbuild Automated deployment of Sprint Builds using Ansible and Puppet or calling Dynatrace Ruxit CloudControl, that uses AWS Cloud Formation (whole infrastructure as code!) Selenium/Appium Automated Customer-Like Deployment Deployment Checks on UI Level

×