DevOps: The Future is Already Here — It’s Just Unevenly Distributed

1. DevOps: The Future is Already Here — It’s Just Unevenly Distributed Damon Edwards @damonedwards

2. @damonedwards Damon Edwards

3. DevOps Consulting Automation Design Support & Services

4. devopscafe.org dev2ops.org

5. The history of DevOps is important…

6. A B

7. A B OpsDev

8. “Agile Infrastructure”

12. June 23, 2009

13. "10+ Deploys Per Day: Dev and Ops Cooperation at Flickr” June 23, 2009

17. October 30 - 31, 2009: Ghent, Belgium

18. Dev October 30 - 31, 2009: Ghent, Belgium

19. Dev October 30 - 31, 2009: Ghent, Belgium Ops

20. Dev October 30 - 31, 2009: Ghent, Belgium OpsDays

21. Dev October 30 - 31, 2009: Ghent, Belgium OpsDays *and many more!

22. Dev October 30 - 31, 2009: Ghent, Belgium # OpsDays *and many more!

23. DevOps Days Ghent 2009

24. DevOps Days Ghent 2009 Sydney 2010

25. DevOps Days Ghent 2009 Sydney 2010 Mountain View 2010

26. DevOps Days Ghent 2009 Sydney 2010 Mountain View 2010 +49 more as of Jan 2015 devopsdays.org

27. The Rise of a New IT Operations Support Model By 2015, DevOps will evolve from a niche strategy employed by large cloud providers into a mainstream strategy employed by 20% of Global 2000 organizations Why DevOps will emerge: !DevOps is not usually driven from Why DevOps will not emerge: !Cultural changes are the hardest to by 20% of Global 2000 organizations. !DevOps is not usually driven from the top down and, thus, may be more easily accepted by IT operations teams. !Cultural changes are the hardest to implement, and DevOps requires a significant rethinking of IT operations conventional wisdom. !ITIL and other best practices frameworks are acknowledged to have not delivered on their goals, enabling IT organizations to look for !There is a large body of work with respect to ITIL and other best practices frameworks that is already accepted within the industry enabling IT organizations to look for new models. !The growing interest in tools such as Chef, Puppet, etc., will help accepted within the industry. !Open source (OSS) management tools, which are more aligned with this approach, have not seen pp p stimulate demand for OSS-based management pp significant enterprise market share traction. March 18, 2011

28. The Rise of a New IT Operations Support Model By 2015, DevOps will evolve from a niche strategy employed by large cloud providers into a mainstream strategy employed by 20% of Global 2000 organizations Why DevOps will emerge: !DevOps is not usually driven from Why DevOps will not emerge: !Cultural changes are the hardest to by 20% of Global 2000 organizations. !DevOps is not usually driven from the top down and, thus, may be more easily accepted by IT operations teams. !Cultural changes are the hardest to implement, and DevOps requires a significant rethinking of IT operations conventional wisdom. !ITIL and other best practices frameworks are acknowledged to have not delivered on their goals, enabling IT organizations to look for !There is a large body of work with respect to ITIL and other best practices frameworks that is already accepted within the industry enabling IT organizations to look for new models. !The growing interest in tools such as Chef, Puppet, etc., will help accepted within the industry. !Open source (OSS) management tools, which are more aligned with this approach, have not seen pp p stimulate demand for OSS-based management pp significant enterprise market share traction. March 18, 2011

29. Next ➔ October 19-21, 2015 in San Francisco devopsenterprise.io

30. 2014 State of DevOps Survey 9,200+ Respondents from 110 countries, across all industries

31. 2014 State of DevOps Survey 9,200+ Respondents from 110 countries, across all industries

32. Why is the history important?

33. Why is the history important? It reminds us that DevOps is...

34. Why is the history important? • from practitioners, by practitioners It reminds us that DevOps is...

35. Why is the history important? • from practitioners, by practitioners • not a “thing” (product, methodology, speciﬁcation, job title) It reminds us that DevOps is...

36. Why is the history important? • from practitioners, by practitioners • not a “thing” (product, methodology, speciﬁcation, job title) • an experience-based movement It reminds us that DevOps is...

37. Why is the history important? • from practitioners, by practitioners • not a “thing” (product, methodology, speciﬁcation, job title) • an experience-based movement • decentralized and open to all It reminds us that DevOps is...

38. Why is the history important? • from practitioners, by practitioners • not a “thing” (product, methodology, speciﬁcation, job title) • an experience-based movement • decentralized and open to all It reminds us that DevOps is... Biggest source of tensions in community is not understanding this!

39. What have we learned?

40. Dev WallofConfusion Ops

41. At ﬁrst we forgot the context!

42. OpsDev We are in business to give the customer... • What they want • When they want it • At the lowest cost possible

43. OpsDev DevOps Problems!

44. Ah-ha! OpsDev Remove Shorten Lead Time Feedback

45. Ah-ha! OpsDev Remove Shorten Lead Time Feedback

46. • “My organization is stuck!… and I don’t know why” • “Business pressuring to route around me” • “I have no real visibility” • “Org has low confidence in ability to deliver” • “Usual coping mechanisms aren’t working” • “Don’t bring me more tools!” • “My job is painful!.. and I know why” • “I’m always waiting for someone or being undermined by someone else” • “If you just let me do it myself…” • “We should automate everything first” • “Using new tools like company x will fix our problems!” Executives Teams “I think DevOps can help me” “I think DevOps can help me”

49. • “My organization is stuck!… and I don’t know why” • “Business pressuring to route around me” • “I have no real visibility” • “Org has low confidence in ability to deliver” • “Usual coping mechanisms aren’t working” • “Don’t bring me more tools!” • “My job is painful!.. and I know why” • “I’m always waiting for someone or being undermined by someone else” • “If you just let me do it myself…” • “We should automate everything first” • “Using new tools like company x will fix our problems!” Executives Teams “I think DevOps can help me” “I think DevOps can help me” Organizational Analysis Process Redesign Management Coaching

50. • “My organization is stuck!… and I don’t know why” • “Business pressuring to route around me” • “I have no real visibility” • “Org has low confidence in ability to deliver” • “Usual coping mechanisms aren’t working” • “Don’t bring me more tools!” • “My job is painful!.. and I know why” • “I’m always waiting for someone or being undermined by someone else” • “If you just let me do it myself…” • “We should automate everything first” • “Using new tools like company x will fix our problems!” Executives Teams “I think DevOps can help me” “I think DevOps can help me” Organizational Analysis Process Redesign Management Coaching Software Tools Design Patterns and Methodologies Training

53. Execs Teams "I want to transform my organization" "I want to transform how I work" DevOps?

54. Organizational Transformation Execs "I want to transform my organization"

55. Predictability Short Time-to-Market High Quality Operations

56. My silo works just fine It’s out of my control Not the way we’ve always done it That just won’t work here Too busy running the business What is important? Didn’t work last time, why now? Where do I start?

57. Improved Time to Market & Quality Quicker Feedback from Users Business Idea Running Services Dev Ops DevOps and Agile Ops Security Auditing Compliance Availability Ops is in a particularly tough spot

58. Success: an organization that learns to fix itself Focus is on: ● Seeing end-to-end ● Fast feedback ● Eliminating waiting, rework ● Catching problems early Predictability Short Time-to-Market High Quality Operations

59. If you stand back far enough…

60. Silos are the #1 enemy of throughput and quality Dev Release OpsBusiness

61. Silos are the #1 enemy of throughput and quality Dev Release OpsBusiness

62. Silos are the #1 enemy of throughput and quality Dev Release OpsBusiness Handoff ! Handoff ! Handoff !

63. Silos are the #1 enemy of throughput and quality Dev Release OpsBusiness Application Knowledge Handoff ! Handoff ! Handoff !

64. Silos are the #1 enemy of throughput and quality Dev Release OpsBusiness Application Knowledge Operational Knowledge Handoff ! Handoff ! Handoff !

65. Silos are the #1 enemy of throughput and quality Dev Release OpsBusiness Application Knowledge Operational Knowledge Business Intent Handoff ! Handoff ! Handoff !

66. Silos are the #1 enemy of throughput and quality Dev Release OpsBusiness Application Knowledge Operational Knowledge Business Intent Handoff ! Handoff ! Handoff ! Ownership but limited Accountability

67. Silos are the #1 enemy of throughput and quality Dev Release OpsBusiness Application Knowledge Operational Knowledge Business Intent Handoff ! Handoff ! Handoff ! Ownership but limited Accountability Accountability but no Ownership

68. Organizational alignment is the key to overcoming silos

69. Organizational alignment is the key to overcoming silos • 1000’s of small decisions made daily • How do we harness that?

70. Organizational alignment is the key to overcoming silos • 1000’s of small decisions made daily • How do we harness that? • Top down is brittle • Conditions, people, tools are always changing... how do we handle that?

71. Organizational alignment is the key to overcoming silos • 1000’s of small decisions made daily • How do we harness that? • Top down is brittle • Conditions, people, tools are always changing... how do we handle that? • Alignment:   When you know that different individuals would independently see a set of conditions and arrive at the same decision that is directionally correct to achieve the company’s goals

72. Organizational alignment is the key to overcoming silos • 1000’s of small decisions made daily • How do we harness that? • Top down is brittle • Conditions, people, tools are always changing... how do we handle that? • Alignment:   When you know that different individuals would independently see a set of conditions and arrive at the same decision that is directionally correct to achieve the company’s goals • Startups have it; Enterprises lose it

73. Value Stream Mapping

74. Value Stream Mapping Process 1

75. Value Stream Mapping Process 1 ? PD TS W EP M M M TS PDM W DWastes 2

76. Value Stream Mapping Process 1 ? PD TS W EP M M M TS PDM W DWastes 2 ? PD TS W EP M M M TS PDM W D Ops involved early Unify ticket/req tools Write service tests in dev Automate from spec System Veriﬁcation Tests Move into Dev Self- Service ACLs Verify from deﬁnition of done Counter- measures 3

77. Core business service deliver Current state value stream m Development Environments Release Build and deploy QA Environments Performance Environment Staging Environment Productio Environme Deploy to Performance b Assets, ws and eframes Delta features and fixesSet of service ranches Documented test plans and cases Request QA build Daily Scrum meeting Daily Scrum meeting Deployment Meeting Business Systems Support Stan Walters Change Control Project Manager Erica Switzer Change request Business Analyst Erica Switzer Steering Committee Erica Switzer Full Projects / Expedites Estimates and requirements Technical Specification Deployment Document Change Control Forms RT tickets Jira issues Jira issues Tested packages Development Joe Harbaugh L/T 8 weeks P/T H/C S/R 4 weeks 5 25% Development Build and deploy Joe Harbaugh L/T 2 days P/T H/C S/R 4 hours 1 10% Development Integration testing Joe Harbaugh L/T 1 day P/T H/C S/R 1 day 5 95% Performance testing Deploy Product Build test plan Raj Lee L/T 5 days P/T H/C S/R 4 days 1 10% Build test cases Raj Lee L/T 2 weeks P/T H/C S/R 8 days 2 20% Fernando Gomez L/T 8 hours P/T H/C S/R 7 hours 1 2% Acceptance testing Raj Lee L/T 5 days P/T H/C S/R 5 hours 2 10% Feature testing Raj Lee L/T 4 weeks P/T H/C S/R 2.5 weeks 4 20% Allen Cannata L/T 1 week P/T H/C S/R 3 days 1 66% RT tickets Deploy to Staging Frances Middleton L/T 6 days P/T H/C S/R 90 minutes 3 66% Staging testing Frances Middleton L/T 1 week P/T H/C S/R 3.5 days 4 66% Allen Can L/T 9 hou P/T H/C S/R 8 hou 6 66% Frances Middleton L/T 6 days P/T H/C S/R 90 minutes 3 66% D DD PD TSx5 Dx2 Wx2 PDx4 EP D PDx3 Mx8 D Wx2 PDx2 What should we be looking for? • Long lead times (vs processing time) • Request and approval queues • Large batch sizes • Human to human information transfer • Scrap / Rework / “Backwash” • Process or tooling inconsistency • Manual deployment / configuration • Reliance on manual testing / verification • Injecting requirements late in the process • Fixing in place (vs rebuilding) • “Snowflake” servers and environments • Manual dependency management • Ad-hoc or unstructured artifact management

78. People Process Tools

79. Align work to customer value Dev & QA Activity Rel. / SRE Activity Ops Activity Business Activity

80. Dev & QA Activity Rel. / SRE Activity Ops Activity Business Activity Customer / Product Aligned Teams Customer / Product Aligned Teams Customer / Product Aligned Teams Align work to customer value

81. Dev & QA Activity Rel. / SRE Activity Ops Activity Business Activity Customer / Product Aligned Teams Customer / Product Aligned Teams Customer / Product Aligned Teams Environments as a Service QA as a Service Insights as a Service Security as a Service Environments Team QA Team Monitoring & Metrics Team Security Team What about cross-cutting concerns?

82. Process and Tooling Transformation Teams "I want to transform how I work"

83. Dev & QA Activity Rel. / SRE Activity Ops Activity Business Activity Customer / Product Aligned Teams Customer / Product Aligned Teams Customer / Product Aligned Teams Environments as a Service QA as a Service Insights as a Service Security as a Service Environments Team QA Team Monitoring & Metrics Team Security Team Align work to customer value

84. Code Source Repo Conﬁg CI Server Package Repo Operations Console Shell Powershell Pre-Production Environments Shell Powershell Production Environment Packages Operations Development SOURCE Drive all changes through a SDLC Tests

85. CodeTests Source Repo Conﬁg Env Spec Run- book Auto- mation CI Server Package Repo Operations Console Shell Powershell Pre-Production Environments Shell Powershell Production Environment Packages Operations Development SOURCE Drive all changes through a SDLC

86. Code Source Repo Conﬁg Env Spec Run- book Auto- mation CI Server Package Repo Operations Console Shell Powershell Pre-Production Environments Shell Powershell Production Environment Packages Operations Development SOURCE Dev Ops * Collaboration Tests Drive all changes through a SDLC

87. Code Versioned Release Tests Source Repo Conﬁg Env Spec Run- book Auto- mation CI Server Package Repo Operations Console Shell Powershell Pre-Production Environments Shell Powershell Production Environment Packages Operations Development SOURCE Dev Ops * Collaboration Drive all changes through a SDLC

88. Code Versioned Release Tests Source Repo Conﬁg Env Spec Run- book Auto- mation CI Server Package Repo Operations Console Shell Powershell Pre-Production Environments Shell Powershell Production Environment Packages Operations Development SOURCE Dev Ops * Collaboration Dev Ops * Execute Operations Procedures Drive all changes through a SDLC

89. Code Versioned Release Tests Source Repo Conﬁg Env Spec Run- book Auto- mation CI Server Package Repo Operations Console Shell Powershell Pre-Production Environments Shell Powershell Production Environment Packages Operations Development SOURCE Dev Ops * Collaboration Dev Ops * Execute Operations Procedures Drive all changes through a SDLC Same People!!

90. Code Versioned Release Tests Source Repo Conﬁg Env Spec Run- book Auto- mation CI Server Package Repo Operations Console Shell Powershell Pre-Production Environments Shell Powershell Production Environment Packages Operations Development SOURCE Dev Ops * Collaboration Dev Ops * Execute Operations Procedures SERVICE Drive all changes through a SDLC

91. What about cross-cutting concerns? (PO • Dev • Test • SRE) Cross Functional Delivery Team CodeTests Source Repo Conﬁg Env Spec Run- book Auto- mation CI Server Package Repo Operations Console Shell Powershell Pre-Production EnvironmentsPackages SOURCE QA Security Environments Monitoring --- Metrics

92. What about cross-cutting concerns? (PO • Dev • Test • SRE) Cross Functional Delivery Team CodeTests Source Repo Conﬁg Env Spec Run- book Auto- mation CI Server Package Repo Operations Console Shell Powershell Pre-Production EnvironmentsPackages SOURCE QA Security Environments Monitoring --- Metrics QA as a  Service Security as a  Service Metrics as a  Service Env. as a  Service

93. What about cross-cutting concerns? (PO • Dev • Test • SRE) Cross Functional Delivery Team CodeTests Source Repo Conﬁg Env Spec Run- book Auto- mation CI Server Package Repo Operations Console Shell Powershell Pre-Production EnvironmentsPackages SOURCE pull pull pull pull QA Security Environments Monitoring --- Metrics QA as a  Service Security as a  Service Metrics as a  Service Env. as a  Service

94. Be a service provider pull Cross-Cutting Concern X ✓ Standardized oﬀerings ✓ Pulled by users (not pushed) ✓ On-demand and self-service ✓ Implementation knowledge not necessary for normal use ✓ Provider spends their time building service and coaching users X as a Service

95. Be a service provider pull Cross-Cutting Concern X ✓ Standardized offerings ✓ Pulled by users (not pushed) ✓ On-demand and self-service ✓ Implementation knowledge not necessary for normal use ✓ Provider spends their time building service and coaching users X as a Service Google: “Cloud Operations at Netflix”  “Actionable Metrics Netflix” Roy Rapoport Mark Maun In this room at 1:30pm!

96. How to start being a service provider pull Cross-Cutting Concern X X as a Service

97. How to start being a service provider pull Cross-Cutting Concern X X as a Service Deﬁne your oﬀerings1

98. How to start being a service provider pull Cross-Cutting Concern X X as a Service Deﬁne your oﬀerings1

99. How to start being a service provider pull Cross-Cutting Concern X X as a Service Deﬁne your oﬀerings1 Tame the tool sprawl2

100. How to start being a service provider pull Cross-Cutting Concern X X as a Service Deﬁne your oﬀerings1 Tame the tool sprawl2

101. How to start being a service provider pull Cross-Cutting Concern X X as a Service Deﬁne your oﬀerings1 Tame the tool sprawl2 Setup self-service interfaces3

102. How to start being a service provider pull Cross-Cutting Concern X X as a Service Deﬁne your oﬀerings1 Tame the tool sprawl2 Setup self-service interfaces3

103. How to start being a service provider pull Cross-Cutting Concern X X as a Service Deﬁne your oﬀerings1 Tame the tool sprawl2 Setup self-service interfaces3 Setup secure access4

104. How to start being a service provider pull Cross-Cutting Concern X X as a Service Deﬁne your oﬀerings1 Tame the tool sprawl2 Setup self-service interfaces3 Setup secure access4

105. How to start being a service provider pull Cross-Cutting Concern X X as a Service Deﬁne your oﬀerings1 Tame the tool sprawl2 Setup self-service interfaces3 Setup secure access4 Plug: Give Rundeck a try --> rundeck.org

106. What’s Next?

107. Microservices

108. Any organization that designs a system (deﬁned broadly) will produce a design whose structure is a copy of the organization's communication structure. Melvyn Conway, 1967 Conway’s Law:

109. Martin Fowler

110. Martin Fowler

111. Martin Fowler

112. Go read: http://martinfowler.com/articles/microservices.html Martin Fowler

113. Adrian Cockcroft

116. Google: “Adrian Cockcroft Microservices” Adrian Cockcroft

117. The Network

118. The pain is legendary “Our cloud is awesome… VMs in minutes… but then it takes two days to get the VM connected to anything” “But the ACL policy worked yesterday” “Firewall rules only get changed on Thursdays” “I didn’t know the load would spike that way” “That’ll take 4 tickets and an act of god” “If we don’t predict traffic patterns right our app is hosed and there is no quick recovery” “Only 3 people here know how to do that” “After all that it turns out it was a DNS issue”

119. The network is changing 1990 2015 95% traffic left the DC New architectures: 5% traffic leaves the DC (Facebook: only 2%!) John Willis

120. The demands are changing • Disaggregation of Hardware/Software • Web Scale • Private Cloud Computing • Containerization • Software Eating the Network • DevOps Faster pace of change! Decentralized control! John Willis

121. Expect acceleration The Network Iceberg Data Gravity Micro Services Who? How? John Willis

122. Expect acceleration The Network Iceberg Data Gravity Micro Services Who? How? John Willis In this room at 3:00pm! @botchagalupe

123. Let’s close with… you. What can you do starting now?

124. Does your solution…. DevOps Litmus Test

125. Reduce cycle time AND improve quality? Does your solution…. DevOps Litmus Test

126. Reduce cycle time AND improve quality? Eliminate handoﬀs or reduce the friction of those handoﬀs that can't be eliminated? Does your solution…. DevOps Litmus Test

127. Reduce cycle time AND improve quality? Eliminate handoffs or reduce the friction of those handoffs that can't be eliminated? Improve tool-to-tool artifact flow and eliminate manual information flow? Does your solution…. DevOps Litmus Test

128. Reduce cycle time AND improve quality? Eliminate handoffs or reduce the friction of those handoffs that can't be eliminated? Improve tool-to-tool artifact flow and eliminate manual information flow? Eliminate manually-fulfilled request queues and other sources of waiting? Does your solution…. DevOps Litmus Test

129. Reduce cycle time AND improve quality? Eliminate handoffs or reduce the friction of those handoffs that can't be eliminated? Improve tool-to-tool artifact flow and eliminate manual information flow? Eliminate manually-fulfilled request queues and other sources of waiting? Improve awareness and understanding of the current state and desired state of the end-to-end system? Does your solution…. DevOps Litmus Test

130. Reduce cycle time AND improve quality? Eliminate handoffs or reduce the friction of those handoffs that can't be eliminated? Improve tool-to-tool artifact flow and eliminate manual information flow? Eliminate manually-fulfilled request queues and other sources of waiting? Improve awareness and understanding of the current state and desired state of the end-to-end system? Does your solution…. DevOps Litmus Test ✓ ✓ ✓ ✓ ✓

131. @damonedwards Damon Edwards