47. The Rise of a New IT Operations
Support Model
By 2015, DevOps will evolve from a niche strategy employed
by large cloud providers into a mainstream strategy employed
by 20% of Global 2000 organizations
Why DevOps will emerge:
!DevOps is not usually driven from
Why DevOps will not emerge:
!Cultural changes are the hardest to
by 20% of Global 2000 organizations.
!DevOps is not usually driven from
the top down and, thus, may be
more easily accepted by IT
operations teams.
!Cultural changes are the hardest to
implement, and DevOps requires a
significant rethinking of IT
operations conventional wisdom.
!ITIL and other best practices
frameworks are acknowledged to
have not delivered on their goals,
enabling IT organizations to look for
!There is a large body of work with
respect to ITIL and other best
practices frameworks that is already
accepted within the industry enabling IT organizations to look for
new models.
!The growing interest in tools such
as Chef, Puppet, etc., will help
accepted within the industry.
!Open source (OSS) management
tools, which are more aligned with
this approach, have not seen pp p
stimulate demand for OSS-based
management
pp
significant enterprise market share
traction.
March 18, 2011
48. The Rise of a New IT Operations
Support Model
By 2015, DevOps will evolve from a niche strategy employed
by large cloud providers into a mainstream strategy employed
by 20% of Global 2000 organizations
Why DevOps will emerge:
!DevOps is not usually driven from
Why DevOps will not emerge:
!Cultural changes are the hardest to
by 20% of Global 2000 organizations.
!DevOps is not usually driven from
the top down and, thus, may be
more easily accepted by IT
operations teams.
!Cultural changes are the hardest to
implement, and DevOps requires a
significant rethinking of IT
operations conventional wisdom.
!ITIL and other best practices
frameworks are acknowledged to
have not delivered on their goals,
enabling IT organizations to look for
!There is a large body of work with
respect to ITIL and other best
practices frameworks that is already
accepted within the industry enabling IT organizations to look for
new models.
!The growing interest in tools such
as Chef, Puppet, etc., will help
accepted within the industry.
!Open source (OSS) management
tools, which are more aligned with
this approach, have not seen pp p
stimulate demand for OSS-based
management
pp
significant enterprise market share
traction.
March 18, 2011
49.
50. Next ➔ October 19-21, 2015 in San Francisco
devopsenterprise.io
51. 2014 State of
DevOps Survey
9,200+ Respondents
from 110 countries,
across all industries
52. 2014 State of
DevOps Survey
9,200+ Respondents
from 110 countries,
across all industries
54. Why is the history important?
It reminds us that DevOps is...
55. Why is the history important?
• from practitioners, by practitioners
It reminds us that DevOps is...
56. Why is the history important?
• from practitioners, by practitioners
• not a “thing” (product, methodology, specification, job title)
It reminds us that DevOps is...
57. Why is the history important?
• from practitioners, by practitioners
• not a “thing” (product, methodology, specification, job title)
• an experience-based movement
It reminds us that DevOps is...
58. Why is the history important?
• from practitioners, by practitioners
• not a “thing” (product, methodology, specification, job title)
• an experience-based movement
• decentralized and open to all
It reminds us that DevOps is...
59. Why is the history important?
• from practitioners, by practitioners
• not a “thing” (product, methodology, specification, job title)
• an experience-based movement
• decentralized and open to all
It reminds us that DevOps is...
Biggest source of tensions in community is not understanding this!
68. • “My organization is stuck!… and
I don’t know why”
• “Business pressuring to route
around me”
• “I have no real visibility”
• “Org has low confidence in
ability to deliver”
• “Usual coping mechanisms
aren’t working”
• “Don’t bring me more tools!”
• “My job is painful!.. and I know
why”
• “I’m always waiting for someone
or being undermined by
someone else”
• “If you just let me do it myself…”
• “We should automate everything
first”
• “Using new tools like company x
will fix our problems!”
Executives Teams
“I think DevOps can help me” “I think DevOps can help me”
69. • “My organization is stuck!… and
I don’t know why”
• “Business pressuring to route
around me”
• “I have no real visibility”
• “Org has low confidence in
ability to deliver”
• “Usual coping mechanisms
aren’t working”
• “Don’t bring me more tools!”
• “My job is painful!.. and I know
why”
• “I’m always waiting for someone
or being undermined by
someone else”
• “If you just let me do it myself…”
• “We should automate everything
first”
• “Using new tools like company x
will fix our problems!”
Executives Teams
“I think DevOps can help me” “I think DevOps can help me”
70. • “My organization is stuck!… and
I don’t know why”
• “Business pressuring to route
around me”
• “I have no real visibility”
• “Org has low confidence in
ability to deliver”
• “Usual coping mechanisms
aren’t working”
• “Don’t bring me more tools!”
• “My job is painful!.. and I know
why”
• “I’m always waiting for someone
or being undermined by
someone else”
• “If you just let me do it myself…”
• “We should automate everything
first”
• “Using new tools like company x
will fix our problems!”
Executives Teams
“I think DevOps can help me” “I think DevOps can help me”
71. • “My organization is stuck!… and
I don’t know why”
• “Business pressuring to route
around me”
• “I have no real visibility”
• “Org has low confidence in
ability to deliver”
• “Usual coping mechanisms
aren’t working”
• “Don’t bring me more tools!”
• “My job is painful!.. and I know
why”
• “I’m always waiting for someone
or being undermined by
someone else”
• “If you just let me do it myself…”
• “We should automate everything
first”
• “Using new tools like company x
will fix our problems!”
Executives Teams
“I think DevOps can help me” “I think DevOps can help me”
Organizational Analysis
Process Redesign
Management Coaching
72. • “My organization is stuck!… and
I don’t know why”
• “Business pressuring to route
around me”
• “I have no real visibility”
• “Org has low confidence in
ability to deliver”
• “Usual coping mechanisms
aren’t working”
• “Don’t bring me more tools!”
• “My job is painful!.. and I know
why”
• “I’m always waiting for someone
or being undermined by
someone else”
• “If you just let me do it myself…”
• “We should automate everything
first”
• “Using new tools like company x
will fix our problems!”
Executives Teams
“I think DevOps can help me” “I think DevOps can help me”
Organizational Analysis
Process Redesign
Management Coaching
Software Tools
Design Patterns and Methodologies
Training
73. • “My organization is stuck!… and
I don’t know why”
• “Business pressuring to route
around me”
• “I have no real visibility”
• “Org has low confidence in
ability to deliver”
• “Usual coping mechanisms
aren’t working”
• “Don’t bring me more tools!”
• “My job is painful!.. and I know
why”
• “I’m always waiting for someone
or being undermined by
someone else”
• “If you just let me do it myself…”
• “We should automate everything
first”
• “Using new tools like company x
will fix our problems!”
Executives Teams
“I think DevOps can help me” “I think DevOps can help me”
Organizational Analysis
Process Redesign
Management Coaching
Software Tools
Design Patterns and Methodologies
Training
74. • “My organization is stuck!… and
I don’t know why”
• “Business pressuring to route
around me”
• “I have no real visibility”
• “Org has low confidence in
ability to deliver”
• “Usual coping mechanisms
aren’t working”
• “Don’t bring me more tools!”
• “My job is painful!.. and I know
why”
• “I’m always waiting for someone
or being undermined by
someone else”
• “If you just let me do it myself…”
• “We should automate everything
first”
• “Using new tools like company x
will fix our problems!”
Executives Teams
“I think DevOps can help me” “I think DevOps can help me”
Organizational Analysis
Process Redesign
Management Coaching
Software Tools
Design Patterns and Methodologies
Training
75. Execs
Teams
"I want to transform
my organization"
"I want to transform
how I work"
DevOps?
78. My silo works just
fine
It’s out of my
control
Not the way
we’ve always
done it
That just
won’t work
here
Too busy
running the
business
What is
important?
Didn’t work
last time, why
now?
Where do I
start?
79. Improved Time to Market & Quality
Quicker Feedback
from Users
Business
Idea
Running
Services
Dev Ops
DevOps and Agile
Ops Security
Auditing
Compliance
Availability
Ops is in a particularly tough spot
80. Success: an organization that learns to fix itself
Focus is on:
● Seeing end-to-end
● Fast feedback
● Eliminating waiting, rework
● Catching problems early
Predictability
Short Time-to-Market
High Quality Operations
82. Silos are the #1 enemy of throughput and quality
Dev Release OpsBusiness
83. Silos are the #1 enemy of throughput and quality
Dev Release OpsBusiness
84. Silos are the #1 enemy of throughput and quality
Dev Release OpsBusiness
Handoff
!
Handoff
!
Handoff
!
85. Silos are the #1 enemy of throughput and quality
Dev Release OpsBusiness
Application Knowledge
Handoff
!
Handoff
!
Handoff
!
86. Silos are the #1 enemy of throughput and quality
Dev Release OpsBusiness
Application Knowledge
Operational Knowledge
Handoff
!
Handoff
!
Handoff
!
87. Silos are the #1 enemy of throughput and quality
Dev Release OpsBusiness
Application Knowledge
Operational Knowledge
Business Intent
Handoff
!
Handoff
!
Handoff
!
88. Silos are the #1 enemy of throughput and quality
Dev Release OpsBusiness
Application Knowledge
Operational Knowledge
Business Intent
Handoff
!
Handoff
!
Handoff
!
Ownership
but limited
Accountability
89. Silos are the #1 enemy of throughput and quality
Dev Release OpsBusiness
Application Knowledge
Operational Knowledge
Business Intent
Handoff
!
Handoff
!
Handoff
!
Ownership
but limited
Accountability
Accountability
but no
Ownership
91. Organizational alignment is the key to overcoming silos
• 1000’s of small decisions made daily
• How do we harness that?
92. Organizational alignment is the key to overcoming silos
• 1000’s of small decisions made daily
• How do we harness that?
• Top down is brittle
• Conditions, people, tools are always
changing... how do we handle that?
93. Organizational alignment is the key to overcoming silos
• 1000’s of small decisions made daily
• How do we harness that?
• Top down is brittle
• Conditions, people, tools are always
changing... how do we handle that?
• Alignment:
When you know that different individuals would
independently see a set of conditions and arrive
at the same decision that is directionally correct
to achieve the company’s goals
94. Organizational alignment is the key to overcoming silos
• 1000’s of small decisions made daily
• How do we harness that?
• Top down is brittle
• Conditions, people, tools are always
changing... how do we handle that?
• Alignment:
When you know that different individuals would
independently see a set of conditions and arrive
at the same decision that is directionally correct
to achieve the company’s goals
• Startups have it; Enterprises lose it
98. Value Stream Mapping
Process
1
?
PD
TS
W
EP M
M
M
TS
PDM
W
DWastes
2
?
PD
TS
W
EP M
M
M
TS
PDM
W
D
Ops
involved
early
Unify
ticket/req
tools
Write
service
tests in
dev
Automate
from
spec
System
Verification
Tests
Move
into
Dev
Self-
Service
ACLs
Verify from
definition
of done
Counter-
measures
3
99. Core business service deliver
Current state value stream m
Development
Environments
Release
Build and deploy
QA
Environments
Performance
Environment
Staging
Environment
Productio
Environme
Deploy to
Performance
b Assets,
ws and
eframes
Delta
features and
fixesSet of
service
ranches
Documented
test plans
and cases
Request QA
build
Daily Scrum
meeting
Daily Scrum
meeting
Deployment
Meeting Business Systems
Support
Stan Walters
Change
Control
Project Manager
Erica Switzer
Change
request
Business Analyst
Erica Switzer
Steering
Committee
Erica Switzer
Full
Projects /
Expedites
Estimates
and
requirements
Technical
Specification
Deployment
Document
Change
Control Forms
RT tickets
Jira issues
Jira issues
Tested
packages
Development
Joe Harbaugh
L/T 8 weeks
P/T
H/C
S/R
4 weeks
5
25%
Development
Build and deploy
Joe Harbaugh
L/T 2 days
P/T
H/C
S/R
4 hours
1
10%
Development
Integration testing
Joe Harbaugh
L/T 1 day
P/T
H/C
S/R
1 day
5
95%
Performance
testing
Deploy
Product
Build test plan
Raj Lee
L/T 5 days
P/T
H/C
S/R
4 days
1
10%
Build test cases
Raj Lee
L/T 2 weeks
P/T
H/C
S/R
8 days
2
20%
Fernando Gomez
L/T 8 hours
P/T
H/C
S/R
7 hours
1
2%
Acceptance testing
Raj Lee
L/T 5 days
P/T
H/C
S/R
5 hours
2
10%
Feature testing
Raj Lee
L/T 4 weeks
P/T
H/C
S/R
2.5 weeks
4
20%
Allen Cannata
L/T 1 week
P/T
H/C
S/R
3 days
1
66%
RT tickets
Deploy to Staging
Frances Middleton
L/T 6 days
P/T
H/C
S/R
90 minutes
3
66%
Staging testing
Frances Middleton
L/T 1 week
P/T
H/C
S/R
3.5 days
4
66%
Allen Can
L/T 9 hou
P/T
H/C
S/R
8 hou
6
66%
Frances Middleton
L/T 6 days
P/T
H/C
S/R
90 minutes
3
66%
D
DD
PD
TSx5 Dx2
Wx2 PDx4
EP
D
PDx3 Mx8
D
Wx2
PDx2
What should we be looking for?
• Long lead times (vs processing time)
• Request and approval queues
• Large batch sizes
• Human to human information transfer
• Scrap / Rework / “Backwash”
• Process or tooling inconsistency
• Manual deployment / configuration
• Reliance on manual testing / verification
• Injecting requirements late in the process
• Fixing in place (vs rebuilding)
• “Snowflake” servers and environments
• Manual dependency management
• Ad-hoc or unstructured artifact management
101. Align work to customer value
Dev & QA
Activity
Rel. / SRE
Activity
Ops
Activity
Business
Activity
102. Dev & QA
Activity
Rel. / SRE
Activity
Ops
Activity
Business
Activity
Customer / Product Aligned Teams
Customer / Product Aligned Teams
Customer / Product Aligned Teams
Align work to customer value
103. Dev & QA
Activity
Rel. / SRE
Activity
Ops
Activity
Business
Activity
Customer / Product Aligned Teams
Customer / Product Aligned Teams
Customer / Product Aligned Teams
Environments
as a
Service
QA
as a
Service
Insights
as a
Service
Security
as a
Service
Environments
Team
QA
Team
Monitoring
& Metrics
Team
Security
Team
What about cross-cutting concerns?
105. Dev & QA
Activity
Rel. / SRE
Activity
Ops
Activity
Business
Activity
Customer / Product Aligned Teams
Customer / Product Aligned Teams
Customer / Product Aligned Teams
Environments
as a
Service
QA
as a
Service
Insights
as a
Service
Security
as a
Service
Environments
Team
QA
Team
Monitoring
& Metrics
Team
Security
Team
Align work to customer value
113. What about cross-cutting concerns?
(PO • Dev • Test • SRE)
Cross Functional Delivery Team
CodeTests
Source
Repo
Config
Env
Spec
Run-
book
Auto-
mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
EnvironmentsPackages
SOURCE
QA Security Environments
Monitoring
---
Metrics
114. What about cross-cutting concerns?
(PO • Dev • Test • SRE)
Cross Functional Delivery Team
CodeTests
Source
Repo
Config
Env
Spec
Run-
book
Auto-
mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
EnvironmentsPackages
SOURCE
QA Security Environments
Monitoring
---
Metrics
QA as a
Service
Security as a
Service
Metrics as a
Service
Env. as a
Service
115. What about cross-cutting concerns?
(PO • Dev • Test • SRE)
Cross Functional Delivery Team
CodeTests
Source
Repo
Config
Env
Spec
Run-
book
Auto-
mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
EnvironmentsPackages
SOURCE
pull pull pull pull
QA Security Environments
Monitoring
---
Metrics
QA as a
Service
Security as a
Service
Metrics as a
Service
Env. as a
Service
116. Be a service provider
pull
Cross-Cutting
Concern X
✓ Standardized offerings
✓ Pulled by users (not pushed)
✓ On-demand and self-service
✓ Implementation knowledge not
necessary for normal use
✓ Provider spends their time building
service and coaching users
X as a Service
117. Be a service provider
pull
Cross-Cutting
Concern X
✓ Standardized offerings
✓ Pulled by users (not pushed)
✓ On-demand and self-service
✓ Implementation knowledge not
necessary for normal use
✓ Provider spends their time building
service and coaching users
X as a Service
Google:
“Cloud Operations at
Netflix”
“Actionable Metrics
Netflix”
Roy Rapoport Mark Maun
In this room at
1:30pm!
118. How to start being a service provider
pull
Cross-Cutting
Concern X
X as a Service
119. How to start being a service provider
pull
Cross-Cutting
Concern X
X as a Service
Define your offerings1
120. How to start being a service provider
pull
Cross-Cutting
Concern X
X as a Service
Define your offerings1
121. How to start being a service provider
pull
Cross-Cutting
Concern X
X as a Service
Define your offerings1
Tame the tool sprawl2
122. How to start being a service provider
pull
Cross-Cutting
Concern X
X as a Service
Define your offerings1
Tame the tool sprawl2
123. How to start being a service provider
pull
Cross-Cutting
Concern X
X as a Service
Define your offerings1
Tame the tool sprawl2
Setup self-service interfaces3
124. How to start being a service provider
pull
Cross-Cutting
Concern X
X as a Service
Define your offerings1
Tame the tool sprawl2
Setup self-service interfaces3
125. How to start being a service provider
pull
Cross-Cutting
Concern X
X as a Service
Define your offerings1
Tame the tool sprawl2
Setup self-service interfaces3
Setup secure access4
126. How to start being a service provider
pull
Cross-Cutting
Concern X
X as a Service
Define your offerings1
Tame the tool sprawl2
Setup self-service interfaces3
Setup secure access4
127. How to start being a service provider
pull
Cross-Cutting
Concern X
X as a Service
Define your offerings1
Tame the tool sprawl2
Setup self-service interfaces3
Setup secure access4
Plug: Give Rundeck a try --> rundeck.org
130. Any organization that designs a system
(defined broadly) will produce a design whose
structure is a copy of the organization's
communication structure.
Melvyn Conway, 1967
Conway’s Law:
140. The pain is legendary
“Our cloud is awesome… VMs in minutes… but then it
takes two days to get the VM connected to anything”
“But the ACL policy worked yesterday”
“Firewall rules only get changed on Thursdays”
“I didn’t know the load would spike that way”
“That’ll take 4 tickets and an act of god”
“If we don’t predict traffic patterns right our app is
hosed and there is no quick recovery”
“Only 3 people here know how to do that”
“After all that it turns out it was a DNS issue”
141. The network is changing
1990 2015
95% traffic left the DC New architectures:
5% traffic leaves the DC
(Facebook: only 2%!)
John Willis
142. The demands are changing
• Disaggregation of Hardware/Software
• Web Scale
• Private Cloud Computing
• Containerization
• Software Eating the Network
• DevOps
Faster pace of change!
Decentralized control!
John Willis
147. Reduce cycle time AND improve quality?
Does your solution….
DevOps Litmus Test
148. Reduce cycle time AND improve quality?
Eliminate handoffs or reduce the friction of those
handoffs that can't be eliminated?
Does your solution….
DevOps Litmus Test
149. Reduce cycle time AND improve quality?
Eliminate handoffs or reduce the friction of those
handoffs that can't be eliminated?
Improve tool-to-tool artifact flow and eliminate
manual information flow?
Does your solution….
DevOps Litmus Test
150. Reduce cycle time AND improve quality?
Eliminate handoffs or reduce the friction of those
handoffs that can't be eliminated?
Improve tool-to-tool artifact flow and eliminate
manual information flow?
Eliminate manually-fulfilled request queues and
other sources of waiting?
Does your solution….
DevOps Litmus Test
151. Reduce cycle time AND improve quality?
Eliminate handoffs or reduce the friction of those
handoffs that can't be eliminated?
Improve tool-to-tool artifact flow and eliminate
manual information flow?
Eliminate manually-fulfilled request queues and
other sources of waiting?
Improve awareness and understanding of the
current state and desired state of the end-to-end
system?
Does your solution….
DevOps Litmus Test
152. Reduce cycle time AND improve quality?
Eliminate handoffs or reduce the friction of those
handoffs that can't be eliminated?
Improve tool-to-tool artifact flow and eliminate
manual information flow?
Eliminate manually-fulfilled request queues and
other sources of waiting?
Improve awareness and understanding of the
current state and desired state of the end-to-end
system?
Does your solution….
DevOps Litmus Test
✓
✓
✓
✓
✓