Updated version here (Dec 2019): https://www.slideshare.net/realgenekim/the-unicorn-project-and-the-five-ideals-updated-dec-2019 It is impossible to overstate how much I’ve learned since co-authoring The Phoenix Project, DevOps Handbook, and Accelerate. I’m so excited that after years of work, The Unicorn Project will be published later this year. This book is my attempt to frame what I’ve learned studying technology leaders adopting DevOps principles and patterns in large, complex organizations, often having to fight deeply entrenched orthodoxies. And yet, despite huge obstacles, they create incredibly effective and innovative teams that create beacons of greatness that inspire us all. In this book, we follow a senior lead developer and architect as she is exiled to the Phoenix Project, to the horror of her friends and colleagues, as punishment for contributing to a payroll outage. She tries to survive in what feels like a heartless and uncaring bureaucracy, forced to work within a system where no one can get anything done without endless committees, paperwork, change requests, and approvals. Decades of technical debt make even small changes difficult or impossible, often causing catastrophic outcomes and fear of punishment. I get tremendous delight and gratification that this book is not about the bridge crew of the Starship Enterprise -- instead, it is about redshirt engineers, which as it turns out, whose heroic work matters most to the long-term survival of almost every organization. In my previous books, I’ve focused on principles and practices (e.g., Three Ways, Four Types of Work). However, I’ve always wanted to describe the spectrum of cultural, experiential and value decisions we make that either enable greatness, or create chronic suffering and underperformance. They are currently as follows: • The First Ideal — Locality and Simplicity • The Second Ideal — Focus, Flow and Joy • The Third Ideal — Improvement of Daily Work • The Fourth Ideal — Psychological Safety • The Fifth Ideal — Customer Focus In this talk, I’ll share with you my goals and aspirations for The Unicorn Project, describe in detail the Five Ideals, along with my favorite case studies of both ideal and non-ideal, and why I believe more than ever that DevOps will be one of the most potent economic forces for decades to come.

1. @RealGeneKim
Session ID:
Gene Kim
The Unicorn Project
And The Five Ideals

2. @RealGeneKim
2013 2016 2017

3. @RealGeneKim
 Nearly 3 years, 1600 hours of
work
 Publication date: November 26
 Wanted to capture the heroic
journeys of the DevOps
Enterprise community

4. @RealGeneKim
There’s Never Been A Better Time
for Infrastructure and Operations

5. @RealGeneKim
The Five Ideals
1. Locality and Simplicity
2. Focus, Flow, and Joy
3. Improvement of Daily Work
4. Psychological Safety
5. Customer Focus

6. @RealGeneKim
Session ID:
Ideal #1:
Locality and Simplicity

7. @RealGeneKim
The Birth And Death Of Etsy Sprouter
 A story about teams of engineers implementing
changes
 2008: Devs and DBAs
 2009: Devs and DBAs and Sprouter team
 2010: Devs

8. @RealGeneKim
Lesson:
The Organization and
The Architecture Of Our Software
Must Be Congruent

9. @RealGeneKim
Lead Time = 9 months
Source: Damon Edwards (@damonedwards)

10. @RealGeneKim
Architecture Enables Teams To…
 …make large scale changes to the design of its system without the
permission of someone outside the team, or depending on other
teams
 ...complete its work without fine-grained communication and
coordination with people outside the team
 ...deploy and release its product or service on demand, independently
of other services the product or service depends upon
 ...do most of its testing on demand, without requiring an integrated
test environment
 ...perform deployments during normal business hours with negligible
downtime
Source: Puppet/DORA: 2017 State Of DevOps Report: https://puppet.com/resources/whitepaper/state-of-devops-report

11. @RealGeneKim
The First Ideal: Organization
 Ideal: any team can independently develop, test,
and deploy value to the customer
 Not Ideal: to deploy value to the customer, every
team must coordinate with tens of other teams,
and any of them can prevent it

12. @RealGeneKim
The First Ideal: A Measure
 Bus Factor
 Lunch Factor

13. @RealGeneKim
How Many People Do You Need To Feed?
 Two pizza team
 Feeding everyone in the building
 Schedule lunch with 43 different people

14. @RealGeneKim
The First Ideal: Organization
 Ideal: every team has the expertise, capability
and authority to satisfy customer needs
 Not Ideal: in order to satisfy customer needs,
every team must escalate up two levels (and over
two, and down two)

15. @RealGeneKim
The First Ideal: Code
 Ideal: anyone can implement what they need by
looking at one file or module, and make the
needed change
- Kubernetes sidecars
 Not Ideal: to make your needed change, you
have to understand all the files and modules

16. @RealGeneKim
Hotel Wi-Fi Story

17. @RealGeneKim
Session ID:
Ideal #2:
Focus, Flow, and Joy

18. @RealGeneKim
As Your Ambassador From Dev
 For decades, I self-identified as an Ops person…
 2 years ago, I’ve started to self-identify as Dev
 Clojure / ClojureScript
 LISP, functional programming, immutability
 3000 lines of Objective C -> 1500 lines of TypeScript/React -
> 500 lines of ClojureScript
 Development is so fun, and these days, you can do
miraculous things with so little effort

19. @RealGeneKim
Why Functional Programming
 The famous French philosopher Claude Lévi-Strauss
would say of certain tools, ‘is it good to think with?’
 Core FP concepts
 Immutability
 Pure functions
 Composability
 Pioneered by Haskell and Ocaml. Popularized by
Clojure, Erlang, Elm, Elixir

20. @RealGeneKim
Never Have I Valued Infrastructure More
 Things I detest now
 Everything outside of my application
 Connecting to anything to anything
 Secrets management
 Bash
 YAML
 Patching
 Building kubernetes deployment files (mostly by
Googling)
 Why my cloud costs are so high

21. @RealGeneKim
Interestingly, It Portends Future Of Ops
 Core concepts
 Immutability
 Pure functions
 Composability
 Look at…
 Docker, Docker Compose
 Kubernetes
 Kubernetes sidecars
 Event streams: Apache Kafka

22. @RealGeneKim
The Second Ideal: Focus and Flow
 Ideal: your energy and time is focused on solving
the business problem, and you’re having fun
 Not Ideal: all your time is spent trying to solve
problems you don’t even want to solve (e.g.,
YAML files, Makefile and spaces in filenames,
bash)

23. @RealGeneKim
Two Types Of Learning
● Procedural Learning
● One-shot Learning

24. @RealGeneKim
 Ideally, I can implement my business feature in
one place, and I have all the expertise I need to
implement it

25. @RealGeneKim

26. @RealGeneKim

27. @RealGeneKim

28. @RealGeneKim
The Value Of Platforms
 Enable developer productivity
 Self-service
 On-demand
 Immediacy and fast feedback
 Focus and flow
 Joy
 Monitoring, deployment, environment creation,
security scans, orchestration…

29. @RealGeneKim
 “bash: the disease you die with, but don’t die of…”
Jeffrey Snover
Technical Fellow, Microsoft
@jsnover

30. @RealGeneKim
The Second Ideal: Focus and Flow
 Ideal: trunk based development
 Not Ideal: 5 days merging, 50 people in
conference rooms

31. @RealGeneKim

32. @RealGeneKim
Session ID:
Ideal #3:
Improvement of Daily Work

33. @RealGeneKim
Not Ideal
“In manufacturing, the absence of effective feedback often
contribute to major quality and safety problems. In one well-
documented case at the General Motors Fremont manufacturing
plant, there were no effective procedures in place to detect
problems during the assembly process, nor were there explicit
procedures on what to do when problems were found.
“As a result, there were instances of engines being put in
backward, cars missing steering wheels or tires, and cars even
having to be towed off the assembly line because they wouldn’t
start.”
Source: DevOps Handbook

34. @RealGeneKim
Create as much feedback in our system, from as
many areas in our system, sooner, faster, and
cheaper, with as much clarity between cause and
effect.
Why? Because the more assumptions we can
invalidate, the more we learn, improving our ability
to fix problems and innovate.
Source: DevOps Handbook
Ideal

35. @RealGeneKim

36. @RealGeneKim
How many times per day is the andon cord
pulled in a typical day at a Toyota
manufacturing plant?
3,500 times per day
Source: http://www.gembapantarei.com/2008/04/how_many_times_do_you_pull_the_andon_cord_each_day.html

37. @RealGeneKim
Fast Push To Market
Debts & Risks
Features
Quality
Defects

38. @RealGeneKim
Fast Push To Market — Continued
Features
Defects
Defect fixing dominates work
Site reliability tanks
Slower and slower velocity
Customers leave
Morale plunges
Devs leave because everything is hard
Quality
Debts & Risks

39. @RealGeneKimSource: https://twitter.com/johncutlefish/status/1046169469268111361
Who hasn’t felt this?
You hire a bunch of developers, but you
still can’t ship the features you
promised…
…and maybe you even have the feeling
that things are slowing down…

40. @RealGeneKimSource: The Unicorn Project (2019)

41. @RealGeneKim
Near Death Experiences
● Ebay (1999)
● Microsoft (2002): Bill Gates memo
● Google (2005): Automated testing culture
● Amazon (2004): Jeff Bezos memo
● Twitter (2008)
● LinkedIn (2009)
● Etsy (2009)

42. @RealGeneKim
2002 Microsoft Security
Standdown
 Famously, Microsoft after
SQL Slammer required
every product group to
freeze feature
Source: https://www.wired.com/2002/01/bill-gates-trustworthy-computing/

43. @RealGeneKim
The Feature Freeze / Standdown
Debt
Features
Quality
Defects
Features

44. @RealGeneKim

45. @RealGeneKim
Quote from Marty Cagan from his book
Inspired
The deal [between product owners and] engineering goes like this: Product
management takes 20% of the team’s capacity right off the top and gives this to
engineering to spend as they see fit. They might use it to rewrite, re-architect, or
re-factor problematic parts of the code base…​whatever they believe is necessary
to avoid ever having to come to the team and say, ‘we need to stop and rewrite [all
our code].’ If you’re in really bad shape today, you might need to make this 30% or
even more of the resources. However, I get nervous when I find teams that think
they can get away with much less than 20%.
Cagan notes that when organizations do not pay their “20% tax,” technical debt
will increase to the point where an organization inevitably spends all of its cycles
paying down technical debt. At some point, the services become so fragile that
feature delivery grinds to a halt because all the engineers are working on reliability
issues or working around problems.

46. @RealGeneKimSource: Satya Nadella, CEO, Microsoft (@satyanadella)

47. @RealGeneKim
First Ideal
 Ideal: 3-5% of developers dedicated to improving
developer productivity
 Not ideal: assigned to summer interns and
“people not good enough to be developers”

48. @RealGeneKim
The Third Ideal: Improvement
 Not Ideal: No one cares if someone breaks the
build, or checks in code that breaks our tests
 Ideal: When someone breaks our build or our
tests, fixing it becomes the most important work
of the moment

49. @RealGeneKim
The Third Ideal: Improvement
 Not ideal: When someone needs a peer review,
that person has to wait until someone else frees
up
 Ideal: Whatever I’m working on, if someone
needs a peer review, I drop whatever I’m doing to
help

50. @RealGeneKim
"Automated tests transform fear into boredom."
-- Eran Messeri, Google
Google Dev And Ops (2013)
 15,000 engineers, working on 4,000+ projects
 All code is checked into one source tree
(billions of files!)
 5,500 code commits/day
 75 million test cases are run daily

51. @RealGeneKim

52. @RealGeneKim

53. @RealGeneKim
Session ID:
Ideal #4:
Psychological Safety

54. @RealGeneKim
One Of The Highest Predictors Of
Performance
Source: Typology Of Organizational Culture (Westrum, 2004)

55. @RealGeneKim
One Of The Highest Predictors Of
Performance
Source: Typology Of Organizational Culture (Westrum, 2004)

56. @RealGeneKim
One Of The Highest Predictors Of
Performance
Source: Typology Of Organizational Culture (Westrum, 2004)

57. @RealGeneKim
Google: Project Aristotle, Oxygen, re:Work
Source: https://rework.withgoogle.com/blog/five-keys-to-a-successful-google-team/

58. @RealGeneKim

59. @RealGeneKim

60. @RealGeneKim

61. @RealGeneKim
Great Practices Enabled
 Blameless post-mortems
 Chaos Monkeys

62. @RealGeneKim
72

63. @RealGeneKim
Inject Failures Often

64. @RealGeneKim
You Don’t Choose Chaos Monkey…
Chaos Monkey Chooses You

65. @RealGeneKim
“Then I remembered all the Chaos Monkey
exercises we’ve gone through. My reaction
was, ‘Bring it on!’”
The 2014 AWS Reboot
“When we got the news about the emergency EC2
reboots, our jaws dropped. When we got the list of
how many Cassandra nodes would be affected, I
felt ill.
– Christos Kalantzis
Netflix Cloud DB EngineeringSource: http://techblog.netflix.com/2014/10/a-state-of-xen-chaos-monkey-cassandra.html

66. @RealGeneKim
The 2014 AWS Reboot
“Out of our 2700+ production Cassandra nodes,
218 were rebooted. 22 Cassandra nodes did not
reboot successfully.
“Netflix customers experienced no downtime that
weekend.”
– Bruce Wong
Netflix Chaos Engineering

67. @RealGeneKim
Netflix and Service Catalog

68. @RealGeneKim
Tom Limoncelli Quote (@yesthattom)

69. @RealGeneKim
Session ID:
Ideal #5:
Customer Focus

70. @RealGeneKim
Session ID:
DevOps Is For The Unicorns…
...And The Horses, Too

71. @RealGeneKim
DevOps Enterprise: Lessons Learned
 In 2018, we’ll hold the fifth year of the DevOps Enterprise Summit, a
conference for horses, by horses
 Over the years, we’ve had over 200 leaders from:
 Capital One, KeyBank, Barclays, GE Capital, ING Bank, Fidelity, PNC, ADP, BofA,
Western Union, BBVA
 Nationwide Insurance, Zurich Insurance, Hiscox, Aviva, LV=
 Walmart, Nordstrom, Target, Macy’s, Marks and Spencer
 Nike, Adidas, Sherwin Williams
 Verizon, Telstra, T-Mobile, Orange, CSG
 Raytheon, Lockheed Martin, Northrop Grumman, CSRA, Jaguar Land Rover
 Disney, Ticketmaster, NBC/Universal
 Kaiser Permanente
 US Citizenship & Immigration Services, UK HM Revenue Collection, DISA Forge.mil, NZ
Ministry of Social Development, UK Welfare and Pensions, US Joint Warfare Analysis
Center
 Amazon PrimeNow, CA, Compuware, Google Search, IBM, MicroFocus, Microsoft, SAP

72. @RealGeneKim

73. @RealGeneKimSource: Puppet/DORA: 2017 State Of DevOps Report: https://puppet.com/resources/whitepaper/state-of-devops-report

74. @RealGeneKim
Leadership Matters
 Teams with the least reported transformational
leadership behaviors (the bottom-third) were one-
half as likely to be high IT performers
 Leaders cannot do it alone! Teams with the top
10% of reported transformational leadership
behaviors performed no better than the median
Source: Puppet/DORA: 2017 State Of DevOps Report: https://puppet.com/resources/whitepaper/state-of-devops-report

75. @RealGeneKim
The Fifth Ideal: Focus On The Customer
 Not ideal: Functional silo managers behave like
union leaders, as opposed to business leaders
 Ideal: Functional silo managers make decisions
based on what the customer values, and helps
ensure their teams have the skills to thrive in the
long term

76. @RealGeneKim

77. @RealGeneKim
The Fifth Ideal: Focus On The Customer
 Core vs. Context

78. @RealGeneKim
Why Do I Think This Is
Important?

79. @RealGeneKim

80. @RealGeneKim

81. @RealGeneKim
 Publication date: November 26
 Excerpts will be released in the
next three weeks

82. @RealGeneKim

83. @RealGeneKim

84. @RealGeneKim
Want More Learn More?
To receive this presentation and the following:
 Announcement and upcoming excerpts from
The Unicorn Project
 Eight excerpts from Beyond The Phoenix Project audio
series w/John Willis
 The 140 page excerpt of The DevOps Handbook
 The 140 page excerpt of The Phoenix Project
 Videos and slides from DevOps Enterprise 2014-2019
 One hour excerpt of The Phoenix Project audiobook
Just pick up your phone, and send an email:
To: realgenekim@SendYourSlides.com
Subject: devops
realgenekim@SendYourSlides.com
devops