A way too long but entertaining talk given at the September 2015 Cloud Foundry Meetups in Vancouver and Calgary, Canada. Content is a mashup of my own slides and from many colleagues @ Pivotal.

1. PLATFORM CLOUDS,
CONTAINERS,
IMMUTABLE INFRA,
OH MY!
1
Stuart Charlton

2. 2
YOUR HOST
2
Stuart Charlton
@svrc
Pivotal Software, my dream company
Led IT Ops & Cloud Architecture
at a Railway (long story)
Former CTO of an early, tragically
executed startup
Assistant in the destruction of the
global economy while on Wall Street
ex-BMC, BEA, Rogers, Infusion

3. PRIOR TO TAKEOFF
• There are many opinions, these are mine
• Some nuance emphasized, some lost
• There will be pictures of nude containers
• There will be digging into Cloud Foundry
3

4. WHY WE ARE HERE
Software is Hard
4

5. AND YET…
• amazon.com
• Mean time between deployments: 11.6s
(in 2011, it’s better now)
• Max deployments in a single hour: 1,079
• Mean hosts receiving a deploy: 10,000
• Max hosts receiving a deploy: 30,000
5

6. WHY WE ARE HERE
• All businesses are software businesses
or losing to one that is
• Demands are getting “impossible” (Mobile, IoT)
• We keep making the same mistakes
• We must deploy at scale quickly and safely
• We (devs/ops) must reclaim our lives from the
mess we’ve made
6

7. THERE’S A LOT OF
CONFUSION OUT THERE
• Hopefully this talk gives a useful roadmap
• Where do things fit, how things “sort of” work
• Be able to call bullshit on statements like…
• X is going to take over the world
• It’s game over for Y
• This one weird trick is all you need
7

8. CLAIMS
• The current trend of the IT industry is to
forcefully eliminate IT operations as we
traditionally know it*
• Stop feeding blood to the machines
• Stop sweating over config/change management
• Evolving from toolchains to platforms
• Leverage: Instead of “operating” software,
we build and maintain
software that operates the software
• *inspired by the professional movements in devops, microservices, continuous delivery;
provocative wording borrowed from Todd Underwood (Google) and his PostOps LISA 2013 talk
8

9. SO?
• We all have operating platforms
• Not everyone’s platform is great
• What makes a platform good?
• Patterns and Constraints in Context
• Encourages good architecture
• Opinionated, not free form
• Will you build your own,
or join a community? 9

10. THIS TALK
• Let’s design a cloud native platform
• Just these easy steps…
• Interlude: Two philosophies of systems
• Converged and Immutable
• Interlude: What’s in a Container?
• Containers, Docker, Droplets
• Interlude: Schedulers for Fun and Profit
• Cloud Foundry Diego
• Combining this all into Cloud Native Platforms
• Designing for Cloud Native 10

11. LET’S DESIGN A CLOUD
NATIVE PLATFORM
In the time allotted
We Hope
11

12. WHAT PROBLEM ARE
WE TRYING TO SOLVE?
12

13. FOLLOW THE
THOUGHT CHAIN
• Some people started solving problem X
and then moved to problem Y
13

14. FOLLOW THE MONEY
• Some people tried to solve problem X,
found it hard to sell or get funding,
moved to problem Y
• This can occur up or down the value chain
14

15. 2001-2006
WE HAVE TOO MANY
SERVERS TO
PROVISION, CONFIG,
INSTALL
15

16. 1. CONVERGENT CONFIG
MANAGEMENT
• Aka “These Scripts Need to Grow Up”
16
Model&Oriented Action&Oriented
• Focused on single server config

17. 2006-2010
CLOUDS ARE FORMED,
MORE THINGS TO DO
• Assume you have an Infrastructure Cloud
• Servers, Networks, Storage, Disks
• On Demand, Fungible Resources
• Many nuances & details here matter
but we need to punt on them today
17

18. 2. CLOUD
ORCHESTRATION
• Make the clouds sing
• “I need X disk attached to Y compute on Z
network”
18
Cloud Formation / HEAT
Terraform / BOSH

19. DAWN OF TIME-2015
“I HATE MY PACKAGE
MANAGER”
• Slow, versioning, dependencies, hard to
build, hard to share, etc.
• I know, I’ll build a new package manager!
19

20. 3. VARIOUS UNITS OF
SHARE, INSTALL, DEPLOY
• There are basically six types
• Deployment Artifact
• OS Package Installer
• OS Container
• Tarball/ZIP
• VM Image
• Composite Release 20

21. WHAT’S IN A PACKAGE
STANDARD?
• File System Format
• Metadata
• Build Script
• Registry
• Dependencies
• Metadata
• Processes
21

22. COMPARING PACKAGES
22
Droplet VM Image Docker
File System Tarball Block Tarball deltas
Metadata YES OVA YES
Build Script Buildpack Various Dockerfile
Registry Droplet Reg Various Docker Reg
Dependencies NOPE NOPE NOPE
Processes YES OVA YES

23. WHAT DOCKER GOT
RIGHT AS A PACKAGE
• Vendor your dependencies
• Make sharing fast (copy-on-write / deltas)
• Make launching fast (OS containers)
• Enable social sharing of full runtimes
(Docker hub)
• A cross-distro “swiss army knife” package
23

24. WHAT DROPLETS GOT
RIGHT AS A PACKAGE
• Droplets: Heroku / Cloud Foundry
• Vendor your dependencies
• Make updating fast (deltas of your dev artifacts)
• Make launching fast (OS virtualization)
• Awesome developer experience
(standard build/deploy in 60 sec that still uses
your familiar artifacts) 24

25. DOCKER IS HERE,
LET’S GO HOME?
25

26. OPEN QUESTIONS
• Do you want to run just any Docker image?
• Opaque contents are a nightmare to maintain
• Opaque contents are a nightmare to secure
• Performance with layered Union FS
• Containers are successful in production with the
proper constraints
26

27. INTERLUDE 2:
TWO PHILOSOPHIES
• Converged Config Managers were
created before clouds were a big deal
• Definitely before containers were a big deal
• Didn’t really have ability to config clouds
(they do now)
• Clouds changed some assumptions on
how we treat servers 27

28. TRADITIONAL INFRA
• “collection of pets”
• pet them, hug them, love them, name them
after Lord of the Rings characters
• upgrade them when they die
28

29. CLOUD INFRA
• Fungible, ephemeral minions
• Fast to start up and dispose
29

30. CONVERGED CONFIG
• Phoenix servers, should be able to re-create
• In practice, lots of silent dependencies
• Day 2? Each evolution requires tinkering on
the manifests & scripts
• Constraints, opinions are up to the user
30

31. IMMUTABLE INFRA
• Immutable by contract, not fully
• s/immutable/(disposable | prefabricated)/
• e.g. assume these things never
change, and if they do, kill them
• Ops tinkering happens in the build
• Ops pipeline rarely changes,
just deploy new stuff
• Use load balancer / router for control 31

32. BOSH:
A PLATFORM FOR COMPOSITE PACKAGES +
DISPOSABLE SERVERS
32

33. IF A CONTAINER IS A
PACKAGE,
WHAT’S IN A
CONTAINER?
33

34. ?

35. ?
isolation
?

36. ? isolation
shared resources
processA
processB
processC
processD
processE
processF
kernel
tenant 1 tenant 2 tenant 3

37. ? isolation
shared resources
kernel
resource isolation
namespace isolation
processA
processB
processC
processD
processE
processF
tenant 1 tenant 2 tenant 3

38. ? isolation
CPU
kernel
resource isolation
namespace isolation
processA
processB
processC
processD
processE
processF
tenant 1 tenant 2 tenant 3

39. ? isolation
resource isolation
namespace isolation
CPU
processA
processB
processC
processD
processE
processF
tenant 1 tenant 2 tenant 3

40. ? isolation
resource isolation
namespace isolation
processA
processB
processC
processD
processE
processF
tenant 1 tenant 2 tenant 3
CPU

41. ? isolation
resource isolation
namespace isolation
processA
processB
processC
processD
processE
processF
tenant 1 tenant 2 tenant 3
CPU

42. ? isolation
resource isolation
namespace isolation
processA
processB
processC
processD
processE
processF
tenant 1 tenant 2 tenant 3
cgroups
CPU

43. ? isolation
resource isolation
namespace isolation
processA
processB
processC
processD
processE
processF
tenant 1 tenant 2 tenant 3
cgroups
processD
processE
processF
CPU

44. ? isolation
shared resources
kernel
resource isolation
namespace isolation
processA
processB
processC
processD
processE
processF
tenant 1 tenant 2 tenant 3

45. ? isolation
kernel
resource isolation
namespace isolation
processA
processB
processC
processD
processE
processF
tenant 1 tenant 2 tenant 3
ProcessID

46. ? isolation
resource isolation
namespace isolation
processA
processB
processC
processD
processE
processF
tenant 1 tenant 2 tenant 3
PID 2 3 4 5 6 7

47. ? isolation
resource isolation
namespace isolation
processA
processB
processC
processD
processE
processF
tenant 1 tenant 2 tenant 3
PID 2 3 4 5 6 7

48. ? isolation
resource isolation
namespace isolation
processA
processB
processC
processD
processE
processF
tenant 1 tenant 2 tenant 3
PID 2 3 4 5 6 7
PID namespace

49. ? isolation
resource isolation
namespace isolation
processA
processB
processC
processD
processE
processF
tenant 1 tenant 2 tenant 3
PID 2 3 4 5 6 7
PID namespace

50. ? isolation
resource isolation
namespace isolation
processA
processB
processC
processD
processE
processF
tenant 1 tenant 2 tenant 3
PID 2 3 4 2 2 3
PID namespace

51. ? isolation
resource isolation
namespace isolation
processA
processB
processC
processD
processE
processF
tenant 1 tenant 2 tenant 3
PID
shared resources
kernel
Network
Mount
User
namespaces

52. ?
?

53. ?
? =
isolation
User
Network
cgroups
PID

54. ? isolation
resource isolation
namespace isolation
processA
processB
processC
processD
processE
processF
tenant 1 tenant 2 tenant 3
PID 2 3 4 5 6 7

55. ?
?
? =
isolation
PID
User
Network
cgroups

56. ?
? =
isolation
PID
User
Network
cgroups
+
contents

57. ?
? =
isolation
PID
User
Network
cgroups
+
contents
+
processes

58. ?
? =

59. ?
Problem 4:
How do I run
these containers
at scale, safely?
Schedulers

60. ?
DIEGO

61. ? DIEGO is
a distributed system that
orchestrates containerized workloads

62. ? DIEGO is
a distributed system that
orchestrates containerized workloads

63. ? DIEGO is
a distributed system that
orchestrates containerized workloads

64. ? DIEGO is
a distributed system that
orchestrates containerized workloads

65. ? DIEGO
a distributed system that orchestrates containerized workloads
Cells
Brain
BBS
(currently etcd)

66. ? DIEGO
a distributed system that orchestrates containerized workloads
Cells
Brain
BBS
(currently etcd)
scheduler

67. ? DIEGO
a distributed system that orchestrates containerized workloads
Cells
Brain
BBS
(currently etcd)
scheduler

68. ? DIEGO
a distributed system that orchestrates containerized workloads
Cells
Brain
BBS
(currently etcd)
scheduler

69. ? DIEGO
a distributed system that orchestrates containerized workloads
Cells
Brain
BBS
(currently etcd)
health-monitor

70. ? DIEGO
a distributed system that orchestrates containerized workloads
Cells
Brain
BBS
(currently etcd)
health-monitor

71. ? DIEGO
a distributed system that orchestrates containerized workloads
Cells
Brain
BBS
(currently etcd)
health-monitor

72. ? DIEGO
a distributed system that orchestrates containerized workloads
Cells
Brain
BBS
(currently etcd)
health-monitor

73. ? DIEGO
a distributed system that orchestrates containerized workloads
Cells
Brain
BBS
(currently etcd)
health-monitor

74. ? DIEGO
a distributed system that orchestrates containerized workloads
Cells
Brain
BBS
(currently etcd)
health-monitor

75. ? DIEGO
a distributed system that orchestrates containerized workloads
Cells
Brain
BBS
(currently etcd)
health-monitor

76. ? DIEGO
a distributed system that orchestrates containerized workloads
Cells
Brain
BBS
(currently etcd)
health-monitor

77. ? DIEGO runs
a distributed system that orchestrates containerized workloads
one-off tasks
long running
processes

78. ? DIEGO runs
a distributed system that orchestrates containerized workloads
Task LRP
generic, platform independent, abstraction

79. ? DIEGO runs
a distributed system that orchestrates containerized workloads
Task LRP
generic, platform independent, abstraction

80. ? DIEGO runs
a distributed system that orchestrates containerized workloads
Task LRP
working today
generic, platform independent, abstraction

81. ? DIEGO runs
a distributed system that orchestrates containerized workloads
Task LRP
successful abstraction
working today

82. ?
Tasks
LRPs
in Garden

83. ?
Cloud Foundry Garden
allows you to programmatically say
“make me a container” “put this in it” “then run this”
via a platform-agnostic API

84. ?
Garden
allows Diego’s abstractions to be flexible

85. ?
cf push

86. ?
cf push haiku
here is my source code
i do not care how
run it on the cloud for me

87. ?
app
source
code
Task
staging
cf push

88. ? cf push
buildpack compiled asset
app + app-specific dependencies
assumes a particular execution context
cflinuxfs2

89. ? cf push
?

90. ? cf push
LRP

91. ? cf push

92. ? cf push
cflinuxfs2
preloaded rootfs

93. ? cf push
cflinuxfs2
preloaded rootfs
download droplet

94. ? cf push
cflinuxfs2
preloaded rootfs
download droplet
start command

95. ?
cf push

96. ?
cf push-docker

97. ? cf push-docker

98. ? cf push-docker
docker image

99. ? cf push-docker
docker image docker metadata

100. ? cf push-docker
docker image docker metadata
docker registry
}

101. ? cf push-docker
docker image docker metadata
docker registry
}

102. ?
?

103. ?
?
(anything)

104. ?
?
(anything)

105. ?
?
(anything) (anything)

106. ?
?
(anything) (anything)

107. ?
?

108. ?
?
appc

109. ?
cf push-docker

110. ?
cf push -stack windows

111. ?
Garden-Windows
resource isolation
kernel job object
disk quotas
namespace isolation
user profiles
Host Web Core
(an isolated IIS instance)
Garden-Linux
resource isolation
cgroups
namespace isolation
PID
Network
User
Mount

112. ?
collaborating with Microsoft
Garden-Windows

113. ?
Garden-Windows
provides a container experience for Windows 2012
that will only get better with Windows 2016
allows us to build a cf push experience

114. ?
3 different contexts

115. ? 1 cluster

116. 2015+
CONTAINER MUD
WRESTLING
• Schedulers abound!
• Mesos, Kubernetes, Cloud Foundry Diego
• Deis, Flynn, Dokku, etc
• How does one choose?
116

117. 5. STRUCTURED
PLATFORMS AND
FRAMEWORKS
• ie. something that will encourage good
architecture and operational constraints
• Don’t do undifferentiated heavy lifting
• Be productive right away
• Join a community making these design
decisions together 117

119. Cells
BrainBBS
ReceptorAPI
Task or LRP

120. Cells
BrainBBS
ReceptorAPI
Task or LRP
meh

121. Cells
BrainBBS
ReceptorAPI
Task or LRP
gorouter
http traffic

122. Cells
BrainBBS
ReceptorAPI
Task or LRP
gorouter
http traffic
loggregator
logs

125. vagrant up

126. vagrant up
terraform apply

127. vagrant up
terraform apply
ltc create <app>

128. lattice.cf

129. lattice.cf
Local VM

130. lattice.cf
Local VM
AWS
Digital Ocean
Google Cloud Platform
OpenStack (thanks!)

131. +
BOSH
Multi-Tenant API
Single Sign On
Capacity & Performance Mgt
Service Brokerage
Service Marketplace
=

132. CLOUD NATIVE
FRAMEWORKS
132
Spring Cloud Services
Config Server Service Registry Circuit Breaker
Dashboard

133. SUMMARY OF PLATFORM
CAPABILITIES
• Immutable Infrastructure all the way
down
• VMs AND Containers
• BOSH-like management AND container
scheduler AND reliable container staging/
build
133

134. SUMMARY OF PLATFORM
CAPABILITIES
• Layer 7 Dynamic
Routing/Load Balancing
(Layer 3 is nice too)
• Log Aggregation
• Multi-Tenant API
• Authentication & Authorization
134

135. SUMMARY OF PLATFORM
CAPABILITIES
• Capacity Management (Metrics Stream)
• Application Performance Management
(Response Times)
• Service Discovery, Brokerage & Marketplace
• Multi-Tenant API
• Authentication & Authorization
• Cloud Native Frameworks 135

136. THE CHOICE
• Build all that from tools
• Hope you get the right constraints &
patterns nailed
• Adopt a structured platform
• Fill any gaps with toolchain
136

137. DESIGNING FOR
CLOUD NATIVE
• 12 Factor
• Microservices
• Support Services
• Discovery, Config, Circuit Breaking
137

138. 12 FACTORS:
A CONTRACT
138
•One Codebase/Many Deploys
•Explicit Isolated Dependencies
•Config via Environment
•Attached Backing Services
•Dev/Prod Parity
•Separate Build/Release/Run
•Ephemeral Processes
•Export Services via Port Bindings
•Scale Out via Processes
•Disposable Instances
•Logs == Event Streams
•Admin Tasks == Processes
Factors for the Developer Factors for App Architecture

139. MONOLITHIC
ARCHITECTURE
139
Relational Database
Data Access
Service
HTML JavaScript MVC
Service
Monolithic ApplicationBrowser

140. MONOLITHIC
ARCHITECTURES
• Modularity Dependent Upon Language /
Frameworks
• Change Cycles Tightly Coupled
• Inefficient Scaling
• Can Be Intimidating to New Developers
• Obstacle to Scaling Development
• Requires Long-Term Commitment to
Technical Stack 140

141. MICROSERVICES
141
…
HTTP
HTTP
HTTP
HTTP
HTTP
HTTP
AMQP
AMQP
Relational
DB
Key/Value
Store
Graph
DB

142. MICROSERVICES
• Services Oriented Architecture AND
Services-Oriented Delivery
• Modularity Based on Bounded Contexts
• Enable Frequent Deploys, Efficient Scaling
• Individual Components Less Intimidating to New
Developers
• Enables Scaling of Development…
142

143. CONWAY’S LAW
143
Any organization that designs a system (defined broadly)
will produce a design whose structure is a copy of the
organization's communication structure.
Melvyn Conway, 1967
http://martinfowler.com/articles/microservices.html#OrganizedAroundBusinessCapabilities

144. SPAN SILOS WITH
MICROSERVICES
144
Data Access
Service
HTML JavaScript MVC
Service
UISpecialists
Middleware
Specialists
DBAs
BusinessCapability
BusinessCapability
BusinessCapability
Siloed
Functional
Teams
http://martinfowler.com/articles/microservices.html#OrganizedAroundBusinessCapabilities
Siloed
Application
Architectures
Cross-
functional
Teams
Microservice
Architectures

145. PARTITIONING
• By Noun (e.g. product info service)
• By Verb (e.g. shipping service)
• Single Responsibility Principle
(http://programmer.97things.oreilly.com/wiki/
index.php/
The_Single_Responsibility_Principle)
145

146. YOU MUST BE THIS TALL
•RAPID PROVISIONING
•BASIC MONITORING
•RAPID APPLICATION
DEPLOYMENT
•DEVOPS CULTURE
146
http://martinfowler.com/bliki/MicroservicePrerequisites.html
https://www.flickr.com/photos/gusset/3723961589

147. ONE PIECE
CONTINUOUS FLOW
147
Product
Mgr
UX Dev QA DBA
Sys
Admin
Net
Admin
Storage
Admin
BUSINESS CAPABILITY TEAMS
USING MICROSERVICES
PLATFORM OPERATIONS
TEAM
Self
Service
API

148. SUMMARY
• The current trend of the IT industry is to forcefully eliminate IT
operations as we traditionally know it
• Cloud Native Applications
• Microservices, Continuously Delivered on a Platform
• Lots of experimentation between the DIY and Structured approach
• Do it Yourself - Evolution of the toolchain approach
• Structured - Community centered around sharing an architecture
codebase & culture
• Cloud Foundry is currently the most successful structured
platform
• Many DIY pieces are growing fast in popularity (Docker, K8S, etc)
148

149. GET THE BOOKS
149

150. WITH THANKS
150
This presentation includes content / inspiration by:
Matt Stine
Andrew Clay Shafer
Onsi Fakouri
+ many others @ Pivotal
+ Todd Underwood @ Google