Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2noNSfO. Michelle Brush discusses modeling complex systems and architectural changes that could introduce new modes of failure, using examples from embedded systems to large stream processing pipelines. Filmed at qconsf.com. Michelle Brush works as an Engineering Director for Cerner Corporation, where she is responsible for the data ingestion and processing platform for Cerner’s Population Health solutions. She also leads several engineering education programs and culture initiatives including Cerner’s software architect development program and internal developer conference.

1. Framing Our
Potential for Failure
Michelle Brush

2. InfoQ.com: News & Community Site
• Over 1,000,000 software developers, architects and CTOs read the site world-
wide every month
• 250,000 senior developers subscribe to our weekly newsletter
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• 2 dedicated podcast channels: The InfoQ Podcast, with a focus on
Architecture and The Engineering Culture Podcast, with a focus on building
• 96 deep dives on innovative topics packed as downloadable emags and
minibooks
• Over 40 new content items per week
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
architecture-failure-modes

3. Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Presented at QCon San Francisco
www.qconsf.com

4. Michelle Brush
Engineering Director, Cerner Corporation
Chapter Leader, Kansas City Girl Develop It
Conference Organizer, Midwest.io
@michellebrush

5. Michelle Brush
Engineering Director, Cerner Corporation
Chapter Leader, Kansas City Girl Develop It
Conference Organizer, Midwest.io
@michellebrush

6. My Mom:
What is it that you do again?

7. Michelle Brush
Software Engineer, Various
Chapter Leader, Kansas City Girl Develop It
Conference Organizer, Midwest.io
@michellebrush

8. My Mom:
What is it that you do again?

9. Me:
I worry about failures and
manage them.

10. Me:
Sometimes I fail.

11. Me:
usually due to people stuff

12. Me:
I, as a person, don’t scale well.

13. Me:
so I teach people to worry
about failure and manage it

14. Me:
redundancy

15. Me:
Sometimes that fails too.

16. Michelle Brush
78.5 million unique lives
6.8 million managed lives
6.3 petabytes of data
Engineering Director, Population Health

18. data

19. computer
data

23. Big Data

26. redundancy

28. risk

29. updates?
?
?

30. updates
?
?
can fail.

31. It gets worse.

32. ?
?

33. ?
?
There’s no guarantee that things arrive in order.
8:01 8:04 8:00
8:00

34. ?
?
There’s no guarantee that they arrive at all.
8:01 8:04 8:00
8:00

35. ?
?
There are still more ways the system can be wrong,
than we can possibly reason about.
8:01 8:04 8:00
8:00

36. Welcome to The Cloud.

37. Welcome to The Cloud.

38. embedded systems

39. embedded systems
the tiny houses of engineering

40. 200 sq. ft. of living space
Foundation optional
Emphasis on design to optimize reuse
of space.

41. 1 GB of hard disk
1 MB of main memory
CPU that’s competitive with the average
computer in the late 90’s.

42. I can reuse my combined
toilet/shower as my closet.

44. While the specific technologies we use
may be the means by which we fail,
they are not the cause.

45. You need technology to manage it,
but if you don't even know how to
worry about failure, you won’t
know where to start.

46. I’m going to change the subject.

47. everyone’s favorite topic: Software Estimation

48. How many man-weeks/iterations/people
will it take to do this project?

49. We’re typically wrong by 30%.

50. confidence.

51. What is the distance in light years to
Alpha Centauri?
What year was helium discovered?

52. lower and upper bounds
with 90% confidence

53. What is the distance in
light years to Alpha
Centauri?

54. What is the distance in
light years to Alpha
Centauri?
10 - 1000

55. What is the distance in
light years to Alpha
Centauri?
10 - 1000
4.367

56. What year was helium
discovered?

57. What year was helium
discovered?
1850 - 1950

58. What year was helium
discovered?
1850 - 1950
1868

59. 6 classes
32 people in each
192 people
best score: 7/10
1:32

60. overconfidence.

61. overoptimism.

62. bias.

63. structure beats bias.

64. eat your vegetables.

65. 1. Develop system model.
2. Record known risk areas.
3. Publish model and risk areas.
4. Perform regular risk reviews. (Premortems)
5. Dissect and document missed risks.

66. 1. Develop system model.
2. Record known risk areas.
3. Publish model and risk areas.
4. Perform regular risk reviews. (Premortems)
5. Dissect and document missed risks.

67. nailed it!

68. The behavior of a system cannot be known
just by knowing the elements of which the
system is made.

69. “Accidents occur due to relationships
not components.”
- Sidney Dekker
Drift Into Failure: From Hunting Broken
Components to Understanding Complex
Systems

70. We deal with a lot of relationships.

71. temporal
We deal with a lot of relationships.

72. temporal
spatial
We deal with a lot of relationships.

73. temporal
spatial
causal
We deal with a lot of relationships.

74. temporal
spatial
familial
causal
We deal with a lot of relationships.

76. System Model
logical development
physicalprocess
scenarios

77. logical
How would a user
reason about the
system?

78. development
How would the
developer reason about
the system?

79. development
What do I deploy?

80. physical
How would a system
engineer reason about
the system?

81. physical
What’s the hardware/
networking profile?

82. process
How would the
operating system
reason about the
system?

83. process
How are things
communicating?
Are they doing things at
the same time?

84. System Model
logical development
physicalprocess
scenarios

85. That seems like a lot of work.

86. Stocks,
Flows, &
Feedback Loops
+

87. Systems Thinking

88. W_Minshull, Hot Tub, CC BY 2.0
https://www.flickr.com/photos/23950335@N07/5497464077/in/photostream/

89. W_Minshull, Hot Tub, CC BY 2.0
https://www.flickr.com/photos/23950335@N07/5497464077/in/photostream/
STOCK

90. W_Minshull, Hot Tub, CC BY 2.0
https://www.flickr.com/photos/23950335@N07/5497464077/in/photostream/
STOCK
FLOW

91. W_Minshull, Hot Tub, CC BY 2.0
https://www.flickr.com/photos/23950335@N07/5497464077/in/photostream/
STOCK
FLOW
FEEDBACK
LOOP

92. “creating mental models.”
- Charles Duhrigg
Smarter Faster Better

93. “Models help us choose where to direct our
attention, so we can make decisions, rather than
just react.”
- Charles Duhrigg
Smarter Faster Better

94. CLOSED LIST EXAMPLE

95. A*
graph: nodes and edges that need to be
searched.
open list: a list of nodes that haven’t
been explored.
closed list: a list of nodes that have been
explored.
a graph search
algorithm

96. Open List Closed List
A
B
2
C
3
2
D
4
2
E
4
1
4
My Example Graph
0
2
2
1
3

97. Open List Closed List
A 3
A
B
2
C
3
D
4
My Example Graph
2
2
1
3

98. Open List Closed List
B 4
C 4
D 6
A 3
A
B
2
C
3
D
4
My Example Graph
2
2
1
3

99. Open List Closed List
A 3
A
B
2
C
3
2
D
4
E
4
My Example Graph
0
2
2
1
3
B 4
C 4
D 6

100. Open List Closed List
A 3
B 4
A
B
2
C
3
2
D
4
E
4
My Example Graph
0
2
2
1
3
C 4
D 6
E 6

101. 1 GB of hard disk
1 MB of main memory
CPU that’s competitive with the average
computer in the late 90’s.

104. open

105. open
open
list

106. close
open
open
list

107. close
open
open
list
closed
list

108. close
open
open
list
found
new
nodes?
closed
list

109. close
open
open
list
visited
before?
found
new
nodes?
closed
list

110. close
open
open
list
visited
before?
closed
list
found
new
nodes?

111. SMALL FILES EXAMPLE

113. thousands a day

114. thousands a day
everyday for a year

115. thousands a day
everyday for a year

116. thousands a day
everyday for a year
In the file were identifiers that needed to be globally unique.

117. thousands a day
everyday for a year
In the file were identifiers that needed to be globally unique.
You could change anything in the file passively -
except that identifier.

118. thousands a day
everyday for a year
In the file were identifiers that needed to be globally unique.
You could change anything in the file passively -
except that identifier.
If you change the identifier , you have to
reprocess the file.

119. thousands a day
everyday for a year
In the file were identifiers that needed to be globally unique.
You could change anything in the file passively -
except that identifier.
If you change the identifier , you have to
reprocess the file.
Someone changed all of the identifiers.

120. hundreds of thousands
of small files
I hate small files.

124. ingest

125. ingest
data
sink

126. ingest
data
sink
update

127. ingest
data
sink
update
record

128. ingest
data
sink
update
record set last
update
marker

129. ingest
data
sink
update
record
rebuild
set last
update
marker

130. ingest
data
sink
update
record
rebuild
set last
update
marker

131. ingest
data
sink
update
record
rebuild
set last
update
marker

132. timing model

133. 1. Ingest some data.
2.Transform it.
3.Map some attributes.
4.Link up some data.
5.Calculate the results.
Loosely scheduled map-reduce jobs.

134. transformingavarietyofdata
mapping
some
data
linking
some
data
calculating
some
results
9:01
9:30
10:00
9:48
10:18
10:15
10:32
8:45
1 hour 47 minutes
latency
9:00
9:05
9:00
9:31
9:00
9:21
9:00
8:45 10:32

135. transformingavarietyofdata
mapping
some
data
linking
some
data
calculating
some
results
9:30
10:00
9:48
10:18
10:15
10:32
8:45
9:01
9:00
9:05
9:00
9:31
9:00
9:21
9:00

136. another big data transportation analogy

138. A simpler metaphor for the nature of our
complex systems helps identify the essential
failure points.

139. It can provide a means to
identify a solution.

140. It can be better.

141. nominal time
“Show up at 8:45, but don’t leave until the bus is full.”

142. Real Systems
Real Problems
transformingavarietyofdata
mapping
some
data
linking
some
data
calculating
some
results
9:01
9:32
10:00
9:49
10:18
10:19
10:36
8:45
1 hour 51 minutes
latency
9:00
9:05
9:00
9:31
9:00
9:21
9:00
8:45 10:36

143. Pick the model(s) that work for you.

144. 1. Develop system model.
2. Record known risk areas.
3. Publish model and risk areas.
4. Perform regular risk reviews. (Premortems)
5. Dissect and document missed risks.

145. time
unknownunknowns
In the beginning, you might
get it all wrong.

146. 1. Develop system model.
2. Record known risk areas.
3. Publish model and risk areas.
4. Perform regular risk reviews. (Premortems)
5. Dissect and document missed risks.

148. RISK-1234

149. RISK-1234
RISK-1234

150. do not erase!!!

151. If you want your people to make good
decisions,they need context to more than
just their piece of the system.

152. Big, visible diagrams
are effective
learning aides.

153. 1. Develop system model.
2. Record known risk areas.
3. Publish model and risk areas.
4. Perform regular risk reviews. (Premortems)
5. Dissect and document missed risks.

154. Premortems
1. Inform everyone the system has failed.
2. Ask the group to identify most likely causes.
3. Adjust the plan to account for those risks.

155. 1. List ALL changes going in the release.
2. Review each change in terms of the risk matrix.
3. If it’s RED, it doesn’t go. (Pull the cord.)
4. If it’s ORANGE, we need a monitoring & mitigation plan.
5. If it’s YELLOW, we need at least a mitigation.
Risk Reviews

156. 1. List ALL changes going in the release.
2. Review each change in terms of the risk matrix.
3. If it’s RED, it doesn’t go. (Pull the cord.)
4. If it’s ORANGE, we need a monitoring & mitigation plan.
5. If it’s YELLOW, we need at least a mitigation.
Risk Reviews

157. 1. List ALL changes going in the release.
2. Review each change in terms of the risk matrix.
3. If it’s RED, it doesn’t go. (Pull the cord.)
4. If it’s ORANGE, we need a monitoring & mitigation plan.
5. If it’s YELLOW, we need at least a mitigation.
Risk Reviews

158. low
medium
high
low medium high
impact
likelihood

159. How do you measure likelihood & impact?

160. I don’t.

161. low
medium
high
low medium high
impact
likelihood

162. 1. List ALL changes going in the release.
2. Review each change in terms of the risk matrix.
3. If it’s RED, it doesn’t go. (Pull the cord.)
4. If it’s ORANGE, we need a monitoring & mitigation plan.
5. If it’s YELLOW, we need at least a mitigation.
Risk Reviews

163. low
medium
high
low medium high
impact
likelihood

164. 1. List ALL changes going in the release.
2. Review each change in terms of the risk matrix.
3. If it’s RED, it doesn’t go. (Pull the cord.)
4. If it’s ORANGE, we need a monitoring & mitigation plan.
5. If it’s YELLOW, we need at least a mitigation.

165. low
medium
high
low medium high
impact
likelihood

166. 1. List ALL changes going in the release.
2. Review each change in terms of the risk matrix.
3. If it’s RED, it doesn’t go. (Pull the cord.)
4. If it’s ORANGE, we need a monitoring & mitigation plan.
5. If it’s YELLOW, we need at least a mitigation.

167. low
medium
high
low medium high
impact
likelihood

168. What if we get this wrong too?

171. return to a well-understood model
systems thinking:

172. “Models help us choose where to direct our
attention, so we can make decisions, rather than
just react.”
- Charles Duhrigg
Smarter Faster Better

174. W_Minshull, Hot Tub, CC BY 2.0
https://www.flickr.com/photos/23950335@N07/5497464077/in/photostream/
STOCK
FLOW
FEEDBACK
LOOP

175. close
open
open
list
visited
before?
closed
list
found
new
nodes?

176. ?
?

177. observe
hypothesize
experiment
refine or reject
validate (repeat)

178. Let’s write down what we know to be true.
(again)

180. 1. Develop system model.
2. Record known risk areas.
3. Publish model and risk areas.
4. Perform regular risk reviews. (Premortems)
5. Dissect and document missed risks.

181. post-incident retrospective

182. not just the technology stuff

183. 1. List ALL changes going in the release.
2. Review each change in terms of the risk matrix.
3. If it’s RED, it doesn’t go. (Pull the cord.)
4. If it’s ORANGE, we need a monitoring & mitigation plan.
5. If it’s YELLOW, we need at least define a mitigation.
It’s really, really hard to pull the cord.

184. “I think we’re not ready, and I
already told leadership we’re
going to be late, should I go tell
them I’m wrong?”

185. Leaders need to be aware that
framing matters.

186. 1. Develop system model.
2. Record known risk areas.
3. Publish model and risk areas.
4. Perform regular risk reviews. (Premortems)
5. Dissect and document missed risks.
As you go through this process…

187. time
unknownunknowns

188. time
unknownunknowns
Over time, you’ll get more right.

189. This feels very similar to…

190. analogy-based risk assessment

191. analogy-based assessment
leveraging a collection of previous
experiences to reflect on how a new
situation might play out

192. fail
bank of
analogies
retrospective
risks

193. analogy-based assessment
This is what experience gives us.

194. 1. Develop system model.
2. Record known risk areas.
3. Publish model and risk areas.
4. Perform regular risk reviews. (Premortems)
5. Dissect and document missed risks.
Process

195. Process externalizes the things
great engineers have internalized.

196. It’s a way of thinking out loud.

197. Thinking out loud is a great
coaching tool.

198. Me:
redundancy

199. Me:
I teach people to worry about
failure and manage it

200. My mom is a very extroverted person.
She thinks out loud.
I learned to think like her.

201. Thank you for letting me think out loud.
Michelle Brush
Engineering Director, Cerner Corporation
Chapter Leader, Kansas City Girl Develop It
Conference Organizer, Midwest.io
@michellebrush

202. References

203. Thinking in Systems
Donella H. Meadows

204. Thinking in Systems
Donella H. Meadows

205. Smarter Faster Better
Charles Duhigg

206. Thank you, again.
Michelle Brush
Engineering Director, Cerner Corporation
Chapter Leader, Kansas City Girl Develop It
Conference Organizer, Midwest.io
@michellebrush

207. Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
architecture-failure-modes