Another installment and iteration of my talk on predictive applications, automated decision making and why cognitive biases prevent us from making the best decisions at scale

2. Automated Decision
Making with Big Data
Lars Trieloff | @trieloff

3. Automated Decision
Making with Big Data
Predictive Applications
Lars Trieloff | @trieloff

4. — Holger Kisker, Forrester Research
“Even after more than 20 years of
using BI, they still base nearly 45%
of business decisions on
qualitative decision factors
instead of quantitative, fact-based
evidence.“

5. 4%Worldwide average profit
margin in retail: 4%

6. 4‰German average profit margin
in retail: 4‰

7. Your Customer gives you this

8. All you got to keep is that

9. — –Libby Rittenberg
“Economic profits in a system of
perfectly competitive markets
will, in the long run, be driven
to zero in all industries.”

10. Physiological
Safety
Love/Belonging
Esteem
Self-Actualization

11. — Abraham Maslov – probably never said this. It’s true anyway.
“Data has Human Needs, too”

12. Collection
Storage
Analysis
Prediction
Decision

13. Collection
Storage
Analysis
Prediction
Decision
Physiological
Safety
Love/Belonging
Esteem
Self-Actualization

15. — W. Edward Deming
“In God we trust, all others bring
data”

16. How Data-Driven Decisions
should work
Computer
Collects
Computer
Stores
Human
Analyzes
Human
Predicts
Human
Decides

17. — Daniel Kahneman
“Prejudice against algorithms is
magnified when the decisions
are consequential.”

18. How Data-Driven Decisions
REALLY work
Computer
Collects
Computer
Stores
Human
Analyzes
C O M M U N I C AT I O N
B R E A K D O W N
Human
Decides

19. — Led Zeppelin
Communication Breakdown, It's
always the same,
I'm having a nervous
breakdown, Drive me insane!

20. • Drill-down analysis … misunderstood or
distorted
• Metrics dashboards … contradictory and
confusing
• Monthly reports … ignored after two
iterations
• In-house analyst teams … overworked
and powerless
How Data-Driven Decisions
REALLY work
CO M M U N I C AT I O N
B R E A K D O W N

21. How Data-Driven Decisions
REALLY work
http://dilbert.com/strips/comic/2007-05-16/

22. How Decisions REALLY should
work
Computer
Collects
Computer
Stores
Computer
Analyzes
Computer
Predicts
CO M P U T E R
D E C I D E S

23. — Everyone at Blue Yonder, all the time
99.9% of all business decisions
can be automated

24. How Decisions are Being Made

25. 90% No Decision is made

26. — Robin Sharma
“Making no decision is a
decision. To do nothing. And
nothing always brings you
nowhere..”

27. Business Rules for Beginners
Not doing anything is the simplest business
rule in the world – and also the most popular

28. 90% No Decision is made

29. 9% Decision Follows Rule

30. Business Rules in Action

31. Advanced Business Rules
Computers are machines following rules. This
means business rules are programs.

32. • Business rules are like programs – written by
non-programmers
• Business rules can be contradictory,
incomplete, and complex beyond
comprehension
• Business rules have no built-in feedback
mechanism:“It is the rule, because it is the rule”
Business rules are Programs,
just not very good ones.

33. — Mark Twain
“It ain’t what we don’t know
that causes trouble, it’s what we
know for sure that just ain’t so”

34. 1% Human Decision making

35. Human Decision Making has
two systems – and only one is
rational.

36. Not quite Almost there That’s it.

37. Quick: What do you see here?

39. — Steven Pinker, describing Moravec’s Paradox
“The hard problems are easy
and the easy problems are
hard.”

40. Quick: Add all even numbers
65 7 1 0
60 63 18 80
547039100

41. 94 39 37 31
92 70 100 67
4956080
69 20 26 73

42. 51 60 23 22
5 48 43 14
9525669
23 67 1 43

43. Correct Result:

44. Correct Result: 1.024

46. — Daniel Kahneman
“All of us would be better
investors if we just made fewer
decisions.”

48. How we are making decisions
(Like the big apes we are)
Anchoring effect
IKEA effect
Confirmation bias
Bandwagon effect
Substitution
Availability heuristic
Texas Sharpshooter Fallacy
Rhyme as reason effect
Over-justification effect
Zero-risk bias
Framing effect
Illusory correlation
Sunk cost fallacy
Overconfidence
Outcome bias
Inattentional Blindness
Benjamin Franklin effect
Hindsight bias
Gambler’s fallacy
Anecdotal evidence
Negativity bias
Loss aversion
Backfire effect

50. • Abraham Lincoln and John F. Kennedy were both
presidents of the United States, elected 100 years
apart.
• Both were shot and killed by assassins who were
known by three names with 15 letters, John Wilkes
Booth and Lee Harvey Oswald, and neither killer
would make it to trial.
• Lincoln had a secretary named Kennedy, and
Kennedy had a secretary named Lincoln.
• They were both killed on a Friday while sitting
next to their wives, Lincoln in the Ford Theater,
Kennedy in a Lincoln made by Ford.

51. K-Means Clustering
Naive Bayes
Support Vector Machines
Affinity Propagation
Least Angle Regression
Nearest Neighbors
Decision Trees
Markov Chain Monte Carlo
Spectral clustering
Restricted Bolzmann Machines
Logistic Regression
Computers making decisions
(cold, fast, cheap, rational)

52. • A machine learning algorithm is a system that
derives a set of rules based on a set of data
• It is based on systematic observation, double-
checking and cross-validation
• There is no magic, just data – and without data
there is no magic either
Machine Learning means
Programs that write Programs

53. Better Decisions through
Predictive Applications

54. How Predictive Applications
Work
Collect & Store
Analyze
Correlations
Build Decision
Model
Decide &
Test
Optimize

55. Why Test?

56. — Randall Munroe
“Correlation doesn’t imply
causation, but it does waggle its
eyebrows suggestively and
gesture furtively while
mouthing‘look over there’”

57. — Warren Buffett
“I checked the actuarial tables,
and the lowest death rate is
among six-year-olds, so I
decided to eat like a six-year-
old.”

58. More than half of the apps on
a typical iPhone home screen
are predictive applications.

59. Fast DataInsight
Big Data
Categorizing Analytics
Past Present Future
No DataHindsight
Foresight
1. By Data Volume 2. By Time Horizon

60. 1
Categorizing Analytics
Descriptive
• Focused on gathering and
collecting data
• Key challenges: data volume
and data variety
• Key outcome: hindsight
• Examples: reports, dashboards
• Answers “What happened?”
Predictive
• Focused on understanding
and explaining data
• Key challenges: data velocity
and complexity
• Key outcome: insight
• Examples: prediction models
• Answers: “Why did it happen
and what will happen next?”
Prescriptive
• Focused on anticipating and
recommending action
• Key challenges: execution
• Key outcome: foresight
• Examples: decision support,
predictive apps
• Answers: “What should we do?”
2 3

61. A
Categorizing Analytics
Explicit
• Analytics are a key visible
feature of the program
• Programs are used by trained
analysts and data scientists
• Regular interaction during
business hours
Integrated
• Analytics are included in
another program
• Analytics are consumed in-
context by business users
• Frequent, but irregular
consumption during business
hours
Automated
• Analytics are invisibly part of a
complex process
• Decisions are made and
executed in the process
• Constant and ongoing
optimization 24/7
B C

62. Analytic Application Matrix
2
3
B
C
+
+
=
=
Predictive Integrated
AutomatedPrescriptive
Decision Support
systems for infrequent
strategic decision-
making
Predictive Applications
for massive, automated
decision-making in
operational processes

63. Building Predictive Applications
Machine Learning ModelPredictive Application
Enterprise Integration

64. Story Time
(Not safe for vegetarians)

65. The Ground Beef Dilemma

66. How much ground
beef are we going to
sell on Friday?

67. How much ground beef are
we going to sell on Friday?
And how much on
Saturday?

68. Challenge #1 Accurately predict
demand

69. Great. But how much
do we need to order
each day?

70. Great. But how much
do we need to order
each day?
Let’s reduce the risk
of running out of
stock to 20%

71. Sales Forecasts for Friday
SalesProbability
0
0,01
0,02
0,03
0,04
0 4 8 12 16
Friday Sales Amount

72. Sales Forecasts for Saturday
SalesProbability
0
0,01
0,02
0,03
0,04
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Saturday Sales Amount

73. Great. But how much
do we need to order
each day?
Let’s reduce the
risk of running out
of stock to 20%
So it’s 3 on Friday and
5,5 on Saturday.

74. Sales Forecasts for Both Days
SalesProbability
0
0,01
0,02
0,03
0,04
0 4 8 12 16
Friday Sales Amount Saturday Sales Amount

75. Bad news…

76. Bad news…
We need to skip the
Saturday delivery.

77. Bad news…
We need to skip the
Saturday delivery.
How big should we
make the Friday delivery
instead?

78. If you need 3 on Friday and 5,5
on Saturday to fulfill 80% of
the demand, how much do you
need to fulfill 80% of the
combined demand?

79. 3 + 5,5 = 8,5
Common Sense isn’t it?

80. — Albert Einstein
Common sense is what tells us
the world is flat.

81. Combined Sales Forecasts
SalesProbability
0
0,01
0,02
0,03
0,04
0 4 8 12 16
Combined Sales Amount

82. If you ordered 8,5 cases, you
would waste a lot of meat,
the ideal order amount is 4
cases.

83. Predictive Apps in a Nutshell
Batch and streaming data ingestion, batch
and streaming delivery (with real-time option)
Reduce risk and cost » increase revenue and profit
Trend Estimation Classification Event Prediction
Optimize Returns
Collect Data Predict Results Drive Decisions

84. — John Maynard Keynes
“When my information
changes, I alter my conclusions.
What do you do, sir?”

85. One Common Platform for
Predictive Applications
Your own and third-
party data, easily
integrated via API
Link
Build Machine
Learning and
application code
Build
Automatically run
and scale ML models
and applications
Run
Monitor and inspect
resource usage and
model quality
View
Your data stored in
high-performance
database as a service
Store

86. — Kevin Kelly
“The business plans of the next
10,000 startups are easy to
forecast: Take X and add AI”

87. Lars Trieloff
@trieloff