This is the latest, but shortened version of my "Automated Decision Making with Big Data^H^H Predictive Applications" presentation.

2. Automated Decision
Making with Big Data
Lars Trieloff | @trieloff

3. Automated Decision
Making with Big Data
Predictive Applications
Lars Trieloff | @trieloff

4. — Holger Kisker, Forrester Research
“Even after more than 20 years of
using BI, they still base nearly 45%
of business decisions on
qualitative decision factors
instead of quantitative, fact-based
evidence.“

5. If data is not used for decision
making, what is used then?

8. 4%Worldwide average profit
margin in retail: 4%

9. 4‰German average profit margin
in retail: 4‰

10. Your Customer gives you this

11. All you got to keep is that

12. — –Libby Rittenberg
“Economic profits in a system of
perfectly competitive markets
will, in the long run, be driven
to zero in all industries.”

13. Physiological
Safety
Love/Belonging
Esteem
Self-Actualization

14. — Abraham Maslov – probably never said this. It’s true anyway.
“Data has Human Needs, too”

15. Collection
Storage
Analysis
Prediction
Decision

16. Collection
Storage
Analysis
Prediction
Decision
Physiological
Safety
Love/Belonging
Esteem
Self-Actualization

18. — W. Edward Deming
“In God we trust, all others bring
data”

19. How Data-Driven Decisions
should work
Computer
Collects
Computer
Stores
Human
Analyzes
Human
Predicts
Human
Decides

20. — Daniel Kahneman
“Prejudice against algorithms is
magnified when the decisions
are consequential.”

21. How Data-Driven Decisions
REALLY work
Computer
Collects
Computer
Stores
Human
Analyzes
C O M M U N I C AT I O N
B R E A K D O W N
Human
Decides

22. — Led Zeppelin
Communication Breakdown, It's
always the same,
I'm having a nervous
breakdown, Drive me insane!

23. • Drill-down analysis … misunderstood or
distorted
• Metrics dashboards … contradictory and
confusing
• Monthly reports … ignored after two
iterations
• In-house analyst teams … overworked
and powerless
How Data-Driven Decisions
REALLY work
CO M M U N I C AT I O N
B R E A K D O W N

24. How Data-Driven Decisions
REALLY work
http://dilbert.com/strips/comic/2007-05-16/

25. How Decisions REALLY should
work
Computer
Collects
Computer
Stores
Computer
Analyzes
Computer
Predicts
CO M P U T E R
D E C I D E S

26. — Everyone at Blue Yonder, all the time
99.9% of all business decisions
can be automated

27. How Decisions are Being Made

28. 90% No Decision is made

29. — Robin Sharma
“Making no decision is a
decision. To do nothing. And
nothing always brings you
nowhere..”

30. Business Rules for Beginners
Not doing anything is the simplest business
rule in the world – and also the most popular

31. 90% No Decision is made

32. 9% Decision Follows Rule

33. Advanced Business Rules
Computers are machines following rules. This
means business rules are programs.

34. • Business rules are like programs – written by
non-programmers
• Business rules can be contradictory,
incomplete, and complex beyond
comprehension
• Business rules have no built-in feedback
mechanism:“It is the rule, because it is the rule”
Business rules are Programs,
just not very good ones.

35. — Mark Twain
“It ain’t what we don’t know
that causes trouble, it’s what we
know for sure that just ain’t so”

36. 1% Human Decision making

37. Human Decision Making has
two systems – and only one is
rational.

38. Not quite Almost there That’s it.

39. Quick: What do you see here?

41. — Steven Pinker, describing Moravec’s Paradox
“The hard problems are easy
and the easy problems are
hard.”

42. Quick: Add all even numbers
65 7 1 0
60 63 18 80
547039100

43. 94 39 37 31
92 70 100 67
4956080
69 20 26 73

44. 51 60 23 22
5 48 43 14
9525669
23 67 1 43

45. Correct Result:

46. Correct Result: 1.024

48. — Daniel Kahneman
“All of us would be better
investors if we just made fewer
decisions.”

50. How we are making decisions
(Like the big apes we are)
Anchoring effect
IKEA effect
Confirmation bias
Bandwagon effect
Substitution
Availability heuristic
Texas Sharpshooter Fallacy
Rhyme as reason effect
Over-justification effect
Zero-risk bias
Framing effect
Illusory correlation
Sunk cost fallacy
Overconfidence
Outcome bias
Inattentional Blindness
Benjamin Franklin effect
Hindsight bias
Gambler’s fallacy
Anecdotal evidence
Negativity bias
Loss aversion
Backfire effect

52. K-Means Clustering
Naive Bayes
Support Vector Machines
Affinity Propagation
Least Angle Regression
Nearest Neighbors
Decision Trees
Markov Chain Monte Carlo
Spectral clustering
Restricted Bolzmann Machines
Logistic Regression
Computers making decisions
(cold, fast, cheap, rational)

53. • A machine learning algorithm is a system that
derives a set of rules based on a set of data
• It is based on systematic observation, double-
checking and cross-validation
• There is no magic, just data – and without data
there is no magic either
Machine Learning means
Programs that write Programs

54. Better Decisions through
Predictive Applications

55. How Predictive Applications
Work
Collect & Store
Analyze
Correlations
Build Decision
Model
Decide &
Test
Optimize

56. — Warren Buffett
“I checked the actuarial tables,
and the lowest death rate is
among six-year-olds, so I
decided to eat like a six-year-
old.”

57. More than half of the apps on
a typical iPhone home screen
are predictive applications.

58. Building Predictive Applications
Machine Learning ModelPredictive Application
Enterprise Integration

59. Story Time
(Not safe for vegetarians)

60. The Ground Beef Dilemma

61. How much ground
beef are we going to
sell on Friday?

62. How much ground beef are
we going to sell on Friday?
And how much on
Saturday?

63. Challenge #1 Accurately predict
demand

64. Great. But how much
do we need to order
each day?

65. Great. But how much
do we need to order
each day?
Let’s reduce the risk
of running out of
stock to 20%

66. Sales Forecasts for Friday
SalesProbability
0
0,01
0,02
0,03
0,04
0 4 8 12 16
Friday Sales Amount

67. Sales Forecasts for Saturday
SalesProbability
0
0,01
0,02
0,03
0,04
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Saturday Sales Amount

68. Great. But how much
do we need to order
each day?
Let’s reduce the
risk of running out
of stock to 20%
So it’s 3 on Friday and
5,5 on Saturday.

69. Sales Forecasts for Both Days
SalesProbability
0
0,01
0,02
0,03
0,04
0 4 8 12 16
Friday Sales Amount Saturday Sales Amount

70. Bad news…

71. Bad news…
We need to skip the
Saturday delivery.

72. Bad news…
We need to skip the
Saturday delivery.
How big should we
make the Friday delivery
instead?

73. If you need 3 on Friday and 5,5
on Saturday to fulfill 80% of
the demand, how much do you
need to fulfill 80% of the
combined demand?

74. 3 + 5,5 = 8,5
Common Sense isn’t it?

75. — Albert Einstein
Common sense is what tells us
the world is flat.

76. Combined Sales Forecasts
SalesProbability
0
0,01
0,02
0,03
0,04
0 4 8 12 16
Combined Sales Amount

77. If you ordered 8,5 cases, you
would waste a lot of meat,
the ideal order amount is 8
cases.

78. Predictive Apps in a Nutshell
Batch and streaming data ingestion, batch
and streaming delivery (with real-time option)
Reduce risk and cost » increase revenue and profit
Trend Estimation Classification Event Prediction
Optimize Returns
Collect Data Predict Results Drive Decisions

79. — John Maynard Keynes
“When my information
changes, I alter my conclusions.
What do you do, sir?”

80. One Common Platform for
Predictive Applications
Your own and third-
party data, easily
integrated via API
Link
Build Machine
Learning and
application code
Build
Automatically run
and scale ML models
and applications
Run
Monitor and inspect
resource usage and
model quality
View
Your data stored in
high-performance
database as a service
Store

81. — Kevin Kelly
“The business plans of the next
10,000 startups are easy to
forecast: Take X and add AI”

82. Lars Trieloff
@trieloff
(this guy is hiring)