SlideShare a Scribd company logo
1 of 85
Download to read offline
How to Get Value out of
Big Data?
Lars Trieloff | @trieloff
How to get value out of this?
Big Data is what you call data
when it becomes hard to deal
with.
— widely quoted, still questionable
“The volume of business data
worldwide doubles every 1.2
years, according to estimates.”
What about Moore’s law? I’ve
heard computers get twice as
fast every 18 month.
— Gordon Moore, 1965
“The complexity for minimum component
costs has increased at a rate of roughly a
factor of two per year. Certainly over the short
term this rate can be expected to continue, if
not to increase. Over the longer term, the rate
of increase is a bit more uncertain, although
there is no reason to believe it will not remain
nearly constant for at least 10 years.”
CPUs are not getting much
faster anymore
But they still get
smaller, cheaper, less
power-hungry, allow
more parallel
operations, etc.
Why can’t we use this?
Big Data is just data that is too
big for your computer.
If computers are getting so
small and cheap, can’t we just
use more computers?
Volume
Velocity
Variety
“What do you do when you
have a problem you can’t
solve?”
“What do you do when you
have a problem you can’t
solve?”
“Just delegate it, and make it
someone else’s problem!”
The cloud. Also known as:
“someone else’s problem”
— Jerry Maguire
“Show me the money value!”
Physiological
Safety
Love/Belonging
Esteem
Self-Actualization
— Abraham Maslov – probably never said this. It’s true anyway.
“Data has Human Needs, too”
Collection
Storage
Analysis
Prediction
Decision
Collection
Storage
Analysis
Prediction
Decision
Physiological
Safety
Love/Belonging
Esteem
Self-Actualization
— W. Edward Deming
“In God we trust, all others bring
data”
How Data-Driven Decisions
should work
Computer
Collects
Computer
Stores
Human
Analyzes
Human
Predicts
Human 

Decides
— Daniel Kahneman
“Prejudice against algorithms is
magnified when the decisions
are consequential.”
How Data-Driven Decisions
REALLY work
Computer
Collects
Computer
Stores
Human
Analyzes
C O M M U N I C AT I O N
B R E A K D O W N
Human 

Decides
— Led Zeppelin
Communication Breakdown, It's
always the same,
I'm having a nervous
breakdown, Drive me insane!
• Drill-down analysis … misunderstood or
distorted
• Metrics dashboards … contradictory and
confusing
• Monthly reports … ignored after two
iterations
• In-house analyst teams … overworked
and powerless
How Data-Driven Decisions
REALLY work
CO M M U N I C AT I O N
B R E A K D O W N
How Decisions REALLY should
work
Computer
Collects
Computer
Stores
Computer
Analyzes
Computer
Predicts
CO M P U T E R 

D E C I D E S
— Everyone at Blue Yonder, all the time
99.9% of all business decisions
can be automated
How Decisions are Being Made
90% No Decision is made
— Robin Sharma
“Making no decision is a
decision. To do nothing. And
nothing always brings you
nowhere..”
Business Rules for Beginners
Not doing anything is the simplest business
rule in the world – and also the most popular
90% No Decision is made
9% Decision Follows Rule
Advanced Business Rules
Computers are machines following rules. This
means business rules are programs.
• Business rules are like programs – written by
non-programmers
• Business rules can be contradictory,
incomplete, and complex beyond
comprehension
• Business rules have no built-in feedback
mechanism:“It is the rule, because it is the rule”
Business rules are Programs,
just not very good ones.
1% Human Decision making
Human Decision Making has
two systems – and only one is
rational.
Not quite Almost there That’s it.
Quick: What do you see here?
— Steven Pinker, describing Moravec’s Paradox
“The hard problems are easy
and the easy problems are
hard.”
Quick: Add all even numbers
65 7 1 0
60 63 18 80
547039100
94 39 37 31
92 70 100 67
4956080
69 20 26 73
51 60 23 22
5 48 43 14
9525669
23 67 1 43
Correct Result:
Correct Result: 1.024
— Daniel Kahneman
“All of us would be better
investors if we just made fewer
decisions.”
How we are making decisions
(Like the big apes we are)
Anchoring effect
IKEA effect
Confirmation bias
Bandwagon effect
Substitution
Availability heuristic
Texas Sharpshooter Fallacy
Rhyme as reason effect
Over-justification effect
Zero-risk bias
Framing effect
Illusory correlation
Sunk cost fallacy
Overconfidence
Outcome bias
Inattentional Blindness
Benjamin Franklin effect
Hindsight bias
Gambler’s fallacy
Anecdotal evidence
Negativity bias
Loss aversion
Backfire effect
K-Means Clustering
Naive Bayes
Support Vector Machines
Affinity Propagation
Least Angle Regression
Nearest Neighbors
Decision Trees
Markov Chain Monte Carlo
Spectral clustering
Restricted Bolzmann Machines
Logistic Regression
Computers making decisions
(cold, fast, cheap, rational)
• A machine learning algorithm is a system that
derives a set of rules based on a set of data
• It is based on systematic observation, double-
checking and cross-validation
• There is no magic, just data – and without data
there is no magic either
Machine Learning means
Programs that write Programs
Better Decisions through
Predictive Applications
How Predictive Applications
Work
Collect & Store
Analyze
Correlations
Build Decision
Model
Decide &

Test
Optimize
— Warren Buffett
“I checked the actuarial tables,
and the lowest death rate is
among six-year-olds, so I
decided to eat like a six-year-
old.”
More than half of the apps on
a typical iPhone home screen
are predictive applications.
Building Predictive Applications
Machine Learning ModelPredictive Application
Enterprise Integration
Story Time
Verlag Grossist Einzelhändler
Improved Demand Forecasts
0
175
350
525
700
sales forecast
time series model
month
last sale
earlier sales
promotion
special issue
Hierarchical Model
Basic Demand per
Store and Publication
Weekly
Sales
Publication
Name
# of
Customers
Geo-
Location
Month
Type of
Store
Demo-
graphics
Demand Forecast
Month
Last year’s
sales
Recent
sales
Promotion
Sales Forecasts for Saturday
SalesProbability
0,01
0,02
0,03
0,04
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Saturday Sales Amount
Sales & Remission Forecasts
Circulation
Remissions Sales
Remissions per issue sold
Sales
Remissions
Business Case
Circulation
Profit Revenue Costs
Results
&
Umfang
Optimierte
Auflagensteuerung im
Einzelhandel
• ca. 2000 Einzelhändler
• 2 Jahre Historie
• 5 Zeitschriften
• Optimierte Remissionszahlen
• Senkung von Ausverkäufen
• Automatisierung der Absatzprognosen
Predictive Apps in a Nutshell
Batch and streaming data ingestion, batch
and streaming delivery (with real-time option)
Reduce risk and cost » increase revenue and profit
Trend Estimation Classification Event Prediction
Optimize Returns
Collect Data Predict Results Drive Decisions
— John Maynard Keynes
“When my information
changes, I alter my conclusions.
What do you do, sir?”
One Common Platform for
Predictive Applications
Multi-Tenant Runtime Environment
Link Store Build Run View
Link your own and
third-party data, easily
integrated via API
Store your data in
high-performance
database as a service
Build machine
learning and
application code
Automatically run
and scale ML models
and applications
Monitor and inspect
resource usage and
model quality
Secure Micro Cloud Infrastructure
Domain Model Predictive Model Application Code
— Kevin Kelly
“The business plans of the next
10,000 startups are easy to
forecast: Take X and add AI”
How Enterprises adopt
Predictive Applications
Learn about
ADDD
Define Target
Process
Build
Predictive App
Go Live
Make Lots of
Money
— Daniel Kahneman
“Prejudice against algorithms is
magnified when the decisions
are consequential.”
How Enterprises REALLY adopt
Predictive Applications
Learn about
ADDD
Define Target
Process
Build
Predictive App
Make Lots of
Money
D O U BT S
CO N C E R N S
O B J E C T I O N S
Decision Quality
Status Quo Predictive Prescriptive Automation
— The Economist, May 2015
“The best chess players in the
world are‘centaurs’,
amalgamated teams of humans
and algorithms.”
Decision Quality
Status Quo Predictive Prescriptive Automation 'Centaurs'
Ready, set, go for ADDD?
Not so fast
Data Availability
Predictability
Understandability
Executability
Financial Impact of Predictive
Apps
-120
-90
-60
-30
0
30
60
90
March April May June July August September
Break-Even
after 3 months
Lars Trieloff
@trieloff

More Related Content

What's hot

Brighttalk reason 114 for learning math - final
Brighttalk   reason 114 for learning math - finalBrighttalk   reason 114 for learning math - final
Brighttalk reason 114 for learning math - final
Andrew White
 
Brighttalk outage insurance- what you need to know - final
Brighttalk   outage insurance- what you need to know - finalBrighttalk   outage insurance- what you need to know - final
Brighttalk outage insurance- what you need to know - final
Andrew White
 
Bright talk running a cloud - final
Bright talk   running a cloud - finalBright talk   running a cloud - final
Bright talk running a cloud - final
Andrew White
 
Stop Getting Crushed By Business Pressure
Stop Getting Crushed By Business PressureStop Getting Crushed By Business Pressure
Stop Getting Crushed By Business Pressure
Arty Starr
 
2009 Ala Region 5 OM01 Technology Management Strategies: Become a Lean, Mean ...
2009 Ala Region 5 OM01 Technology Management Strategies: Become a Lean, Mean ...2009 Ala Region 5 OM01 Technology Management Strategies: Become a Lean, Mean ...
2009 Ala Region 5 OM01 Technology Management Strategies: Become a Lean, Mean ...
Nancy Duhon
 

What's hot (20)

Data Natives 2015: Predictive Applications are Going to Steal Your Job: this ...
Data Natives 2015: Predictive Applications are Going to Steal Your Job: this ...Data Natives 2015: Predictive Applications are Going to Steal Your Job: this ...
Data Natives 2015: Predictive Applications are Going to Steal Your Job: this ...
 
Big Data Berlin – Automating Decisions is the Next Frontier for Big Data
Big Data Berlin – Automating Decisions is the Next Frontier for Big DataBig Data Berlin – Automating Decisions is the Next Frontier for Big Data
Big Data Berlin – Automating Decisions is the Next Frontier for Big Data
 
Brighttalk reason 114 for learning math - final
Brighttalk   reason 114 for learning math - finalBrighttalk   reason 114 for learning math - final
Brighttalk reason 114 for learning math - final
 
Brighttalk outage insurance- what you need to know - final
Brighttalk   outage insurance- what you need to know - finalBrighttalk   outage insurance- what you need to know - final
Brighttalk outage insurance- what you need to know - final
 
Bright talk running a cloud - final
Bright talk   running a cloud - finalBright talk   running a cloud - final
Bright talk running a cloud - final
 
GrrCON 2018: Stop boiling the ocean!
GrrCON 2018: Stop boiling the ocean!GrrCON 2018: Stop boiling the ocean!
GrrCON 2018: Stop boiling the ocean!
 
Brighttalk learning to cook- network management recipes - final
Brighttalk   learning to cook- network management recipes - finalBrighttalk   learning to cook- network management recipes - final
Brighttalk learning to cook- network management recipes - final
 
David Turnbull - Hotel data - In the kingdom of the blind, the one eyed man i...
David Turnbull - Hotel data - In the kingdom of the blind, the one eyed man i...David Turnbull - Hotel data - In the kingdom of the blind, the one eyed man i...
David Turnbull - Hotel data - In the kingdom of the blind, the one eyed man i...
 
Wandering Through Software
Wandering Through SoftwareWandering Through Software
Wandering Through Software
 
Want to change people's behaviour just ask (and nudge, hug, shove and smack) ...
Want to change people's behaviour just ask (and nudge, hug, shove and smack) ...Want to change people's behaviour just ask (and nudge, hug, shove and smack) ...
Want to change people's behaviour just ask (and nudge, hug, shove and smack) ...
 
How to use data to make a hit tv show
How to use data to make a hit tv showHow to use data to make a hit tv show
How to use data to make a hit tv show
 
Nudge theory
Nudge theory Nudge theory
Nudge theory
 
Stop Getting Crushed By Business Pressure
Stop Getting Crushed By Business PressureStop Getting Crushed By Business Pressure
Stop Getting Crushed By Business Pressure
 
The kickstarter to measuring what matters Evanta CISO 2017
The kickstarter to measuring what matters   Evanta CISO 2017The kickstarter to measuring what matters   Evanta CISO 2017
The kickstarter to measuring what matters Evanta CISO 2017
 
Creating a Results Oriented Culture
Creating a Results Oriented CultureCreating a Results Oriented Culture
Creating a Results Oriented Culture
 
Creating a results oriented culture
Creating a results oriented cultureCreating a results oriented culture
Creating a results oriented culture
 
Moving Mountains Through Measurement
Moving Mountains Through MeasurementMoving Mountains Through Measurement
Moving Mountains Through Measurement
 
2009 Ala Region 5 OM01 Technology Management Strategies: Become a Lean, Mean ...
2009 Ala Region 5 OM01 Technology Management Strategies: Become a Lean, Mean ...2009 Ala Region 5 OM01 Technology Management Strategies: Become a Lean, Mean ...
2009 Ala Region 5 OM01 Technology Management Strategies: Become a Lean, Mean ...
 
Top 5 Reasons Why Improvement Efforts Fail
Top 5 Reasons Why Improvement Efforts FailTop 5 Reasons Why Improvement Efforts Fail
Top 5 Reasons Why Improvement Efforts Fail
 
Accelerated finance and tech leadership
Accelerated finance and tech leadershipAccelerated finance and tech leadership
Accelerated finance and tech leadership
 

Similar to How to get value out of data

Customer Intelligence & Analytics - Part I
Customer Intelligence & Analytics - Part ICustomer Intelligence & Analytics - Part I
Customer Intelligence & Analytics - Part I
Vivastream
 

Similar to How to get value out of data (20)

Automated decision making with predictive applications – Big Data Amsterdam
Automated decision making with predictive applications – Big Data AmsterdamAutomated decision making with predictive applications – Big Data Amsterdam
Automated decision making with predictive applications – Big Data Amsterdam
 
Automated decision making with big data – Big Data Vienna
Automated decision making with big data – Big Data ViennaAutomated decision making with big data – Big Data Vienna
Automated decision making with big data – Big Data Vienna
 
Nick brown - Coaching in a Data Driven World | Agile Delivery 2017
Nick brown - Coaching in a Data Driven World | Agile Delivery 2017Nick brown - Coaching in a Data Driven World | Agile Delivery 2017
Nick brown - Coaching in a Data Driven World | Agile Delivery 2017
 
Data capabilities and competitive advantage
Data capabilities and competitive advantageData capabilities and competitive advantage
Data capabilities and competitive advantage
 
Monitoring Complex Systems - Chicago Erlang, 2014
Monitoring Complex Systems - Chicago Erlang, 2014Monitoring Complex Systems - Chicago Erlang, 2014
Monitoring Complex Systems - Chicago Erlang, 2014
 
Barga Galvanize Sept 2015
Barga Galvanize Sept 2015Barga Galvanize Sept 2015
Barga Galvanize Sept 2015
 
Predictive Analytics
Predictive Analytics Predictive Analytics
Predictive Analytics
 
Machine learning101 v1.2
Machine learning101 v1.2Machine learning101 v1.2
Machine learning101 v1.2
 
The Most Important Thing: How Mozilla Does Security and What You Can Steal
The Most Important Thing: How Mozilla Does Security and What You Can StealThe Most Important Thing: How Mozilla Does Security and What You Can Steal
The Most Important Thing: How Mozilla Does Security and What You Can Steal
 
Measure All the Things! - Austin Data Day 2014
Measure All the Things! - Austin Data Day 2014Measure All the Things! - Austin Data Day 2014
Measure All the Things! - Austin Data Day 2014
 
Intel Faster Risk Oct08 - Andrew Parry
Intel Faster Risk Oct08 - Andrew ParryIntel Faster Risk Oct08 - Andrew Parry
Intel Faster Risk Oct08 - Andrew Parry
 
Pitfalls and pro-tips for effective and transparent Business Intelligence too...
Pitfalls and pro-tips for effective and transparent Business Intelligence too...Pitfalls and pro-tips for effective and transparent Business Intelligence too...
Pitfalls and pro-tips for effective and transparent Business Intelligence too...
 
Using data analytics midlands think tank 2013
Using data analytics midlands think tank 2013Using data analytics midlands think tank 2013
Using data analytics midlands think tank 2013
 
Inside Out and Upside Down - FOO Camp 2016 - Peter Coffee
Inside Out and Upside Down - FOO Camp 2016 - Peter CoffeeInside Out and Upside Down - FOO Camp 2016 - Peter Coffee
Inside Out and Upside Down - FOO Camp 2016 - Peter Coffee
 
Ml in a Day Workshop 5/1
Ml in a Day Workshop 5/1Ml in a Day Workshop 5/1
Ml in a Day Workshop 5/1
 
What are the odds of making that number risk analysis with crystal ball - O...
What are the odds of making that number   risk analysis with crystal ball - O...What are the odds of making that number   risk analysis with crystal ball - O...
What are the odds of making that number risk analysis with crystal ball - O...
 
Microsoft Dynamics: The Truth About Analytics
Microsoft Dynamics: The Truth About AnalyticsMicrosoft Dynamics: The Truth About Analytics
Microsoft Dynamics: The Truth About Analytics
 
Customer Intelligence & Analytics - Part I
Customer Intelligence & Analytics - Part ICustomer Intelligence & Analytics - Part I
Customer Intelligence & Analytics - Part I
 
Big Data Munich – Decision Automation and Big Data
Big Data Munich – Decision Automation and Big DataBig Data Munich – Decision Automation and Big Data
Big Data Munich – Decision Automation and Big Data
 
Analytics in business
Analytics in businessAnalytics in business
Analytics in business
 

More from Lars Trieloff

Mastering the customer engagement ecosystem with CQ5
Mastering the customer engagement ecosystem with CQ5Mastering the customer engagement ecosystem with CQ5
Mastering the customer engagement ecosystem with CQ5
Lars Trieloff
 

More from Lars Trieloff (17)

Putting the F in FaaS: Functional Compositional Patterns in a Serverless World
Putting the F in FaaS: Functional Compositional Patterns in a Serverless WorldPutting the F in FaaS: Functional Compositional Patterns in a Serverless World
Putting the F in FaaS: Functional Compositional Patterns in a Serverless World
 
Serverless adventures with AWS Lambda and Clojure
Serverless adventures with AWS Lambda and ClojureServerless adventures with AWS Lambda and Clojure
Serverless adventures with AWS Lambda and Clojure
 
10 Things I Learned About Pricing – Product Camp Berlin 2014
10 Things I Learned About Pricing – Product Camp Berlin 201410 Things I Learned About Pricing – Product Camp Berlin 2014
10 Things I Learned About Pricing – Product Camp Berlin 2014
 
The DNA of Marketing
The DNA of MarketingThe DNA of Marketing
The DNA of Marketing
 
Cross community campaigns with CQ5
Cross community campaigns with CQ5Cross community campaigns with CQ5
Cross community campaigns with CQ5
 
Mastering the customer engagement ecosystem with CQ5
Mastering the customer engagement ecosystem with CQ5Mastering the customer engagement ecosystem with CQ5
Mastering the customer engagement ecosystem with CQ5
 
Web Forms 2.0
Web Forms 2.0Web Forms 2.0
Web Forms 2.0
 
Advanced Collaboration And Beyond
Advanced Collaboration And BeyondAdvanced Collaboration And Beyond
Advanced Collaboration And Beyond
 
Getting Into The Flow With CQ DAM
Getting Into The Flow With CQ DAMGetting Into The Flow With CQ DAM
Getting Into The Flow With CQ DAM
 
The Zero Bullshit Architecture
The Zero Bullshit ArchitectureThe Zero Bullshit Architecture
The Zero Bullshit Architecture
 
Creating Value In Social Networking
Creating Value In Social NetworkingCreating Value In Social Networking
Creating Value In Social Networking
 
5 Ways To Build Asset Centric Applications
5 Ways To Build Asset Centric Applications5 Ways To Build Asset Centric Applications
5 Ways To Build Asset Centric Applications
 
REST and AJAX Reconciled
REST and AJAX ReconciledREST and AJAX Reconciled
REST and AJAX Reconciled
 
µjax in 30 minutes (for Stockholm)
µjax in 30 minutes (for Stockholm)µjax in 30 minutes (for Stockholm)
µjax in 30 minutes (for Stockholm)
 
µjax in 30 minutes
µjax in 30 minutesµjax in 30 minutes
µjax in 30 minutes
 
Living in a multiligual world: Internationalization for Web 2.0 Applications
Living in a multiligual world: Internationalization for Web 2.0 ApplicationsLiving in a multiligual world: Internationalization for Web 2.0 Applications
Living in a multiligual world: Internationalization for Web 2.0 Applications
 
Mindquarry For Cocoon Users
Mindquarry For Cocoon UsersMindquarry For Cocoon Users
Mindquarry For Cocoon Users
 

How to get value out of data