Backward Engineering: Plan Machine Learning Deployment in Reverse

@jamet123 #decisionmgt © 2019 Decision Management Solutions
James Taylor
CEOBackwards Engineering:
Plan Machine Learning Deployment in Reverse

@jamet123 #decisionmgt © 2019 Decision Management Solutions 2
Typical Approach To Predictive Analytics and ML
Assemble
Data
Do
Analysis
Score
Data
Improve
Results
?

This Approach Isn’t Working Very Well
▪ Studies show that many organizations are not getting great results this way.
▪ 70% of organizations say analytics is really important but only 2% have
delivered on the promise.
▪ There are far more companies failing to deliver high impact (80%) than
succeeding in generating impact.
Broken links: Why analytics investments have yet to pay out” ZS and the Economist Information Unit, Ltd. June 2019
Raising returns on analytics investments in insurance” by McKinsey
“Analytics
efforts is
very/extremely
important”
70% of organizations say
…but only 2% have delivered
on the promise.
80% of organizations are
failing to have a broad impact
with their analytic efforts.

Because the Last Mile is Hard
Broken links: Why analytics investments have yet to pay out”
ZS and the Economist Information Unit, Ltd. June 2019
Problem definition/framing
Solution approach/design
Data integration/preparation
Scoping/triage/supplier selection
Analysis execution
Interpretation and synthesis
Presentation/communication
Action/change management
43%
47%
39%
21%
18%
15%
8%
47%

Because the Last Mile is Hard
Source: McKinsey - “Raising returns on analytics investments in insurance”
Failure Mode
Limited adoption
of integration
Lack of strategic
alignment and
direction
Poor data quality
Other
Description
Inability to integrate analytics
solutions into work flows
Limited frontline adoption
Lack of stakeholder alignment
or support
Lack of clear road map
Missing or incomplete data
Data quality or accuracy issues
Data fragmentation
Missing team skills or capabilities
Unclear case scope
Inability to articulate value
19
17
26
38

“Most important, breakaway companies target much
of [their] spending toward the biggest challenge
companies face in extracting value from analytics—
the last mile...”
Leaders Understand This
“Most important, breakaway companies target much of this spending toward the biggest challenge companies face in
extracting value from analytics—the last mile, or embedding analytics into the core of all workflows and decision-
making processes (more on this later). Nearly 90 percent of breakaway organizations devote more than half of their
analytics budgets to this effort, versus only 23 percent of all other organizations that do so.”
Breaking away: The secrets to scaling analytics, May 2019 By Peter Bisson, Bryce Hall, Brian McCarthy, and Khaled Rifai

Why Is The Last Mile Important?
Why Is The Last Mile Hard?
Succeed By Putting The Last Mile, First

Why Is The Last Mile Important?

Because Analytic Value Requires Business Value
valuable analytic
Adjective-Noun Pair: an analytic model of any
type (regression, classification, ensemble,
machine learning, neural network etc.) that has
caused the organization that paid for its
development to change its behavior in a way
that adds business value to that organization.

Because Predictive Analytics Don’t DO Anything
Acting on the prediction doesHaving a prediction doesn’t change behavior
Simply having a prediction is not enough to change behavior and so improve results. Organizations must act on those predictions, they must
change the way they behave because of the prediction, if that prediction is to have value.

Because Delivering Value Means Automated Decisions
 Partially or fully automated
 Digital Decisioning
Whether the decision-making in those systems is
completely automated or just partially automated
and reliant on an external human to participate
does not matter. Unless the decision-making in your
systems is changed by your analytics, delivering
value will prove impossible.

Why Is The Last Mile Hard?

Because IT Approaches Are Not Analytic Approaches
Data Structure
Deterministic
Requirements
Testing
Data History
Probabilistic
Possibilities
Simulation
They worry about the structure of data, not history and analysis. They care about the fields and allowed values but not about the history or distribution of those values.
They focus on deterministic results not probabilistic ones. They assume that a solution will continue to be correct, not thinking that new data may undermine assumptions.
They focus on specific, known requirements not the possibilities – the if only – of prediction and insight.
And they focus on testing what is built to see if it meets requirements rather than simulating the business impact of the possibilities.

Because You Need A Three Legged Stool
Success
IT
Business
Operations
Analytics
Success in the last mile involves
coordinating three groups – IT,
Analytics and Business Operations. IT
methodologies don’t work for
analytics, analytics teams don’t like
thinking about systems and
integration, and the business does
not really understand either of them.

Because The Value Delivery Landscape Is complex
 Many Tools
 Many Platforms
 Multiple UIs
 Complex Processes
There’s a tremendous amount of complexity in the
delivery landscape. This allows both the IT and
analytics teams to geek out over their own tools
and platforms. Plus there are often many user
interfaces impacting those involved and complex
processes that will need to change.

Succeed By Putting the Last Mile First

1. Put Decisions First For Business Value
What (business) measures?
What (business) decisions have an impact?
Which decision should we improve?
What does improve mean?
Customer satisfaction
Pricing, claims handling, renewal
Claims handling
Increase STP rate without more fraud or waste
Make the project about the Decision, not the Analytic

“Breakaway companies …
have identified and
prioritized the …
decision-making
processes in which to
embed analytics.”
Because Leaders Prioritize Decision-Making
“Breakaway companies are almost
twice as likely to have identified
and prioritized the top ten to 15
decision-making processes in which
to embed analytics.”
Breaking away: The secrets to
scaling analytics, May 2019 By Peter
Bisson, Bryce Hall, Brian McCarthy,
and Khaled Rifai
1.75X more likely to
prioritize top decision-
making processes
31
55
% of respondents who strongly
agree that they have clearly
prioritized the top 10-15 key
decision-making processes in
which to embed analytics
insights

External
Data
Big Data
2. Deliver Analytics To The Last Mile With Decision Services
Analytics,
ML and AI
Business
Rules
• Business Rules are quick
to change
• Good for regulations,
policies, flash updates
• Less insight-rich than
analytics
• Analytics are insight-rich
but often opaque,
especially ML and AI
• Good for patterns, trends,
categorization
• Must be fed new data and
continuously improved
A decision service
encapsulates analytics, ML
and AI to deliver automated
decisions across the last mile
Data about business outcomes and
decisions made is integrated with
external data to close the loop and
improve both rules and analytics

Because Decisions Are Easier To Integrate Than Scores
Integrate Decisions, not Scores
Consistent integration
Can explain the use of the score
Focus on improving decision-making
Analytics, business, IT collaborate
Every environment different
Can only explain the score itself
Focus on improving the math
IT on point

3. Put Analytics in a Decision Context
Decisions involve more than just your analytic
score. They need to apply regulations, best
practices, policies and business domain knowledge
also. All these elements need to be mixed and
matched to create a decisioning solution.
Decision Modeling shows all the elements of a
decision and enables you to mix and match the
right technologies to automate enough of the
decision to take advantage of your predictive
analytic models and machine learning.
Mix and match Decision
Technology

Because Business Rules Make Analytics Actionable
▪ While analytics can improve decision-making, they often must
be combined with business rules that define thresholds or
constraints.
▪ Decision models explicitly document what the different
elements of a decision are and how they interact.
▪ This allows analytics, business rules, ML and AI to be
combined to make a better business decision.
Business
Rules
Descriptive
Analytic
Machine
Learning Model

New Approach To Succeed With Machine Learning:
Put Decisions First
Results
Improve
Decisions
Build
Analytics
Find
Data
Focusing on operational decisions first ensures a strong business context for predictive
analytics and machine learning projects. Starting with the business goals of the
project, the decisions that have an impact on those goals can be identified.
Understanding these decisions will reveal which analytics are required and then the
data needed to build these analytics can be found, organized, cleaned and prepared.

“Most companies start their analytics journey with
data. Almost by definition, that approach will limit
analytics’ impact.”
“To achieve analytics at scale, companies should …
start by identifying the decision-making … they
could improve to generate additional value”
Be A Leader
“Most companies start their analytics journey with data; they determine what they have and figure out where it can be
applied. Almost by definition, that approach will limit analytics’ impact. To achieve analytics at scale, companies
should work in the opposite direction. They should start by identifying the decision-making processes they could
improve to generate additional value in the context of the company’s business strategy and then work backward to
determine what type of data insights are required to influence these decisions and how the company can supply them.”
Breaking away: The secrets to scaling analytics, May 2019 By Peter Bisson, Bryce Hall, Brian McCarthy, and Khaled Rifai

Make the project about the Decision, not the Analytic
Integrate Decisions, not Scores
Mix and match Decision technology
Make sure it will be actionable,
before you build it

Thank You
For more on
Decision Management, go to:
decisionmanagementsolutions.com
@jamet123 #decisionmgt © 2019 Decision Management Solutions
If you have further questions or comments:
james@decisionmanagementsolutions.com
+1 650 400 3029

Backward Engineering: Plan Machine Learning Deployment in Reverse

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Backward Engineering: Plan Machine Learning Deployment in Reverse

Similar to Backward Engineering: Plan Machine Learning Deployment in Reverse (20)

More from Decision Management Solutions

More from Decision Management Solutions (12)

Recently uploaded

Recently uploaded (20)

Backward Engineering: Plan Machine Learning Deployment in Reverse