Analytics Teams: 5 Things You Need to Know Before You Deploy Your Model

@jamet123 #decisionmgt © 2016 Decision Management Solutions
James Taylor
CEO
Analytics Teams: 5 Things You Need to
Know Before You Deploy Your Model

@jamet123 #decisionmgt © 2016 Decision Management Solutions 2
James Taylor
Decision Management Solutions
▶ We work with clients to
improve their business by
applying business rules &
analytic technology to
automate & improve decisions.
▶ Vendor-neutral
▶ Original DMN submitter
▶ Using decision modeling since 2011
▶ I have spent the last 14 years
focused on the development of
analytic applications using
Decision Management

Agenda
▶ Challenges Deploying Analytics
▶ Five Questions To Ensure Deployment
▶ Decision Modeling for Analytics
▶ Decision Models Answer The Five Questions
▶ Wrap Up

Challenges Deploying Analytics

Data Science
Insights
Business
Value
There Is A Dangerous Gap In Data Science

Closing The Gap Delivers Analytic Success
Data Science
Insights
Business
Value
Frame The Problem Build The Model
Plan For
Deployment

Challenges Deploying Analytics
Analytic results are
prioritized, not
business resultsIneffective
deployment leaves
analytics on the
shelf
Timeliness and
sustainability are
not considered

Operational
Systems
Analytic
Systems
Analytics
need to drive
operations to
add value
40%-60% of analytic
models are not
deployed and used
Models On The Shelf Add No Value

Analytics Must be Deployed
?
?
“A presentation ABOUT a model, however
interesting, can’t power the website”
“A 40 page report,
full of math”

Business Results
▶ Too many projects focus on analytic accuracy
▶ Need to focus on business value
▶ Teams must connect their analytic to this value

Timeliness and Sustainability
▶ Time to deploy matters ▶ Deployed models age

Five Questions To Ensure Deployment
1. Does the model evaluate well against business
objectives?
2. Who’s using the model, to decide what, where and
how?
3. What will it take to make the model actionable?
4. What’s the plan for capturing and analyzing data
about model effectiveness?
5. When do we need to revisit the model to make sure
it is still effective?

Decision Modeling for Analytics

Decision Modeling Frames Analytics
▶ Provides “structure” - who, what, how, when
▶ Standardizes approach to decision making
▶ Provides “transparency” of decision approach
▶ Fosters innovation in the business model
▶ Improves/changes the business model
▶ Promotes “buy in”

Decision Models Frame The Problem
Decision
Input Data
Knowledge
Sources
Sub-
decisions
Analytic
Source: DecisionsFirst Modeler
@jamet123 #decisionmgt

Decision Models Put Analytics In Context
Process
Decision
Motivation Organization
Department
Team
Role
Role
Team
Department
Organization
Analytic

Decision Models Answer
The Five Questions

Five Questions To Ensure Deployment
1. Does the model evaluate well against business
objectives?
2. Who’s using the model, to decide what, where and
how?
3. What will it take to make the model actionable?
4. What’s the plan for capturing and analyzing data
about model effectiveness?
5. When do we need to revisit the model to make sure
it is still effective?

1. Does the model evaluate well against
business objectives?
▶ Objectives of the
sponsors
▶ Measures of success
▶ For instance
▶ Reduce claims in litigation
▶ Improve the loss ratio
▶ Objectives of the
modelers
▶ Measures of accuracy
▶ For instance
▶ Error rate
▶ Lift
Business Measures Not Analytic Ones

Consider Impact On All Business Objectives
Loan Repayment
Insurance Revenue
Customer Satisfaction

2. Who’s using the model, to decide what,
where and how?
▶ What is the analytic
used for?
▶ Who has to use the
analytic model?
▶ Where is the analytic
used?
▶ How is the analytic
used?
▶ The analytic is used in
an explicit decision
▶ Those that make the
decision
▶ The decision has a
business process context
▶ The other elements of
the decision are clear

3. What will it take to make the model
actionable?
▶ Decisions often involve analytics and business rules
Business
Rules
Analytics

Take Action With Business Rules
▶ Adding business rules to analytic models turns
predictions into actions
▶ Managing sets of business rules and analytic models
in a single decision flow, a single execution,
encapsulate decision-making
▶ Business user find business rules approachable,
increasing their comfort with analytic decision-
making
▶ Business rules give them some control over what to
DO about the predictions you give them

4. What’s the plan for capturing and
analyzing data about model effectiveness?
Decision
Made
Supporting
Decisions
Scores
Business
Results
▶ Decision Models show what to store

5. When do we need to revisit the model to
make sure it is still effective?
Data Changes
Direct
Outcomes
Indirect
Outcomes
Business
Results
▶ Decisions models reveal tracking needs

Champion Challengers
Champion/Challenger in Decision Models

Five Questions To Manage Deployment
1. Does the model evaluate well
against business objectives?
2. Who’s using the model, to
decide what, where and how?
3. What will it take to make the
model actionable?
4. What’s the plan for capturing
and analyzing data about model
effectiveness?
5. When do we need to revisit the
model to make sure it is still
effective?
▶ Decision models link analytics to
decisions to business metrics
▶ Decision models show who owns,
makes and cares about decisions
▶ Decision models wrap analytics in
rules-based decisions
▶ Decision outcomes can be
captured with input data, business
results and scores
▶ Decision models clarify business
context, pace of change and show
data usage clearly

Getting Started

Decision
Most Analytic Projects Leave Decisions
In The Dark
Modeling Decisions
Brings Transparency
29©2015 Decision Management Solutions

Data
Business
Understanding
Data
Understanding
Modeling
Data
Preparation
Evaluation
Deployment
CRISP-DM Phases
1. Business Understanding
2. Data Understanding
3. Data Preparation
4. Modeling
5. Evaluation
6. Deployment

Specific Analytic
Decision
Analytic Models
Overall decision to be
improved
Business Understanding As a Decision Model
Regulations and
Policies
Data

Decision Models Ensure Analytic
Deployment
Tie the analytic to
decisions and decisions
to business results
Specify the
decision(s) that use
the analytic
Use the decision
context to
frame
deployment and
sustainability
needs

Decision Modeling For Analytic Projects
▶ Steps
1. Training in decision modeling
2. Establish business value
3. Model effective decision-making
4. Add process, organization, business context
5. Frame and scope the analytic project
▶ Deliverables
▶ Decision Model showing impact of proposed analytic
▶ Analytic, business and IT teams trained & self-sufficient
▶ Best practices and go-forward recommendations
▶ Software, consulting and training

Decision Modeling for Analytic Projects
▶ Learn more
▶ Download the five
questions brief
▶ bit.ly/5QAnalytics
▶ Ready to start?
▶ Kick off a decision
modeling project today
▶ bit.ly/DMNAnalytics

Questions?

Decision Modeling for Analytic Projects
▶ Learn more
▶ Download the five
questions brief
▶ bit.ly/5QAnalytics
▶ Ready to start?
▶ Kick off a decision
modeling project today
▶ bit.ly/DMNAnalytics

DecisionsFirst Modeler
Collaborative, cloud-based decision modeling software
that puts decisions at the heart of your business
architecture, building better requirements and driving
successful business rules implementations
Key Features
Multi-user, Collaborative
Decision Model and Notation
(DMN) Support
Shared Database, Reusable
components
Automated Consistency and
Completeness Checks
Living Repository

Thank You
@jamet123 #decisionmgt © 2016 Decision Management Solutions
For more on
Decision Management, go to:
decisionmanagementsolutions.com
James Taylor
james@decisionmanagementsolutions.com

Analytics Teams: 5 Things You Need to Know Before You Deploy Your Model

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (9)

Similar to Analytics Teams: 5 Things You Need to Know Before You Deploy Your Model

Similar to Analytics Teams: 5 Things You Need to Know Before You Deploy Your Model (20)

Recently uploaded

Recently uploaded (20)

Analytics Teams: 5 Things You Need to Know Before You Deploy Your Model