SlideShare a Scribd company logo
1 of 10
UPLIFT MODELING USING
CONCURRENT DECISION TREES
PRESENTED BY: And Ozbay
June 22, 2015
INTRODUCTION
•Which customers should I target with my <treatment> in order to improve
my <metric>? (traffic / sales / revenue)
•Traditional wisdom suggests: Find people who have a high propensity to
respond and target them. Is this a sensible approach?
•Lift is measured by the difference in the average <metric> between
two groups
•Uplift Modeling is a predictive modeling technique that directly models
the incremental impact of a treatment
UPLIFT MODELING 2
AB TESTING
Test Control
UPLIFT MODELING 3
AB TESTING
USING SEGMENTS
Test Control
Red
Segment
Blue
Segment
Green
Segment
UPLIFT MODELING 4
AB TESTING
USING UPLIFT M.
Test Control
Micro
Segments
UPLIFT MODELING 5
MODELING MECHANICS – DECISION TREES
UPLIFT MODELING 6
1 1
0
1
1
11
1
0
1
1
0
0
0
0
0
0
0
TEST
POPULATION
MODELING MECHANICS – UPLIFT TREES
UPLIFT MODELING 7
1 1
0
1
1
11
1
0
1
1
0
0
0
0
0
0
0
1
0
1
1
1
1
0
1
1
0 0
0
0
0
0
0
0
0
TEST
POPULATION
CONTROL
POPULATION
MODELING MECHANICS – UPLIFT TREES
•Check to see where the greatest difference between the test and control
groups exist, testing all possible cuts
•Once a cut is established, keep identifying new cuts for the newly
identified sub-groups
•Run a t-test for every cut to see if the difference is statistically
significant using a Bonferroni Correction
•Stopping condition is finding no further cuts that are statistically significant
UPLIFT MODELING 8
QUESTIONS?
UPLIFT MODELING 9
DIFFERENCES BETWEEN SEGMENTATION MODELS
AND UPLIFT MODELS
UPLIFT MODELING 10
Uplift Segmentation
Specific for a stated objective – e.g.
interest in handbags, sports shoes, etc.
More generic, activity oriented – e.g.
unawares, “one and done”s, etc.
Identifies people who will purchase
conditional upon an action (e.g.
sending an email)
Identifies people who are alike, does
not recommend any course of action
Self correcting – changes happen both
when the user does something and
when marketer does something
Static – changes happen when the
user does something
Suitable for trade off decisions (e.g. get
more total iGMB as long as iGMB / e-
mail is greater than $0.14)
Trade off decisions harder to
implement

More Related Content

What's hot

Causality without headaches
Causality without headachesCausality without headaches
Causality without headachesBenoît Rostykus
 
Maximizing a churn campaign’s profitability with cost sensitive predictive an...
Maximizing a churn campaign’s profitability with cost sensitive predictive an...Maximizing a churn campaign’s profitability with cost sensitive predictive an...
Maximizing a churn campaign’s profitability with cost sensitive predictive an...Alejandro Correa Bahnsen, PhD
 
Counterfactual Learning for Recommendation
Counterfactual Learning for RecommendationCounterfactual Learning for Recommendation
Counterfactual Learning for RecommendationOlivier Jeunen
 
Causal reasoning and Learning Systems
Causal reasoning and Learning SystemsCausal reasoning and Learning Systems
Causal reasoning and Learning SystemsTrieu Nguyen
 
Data Analysis: Evaluation Metrics for Supervised Learning Models of Machine L...
Data Analysis: Evaluation Metrics for Supervised Learning Models of Machine L...Data Analysis: Evaluation Metrics for Supervised Learning Models of Machine L...
Data Analysis: Evaluation Metrics for Supervised Learning Models of Machine L...Md. Main Uddin Rony
 
Basic Analytics Module for Sponsors
Basic Analytics Module for SponsorsBasic Analytics Module for Sponsors
Basic Analytics Module for SponsorsDee Daley
 
Efficient Similarity Computation for Collaborative Filtering in Dynamic Envir...
Efficient Similarity Computation for Collaborative Filtering in Dynamic Envir...Efficient Similarity Computation for Collaborative Filtering in Dynamic Envir...
Efficient Similarity Computation for Collaborative Filtering in Dynamic Envir...Olivier Jeunen
 
Six sigma
Six sigmaSix sigma
Six sigmakmsonam
 
Research Paper C - 1530 - 13-05-15
Research Paper C - 1530 - 13-05-15Research Paper C - 1530 - 13-05-15
Research Paper C - 1530 - 13-05-15Andrew Dash
 
L2. Evaluating Machine Learning Algorithms I
L2. Evaluating Machine Learning Algorithms IL2. Evaluating Machine Learning Algorithms I
L2. Evaluating Machine Learning Algorithms IMachine Learning Valencia
 
Module 5: Decision Trees
Module 5: Decision TreesModule 5: Decision Trees
Module 5: Decision TreesSara Hooker
 
Hypothesis Testing: Proportions (Compare 2+ Factors)
Hypothesis Testing: Proportions (Compare 2+ Factors)Hypothesis Testing: Proportions (Compare 2+ Factors)
Hypothesis Testing: Proportions (Compare 2+ Factors)Matt Hansen
 
An Introduction to boosting
An Introduction to boostingAn Introduction to boosting
An Introduction to boostingbutest
 
Hypothesis Testing: Relationships (Compare 2+ Factors)
Hypothesis Testing: Relationships (Compare 2+ Factors)Hypothesis Testing: Relationships (Compare 2+ Factors)
Hypothesis Testing: Relationships (Compare 2+ Factors)Matt Hansen
 
Hypothesis Testing: Spread (Compare 2+ Factors)
Hypothesis Testing: Spread (Compare 2+ Factors)Hypothesis Testing: Spread (Compare 2+ Factors)
Hypothesis Testing: Spread (Compare 2+ Factors)Matt Hansen
 
Hypothesis Testing: Spread (Compare 1:Standard)
Hypothesis Testing: Spread (Compare 1:Standard)Hypothesis Testing: Spread (Compare 1:Standard)
Hypothesis Testing: Spread (Compare 1:Standard)Matt Hansen
 
Module 2: Machine Learning Deep Dive
Module 2:  Machine Learning Deep DiveModule 2:  Machine Learning Deep Dive
Module 2: Machine Learning Deep DiveSara Hooker
 
Hypothesis Testing: Proportions (Compare 1:1)
Hypothesis Testing: Proportions (Compare 1:1)Hypothesis Testing: Proportions (Compare 1:1)
Hypothesis Testing: Proportions (Compare 1:1)Matt Hansen
 
Module 1.2 data preparation
Module 1.2  data preparationModule 1.2  data preparation
Module 1.2 data preparationSara Hooker
 

What's hot (20)

Causality without headaches
Causality without headachesCausality without headaches
Causality without headaches
 
Maximizing a churn campaign’s profitability with cost sensitive predictive an...
Maximizing a churn campaign’s profitability with cost sensitive predictive an...Maximizing a churn campaign’s profitability with cost sensitive predictive an...
Maximizing a churn campaign’s profitability with cost sensitive predictive an...
 
Counterfactual Learning for Recommendation
Counterfactual Learning for RecommendationCounterfactual Learning for Recommendation
Counterfactual Learning for Recommendation
 
Causal reasoning and Learning Systems
Causal reasoning and Learning SystemsCausal reasoning and Learning Systems
Causal reasoning and Learning Systems
 
Data Analysis: Evaluation Metrics for Supervised Learning Models of Machine L...
Data Analysis: Evaluation Metrics for Supervised Learning Models of Machine L...Data Analysis: Evaluation Metrics for Supervised Learning Models of Machine L...
Data Analysis: Evaluation Metrics for Supervised Learning Models of Machine L...
 
Basic Analytics Module for Sponsors
Basic Analytics Module for SponsorsBasic Analytics Module for Sponsors
Basic Analytics Module for Sponsors
 
Efficient Similarity Computation for Collaborative Filtering in Dynamic Envir...
Efficient Similarity Computation for Collaborative Filtering in Dynamic Envir...Efficient Similarity Computation for Collaborative Filtering in Dynamic Envir...
Efficient Similarity Computation for Collaborative Filtering in Dynamic Envir...
 
Six sigma
Six sigmaSix sigma
Six sigma
 
Research Paper C - 1530 - 13-05-15
Research Paper C - 1530 - 13-05-15Research Paper C - 1530 - 13-05-15
Research Paper C - 1530 - 13-05-15
 
L2. Evaluating Machine Learning Algorithms I
L2. Evaluating Machine Learning Algorithms IL2. Evaluating Machine Learning Algorithms I
L2. Evaluating Machine Learning Algorithms I
 
Module 5: Decision Trees
Module 5: Decision TreesModule 5: Decision Trees
Module 5: Decision Trees
 
Hypothesis Testing: Proportions (Compare 2+ Factors)
Hypothesis Testing: Proportions (Compare 2+ Factors)Hypothesis Testing: Proportions (Compare 2+ Factors)
Hypothesis Testing: Proportions (Compare 2+ Factors)
 
An Introduction to boosting
An Introduction to boostingAn Introduction to boosting
An Introduction to boosting
 
Hypothesis Testing: Relationships (Compare 2+ Factors)
Hypothesis Testing: Relationships (Compare 2+ Factors)Hypothesis Testing: Relationships (Compare 2+ Factors)
Hypothesis Testing: Relationships (Compare 2+ Factors)
 
Hypothesis Testing: Spread (Compare 2+ Factors)
Hypothesis Testing: Spread (Compare 2+ Factors)Hypothesis Testing: Spread (Compare 2+ Factors)
Hypothesis Testing: Spread (Compare 2+ Factors)
 
Hypothesis Testing: Spread (Compare 1:Standard)
Hypothesis Testing: Spread (Compare 1:Standard)Hypothesis Testing: Spread (Compare 1:Standard)
Hypothesis Testing: Spread (Compare 1:Standard)
 
Dbm630 lecture08
Dbm630 lecture08Dbm630 lecture08
Dbm630 lecture08
 
Module 2: Machine Learning Deep Dive
Module 2:  Machine Learning Deep DiveModule 2:  Machine Learning Deep Dive
Module 2: Machine Learning Deep Dive
 
Hypothesis Testing: Proportions (Compare 1:1)
Hypothesis Testing: Proportions (Compare 1:1)Hypothesis Testing: Proportions (Compare 1:1)
Hypothesis Testing: Proportions (Compare 1:1)
 
Module 1.2 data preparation
Module 1.2  data preparationModule 1.2  data preparation
Module 1.2 data preparation
 

Similar to Tree Based Uplift Modeling

Measuring the Effectiveness of Customer Health Model
Measuring the Effectiveness of Customer Health ModelMeasuring the Effectiveness of Customer Health Model
Measuring the Effectiveness of Customer Health ModelTotango
 
Maneco-Report.pptx
Maneco-Report.pptxManeco-Report.pptx
Maneco-Report.pptxasdfg hjkl
 
The Always-On Approach: How to Continually Improve Your Streaming Advertising...
The Always-On Approach: How to Continually Improve Your Streaming Advertising...The Always-On Approach: How to Continually Improve Your Streaming Advertising...
The Always-On Approach: How to Continually Improve Your Streaming Advertising...Tinuiti
 
Conversion Rate Optimisation and the art of Testing
Conversion Rate Optimisation and the art of Testing Conversion Rate Optimisation and the art of Testing
Conversion Rate Optimisation and the art of Testing User Vision
 
Continuous Improvement Strategy
Continuous Improvement StrategyContinuous Improvement Strategy
Continuous Improvement StrategyCraig Marton
 
Can I Test More Than One Variable at a Time? Statisticians answer some of th...
Can I Test More Than One Variable at a  Time? Statisticians answer some of th...Can I Test More Than One Variable at a  Time? Statisticians answer some of th...
Can I Test More Than One Variable at a Time? Statisticians answer some of th...MarketingExperiments
 
Smu mba sem 4 om fall 2015 assignments
Smu mba sem 4 om fall 2015 assignmentsSmu mba sem 4 om fall 2015 assignments
Smu mba sem 4 om fall 2015 assignmentssolved_assignments
 
Digital Disruption -> The New Reality: Acronis Keynote_20.11.14
Digital Disruption -> The New Reality: Acronis Keynote_20.11.14Digital Disruption -> The New Reality: Acronis Keynote_20.11.14
Digital Disruption -> The New Reality: Acronis Keynote_20.11.14SalesChannel International
 
EMF_ Circular Business Model Design Guide.pdf
EMF_ Circular Business Model Design Guide.pdfEMF_ Circular Business Model Design Guide.pdf
EMF_ Circular Business Model Design Guide.pdfHeriHaryono3
 
Smu mba sem 4 om winter 2014 assignments
Smu mba sem 4 om winter 2014 assignmentsSmu mba sem 4 om winter 2014 assignments
Smu mba sem 4 om winter 2014 assignmentssolved_assignments
 
Developing the Dashboard
Developing the DashboardDeveloping the Dashboard
Developing the DashboardJane Chiang
 
Pharmaceutical Quality Management System
Pharmaceutical Quality Management SystemPharmaceutical Quality Management System
Pharmaceutical Quality Management SystemDhawal_Raghuvanshi
 
ELFT Quality improvement roadshow - 2014
ELFT Quality improvement roadshow - 2014ELFT Quality improvement roadshow - 2014
ELFT Quality improvement roadshow - 2014ELFT_QI
 
Predictive Analytics Demystified
Predictive Analytics DemystifiedPredictive Analytics Demystified
Predictive Analytics DemystifiedSenturus
 

Similar to Tree Based Uplift Modeling (20)

Measuring the Effectiveness of Customer Health Model
Measuring the Effectiveness of Customer Health ModelMeasuring the Effectiveness of Customer Health Model
Measuring the Effectiveness of Customer Health Model
 
Results measurement
Results measurementResults measurement
Results measurement
 
Maneco-Report.pptx
Maneco-Report.pptxManeco-Report.pptx
Maneco-Report.pptx
 
1000 track2 boire
1000 track2 boire1000 track2 boire
1000 track2 boire
 
LESSONS
LESSONS LESSONS
LESSONS
 
The Always-On Approach: How to Continually Improve Your Streaming Advertising...
The Always-On Approach: How to Continually Improve Your Streaming Advertising...The Always-On Approach: How to Continually Improve Your Streaming Advertising...
The Always-On Approach: How to Continually Improve Your Streaming Advertising...
 
Customer Segmentation
Customer SegmentationCustomer Segmentation
Customer Segmentation
 
Conversion Rate Optimisation and the art of Testing
Conversion Rate Optimisation and the art of Testing Conversion Rate Optimisation and the art of Testing
Conversion Rate Optimisation and the art of Testing
 
Continuous Improvement Strategy
Continuous Improvement StrategyContinuous Improvement Strategy
Continuous Improvement Strategy
 
Can I Test More Than One Variable at a Time? Statisticians answer some of th...
Can I Test More Than One Variable at a  Time? Statisticians answer some of th...Can I Test More Than One Variable at a  Time? Statisticians answer some of th...
Can I Test More Than One Variable at a Time? Statisticians answer some of th...
 
Smu mba sem 4 om fall 2015 assignments
Smu mba sem 4 om fall 2015 assignmentsSmu mba sem 4 om fall 2015 assignments
Smu mba sem 4 om fall 2015 assignments
 
Digital Disruption -> The New Reality: Acronis Keynote_20.11.14
Digital Disruption -> The New Reality: Acronis Keynote_20.11.14Digital Disruption -> The New Reality: Acronis Keynote_20.11.14
Digital Disruption -> The New Reality: Acronis Keynote_20.11.14
 
EMF_ Circular Business Model Design Guide.pdf
EMF_ Circular Business Model Design Guide.pdfEMF_ Circular Business Model Design Guide.pdf
EMF_ Circular Business Model Design Guide.pdf
 
Smu mba sem 4 om winter 2014 assignments
Smu mba sem 4 om winter 2014 assignmentsSmu mba sem 4 om winter 2014 assignments
Smu mba sem 4 om winter 2014 assignments
 
Knowledge Management
Knowledge ManagementKnowledge Management
Knowledge Management
 
PQMS.pptx
PQMS.pptxPQMS.pptx
PQMS.pptx
 
Developing the Dashboard
Developing the DashboardDeveloping the Dashboard
Developing the Dashboard
 
Pharmaceutical Quality Management System
Pharmaceutical Quality Management SystemPharmaceutical Quality Management System
Pharmaceutical Quality Management System
 
ELFT Quality improvement roadshow - 2014
ELFT Quality improvement roadshow - 2014ELFT Quality improvement roadshow - 2014
ELFT Quality improvement roadshow - 2014
 
Predictive Analytics Demystified
Predictive Analytics DemystifiedPredictive Analytics Demystified
Predictive Analytics Demystified
 

Tree Based Uplift Modeling

  • 1. UPLIFT MODELING USING CONCURRENT DECISION TREES PRESENTED BY: And Ozbay June 22, 2015
  • 2. INTRODUCTION •Which customers should I target with my <treatment> in order to improve my <metric>? (traffic / sales / revenue) •Traditional wisdom suggests: Find people who have a high propensity to respond and target them. Is this a sensible approach? •Lift is measured by the difference in the average <metric> between two groups •Uplift Modeling is a predictive modeling technique that directly models the incremental impact of a treatment UPLIFT MODELING 2
  • 4. AB TESTING USING SEGMENTS Test Control Red Segment Blue Segment Green Segment UPLIFT MODELING 4
  • 5. AB TESTING USING UPLIFT M. Test Control Micro Segments UPLIFT MODELING 5
  • 6. MODELING MECHANICS – DECISION TREES UPLIFT MODELING 6 1 1 0 1 1 11 1 0 1 1 0 0 0 0 0 0 0 TEST POPULATION
  • 7. MODELING MECHANICS – UPLIFT TREES UPLIFT MODELING 7 1 1 0 1 1 11 1 0 1 1 0 0 0 0 0 0 0 1 0 1 1 1 1 0 1 1 0 0 0 0 0 0 0 0 0 TEST POPULATION CONTROL POPULATION
  • 8. MODELING MECHANICS – UPLIFT TREES •Check to see where the greatest difference between the test and control groups exist, testing all possible cuts •Once a cut is established, keep identifying new cuts for the newly identified sub-groups •Run a t-test for every cut to see if the difference is statistically significant using a Bonferroni Correction •Stopping condition is finding no further cuts that are statistically significant UPLIFT MODELING 8
  • 10. DIFFERENCES BETWEEN SEGMENTATION MODELS AND UPLIFT MODELS UPLIFT MODELING 10 Uplift Segmentation Specific for a stated objective – e.g. interest in handbags, sports shoes, etc. More generic, activity oriented – e.g. unawares, “one and done”s, etc. Identifies people who will purchase conditional upon an action (e.g. sending an email) Identifies people who are alike, does not recommend any course of action Self correcting – changes happen both when the user does something and when marketer does something Static – changes happen when the user does something Suitable for trade off decisions (e.g. get more total iGMB as long as iGMB / e- mail is greater than $0.14) Trade off decisions harder to implement