The document discusses using data science to drive healthcare operations. It describes using models to close gaps in patient care by predicting which diabetic patients will develop complications in the next 6 months based on demographic data, medical history, medications and lab tests. The challenges are class imbalance, with few patients historically developing complications, and missing lab data. Gradient boosting decision trees are able to handle these issues better than logistic regression. Testing shows the model can identify high-risk patients to call with a 24% precision and 66% recall. A trial using the model to select patients for home visits found more complications than random selection, showing the approach can improve outcomes.

1. Driving Healthcare
Operations with Data
Science

2. "literally a health insurance company"

3. "Operations"
Clinical Operations
● Close member "gaps in care"
○ Not taking their meds
○ Not seeing their doctors
○ Not getting tested
● Document conditions
Insurance Operations
● Approve / deny claims
● Approve / deny authorizations
● Catch fraud

4. "Operations"
Clinical Operations
● Close member "gaps in care"
○ Not taking their meds
○ Not seeing their doctors
○ Not getting tested
● Document conditions
Insurance Operations
● Approve / deny claims
● Approve / deny authorizations
● Catch fraud

5. E.g.

6. Talk to them about consequences of not
doing so?
Knock on the doors of most
non-adherent members?
Ask members politely? Use different messages for rich and
poor members?

7. Enter

8. Data ScienceEnter

9. Data Science
What should we do?
For whom?
Did it work?
Enter

10. Case Study: Whom to Call for Home
Visits?

12. Can we predict which of our diabetic
members will have complications in the
next 6 months?

13. Time
Observation Interval Prediction Interval

14. Time
Observation Interval Prediction Interval
Demographic info, lab
tests, medications,
other diagnoses
Diagnosed with
diabetes
complications?

15. Features Labels
Member Age Hypertension hba1c
CP001 65 Yes 6.5
CP002 77 No 8.3
CP002 84 Yes 7.4
Diagnosed with
Complication in
6-month Interval
Yes
No
Yes

16. Challenge: High Class Imbalance
● Historically, only 8% of diabetic members have been diagnosed with
complications over a 6-month period.
● Easy to get "high" accuracy, but hard to get decent precision/recall tradeoff.

17. Approach: High Class Imbalance
● Evaluate using area under ROC curve.
● Empirically, tree ensemble models appear to handle the imbalance better
than logistic regression.

18. Challenge: Missing Data
● Glycated hemoglobin clearly an important feature… but we only have
measurements for ~60% of members.
● Whether we have a measurement correlates with both:
○ Diabetes complications.
○ How well a model trained without the lab measurement performs.

19. Approach: Missing Data
● Simply hardcode all missing values to something outside the measurement
range.
○ In our case, 0.0.
● This way, tree models can split on "have a measurement" vs. "don't have a
measurement".

20. Final Model: Gradient Boosting Tree Ensemble

21. Evaluation
AUROC: 0.8
Precision: 24%
Recall: 66%

22. Most Predictive Features
Glycated Hemoglobin
Age
Hypertension
Takes Insulin

24. Did it work?

25. Do we catch more complications if we
make calls using the model?

26. Control Group Treatment Group

27. Control Group Treatment Group
Call Group
(Chosen at Random)
Call Group
(Chosen by Model)

28. Found Complications Didn't Find Complications
Control Group 8 92
Treatment Group 24 76
FAKE RESULTS

29. Found Complications Didn't Find Complications
Control Group 8 92
Treatment Group 24 76
FAKE RESULTS
Chi-Squared Test