The document discusses key differentiators that lead to effective machine learning models in healthcare. It identifies 5 differentiators: 1) the vendor's exclusive healthcare expertise, 2) access to extensive healthcare data sources, 3) ease of implementing the machine learning model, 4) the model's interpretability and ability to gain clinician buy-in, and 5) conforming to privacy standards. The document emphasizes that effective machine learning requires more than just algorithms and data - the model must be accurately tailored to the healthcare setting and integrated into clinical workflows.

1. The Dangers of
Commoditized Machine
Learning in Healthcare:
5 Key Differentiators
that Lead to Success

2. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Machine Learning in Healthcare
Machine learning’s popularity in healthcare
is growing thanks to its expanding
capabilities in medical image analysis,
predictive analytics, and prescriptive
analytics for clinical decision support.
The machine-learning-as-a-service market
is expected to grow to almost $5.4 billion
by 2022, with healthcare certainly being
one of the industries driving that trend.
Even today, many technology companies
already deliver machine learning models
specific to healthcare.

3. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Machine Learning in Healthcare
But let the buyer beware. In a certain sense,
machine learning in healthcare is being
commoditized.
The combination of EMR ubiquity, increased
computational power, the open source
movement, and the rise of cloud providers
has made training machine learning models
easier than ever.
But just because a vendor develops machine
learning models and delivers them to a client
doesn’t mean it offers a complete package to
make machine learning work toward its
intended purpose.

4. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Declining Medicare Reimbursements
Several qualities define machine learning’s
effectiveness in a healthcare setting.
• Is machine learning accurate?
• Does it apply to the business or clinical need?
• Does it fit well into the workflow?
• What interventions can users make to affect the
corresponding outcome metric?
To answer these questions around properly
using machine learning, health systems and
payers should consider a few factors that
distinguish one machine learning vendor and
model from another.

5. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Five Key Machine Learning Differentiators
Effective machine learning is the product of more than data, features, and
algorithms, and is defined by five key differentiators:
1. Vendor’s expertise and exclusive focus on healthcare.
2. Machine learning model’s access to extensive data sources.
3. Machine learning model’s ease of implementation.
4. Machine learning model’s interpretability and buy-in.
5. Machine learning model’s conformance with privacy standards.

6. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Five Key Machine Learning Differentiators
#1: Vendor’s Expertise and Exclusive Focus on Healthcare
A machine learning vendor that’s exclusively
focused on healthcare commits its expertise,
products, and services to help health
systems and payers improve outcomes.
This focus implies a dedication to
healthcare-specific analytics and decision
support technology.
Many machine learning vendors are only
partially focused on healthcare, also
spreading their resources to other industries.

7. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Five Key Machine Learning Differentiators
#1: Vendor’s Expertise and Exclusive Focus on Healthcare
Vendor expertise means knowing the pain
points of health systems and payers and the
workflows where machine learning can
most effectively be leveraged.
Many companies focusing on machine
learning technology don’t have a healthcare
background or subject matter expertise.
They lack the staff with experience in
building an accurate risk model appropriate
for a specific use case, much less how to
apply the risk scores that machine
learning generates.

8. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Five Key Machine Learning Differentiators
#1: Vendor’s Expertise and Exclusive Focus on Healthcare
It takes experience to understand clinical
workflows and their inefficiencies, and
guide clinicians in making connections
between risk scores and actionable
decisions that risk scores produce.
A vendor with process improvement
experience across a broad base of health
system and payer partners knows what it
takes to change workflows in response to
machine-learning generated insights.

9. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Five Key Machine Learning Differentiators
#2: Machine Learning Model’s Access to Extensive Data Sources
A machine learning model shouldn’t be limited to
a single data source, like an EMR.
The model needs access to multiple data
sources through an analytics platform that can
aggregate data from claims, labs, pharmacy,
radiology, HIEs, billing, patient satisfaction
surveys, multiple EMRs, and more.
More data means more accurate models, so
clinicians can focus interventions on patients
who need them most, ensuring that no patient is
accidentally overlooked or unnecessarily treated.

10. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Five Key Machine Learning Differentiators
#3: Machine Learning Model’s Ease of Implementation
EMRs have generated significant clinical and IT
staff fatigue around implementing, learning, and
using technology. The prospect of adding another
complex technology layer to their workloads could
be daunting.
If the idea behind machine learning is to create
clinical efficiencies, then it must be easy to
implement and use.
Healthcare experience and technological know-how
help expedite machine learning implementation.

11. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Five Key Machine Learning Differentiators
#3: Machine Learning Model’s Ease of Implementation
A best-practice machine learning initiative
should work within a health system’s existing
IT infrastructure.
The majority of practical machine learning
models can be trained with less than 16GB of
RAM, for example. Installing expensive new
servers isn’t necessary and only adds to the
cost, time, and energy of implementation.

12. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Five Key Machine Learning Differentiators
#3: Machine Learning Model’s Ease of Implementation
On the software side, healthcare firms should
leverage support from the broader machine learning
community that includes experts and others going
through a similar implementation process.
Healthcare.ai is a good example of a community
that fosters machine learning model development
through education and open source tools.
Support from healthcare.ai adds value to the
implementation process, helps analysts learn
data science work, and positions the machine
learning vendor as an extension to a health
system’s analytics team.

13. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Five Key Machine Learning Differentiators
#4: Machine Learning Model’s Interpretability and Buy-In
Machine learning must be placed in the right point
of the clinical workflow to most effectively identify
interventions.
Machine learning-based decision support must
present all possible interventions and help clinicians
make the best choice on a per-patient basis. An
ideal model should not only present a risk score,
but also provide actionable interpretation.
Interpretability is compulsory because clinicians
must know why the model is producing certain
risk scores so clinicians, not models, can
ultimately make the right clinical decisions.

14. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Five Key Machine Learning Differentiators
#4: Machine Learning Model’s Interpretability and Buy-In
Predicting readmissions is a common use case for
machine learning.
Many technology companies can deliver a model
that’s predictive and generates a risk score, but
clinicians need to know what levers are available
within their health system around readmissions or
they won’t know how to use the risk score.
It’s important to link a risk score to available levers
that can directly improve a patient’s outcome.

15. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Five Key Machine Learning Differentiators
#4: Machine Learning Model’s Interpretability and Buy-In
Healthcare professionals are skeptical of new
tools, and for good reason. EMRs have generally
been difficult to use.
It is critical to get buy-in from the end-user on any
machine learning project. This is accomplished
throughout the development process.
As mentioned above, the model must be
interpretable and provide simple, actionable
suggestions.
If it doesn’t, there’s little chance of the model
moving the needle on the associated
outcomes metric.

16. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Five Key Machine Learning Differentiators
#4: Machine Learning Model’s Interpretability and Buy-In
It’s one thing to create a model to make
predictions, but it’s more beneficial to know when
a use case calls for machine learning.
And then know how to tie the appropriate levers
to the model, how to effectively alter clinical
workflow, how to get buy-in, and how to make the
output easy to use.

17. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Five Key Machine Learning Differentiators
#5: Machine Learning Model’s Conformance with Privacy Standards
Healthcare technology, including machine
learning, is bound by certain privacy and
security requirements around patient data,
particularly when it comes to heeding
HIPAA privacy rules.
Some machine learning solutions require that
data be sent to the machine learning tool’s
location, which takes data out of its native,
protected environment (i.e., outside of the
data warehouse or analytics environment).

18. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Five Key Machine Learning Differentiators
#5: Machine Learning Model’s Conformance with Privacy Standards
Moving data to the cloud can be a smart
decision as long as the machine learning
vendor influencing this decision can meet all
of the client’s healthcare analytics needs.
A best-practice machine learning solution is
flexible enough to deliver the tool to the data,
alleviating any privacy and security concerns.
Doing the machine learning work in the pre-
established analytics environment makes it
much easier to deliver a machine learning
project on budget and on time.

19. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Consider Every Angle of Machine
Health systems and payers search for every
opportunity to improve clinical and financial
efficiencies to help them deliver better care
at a lower cost.
Machine learning is an emerging opportunity
that holds significant promise for fulfilling
these goals.
It’s also an investment that is more likely to
pay off if health systems ask the right
questions about what differentiates one
machine learning model, and vendor,
from another.

20. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
For more information:
“This book is a fantastic piece of work”
– Robert Lindeman MD, FAAP, Chief Physician Quality Officer

21. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
More about this topic
Link to original article for a more in-depth discussion.
The Dangers of Commoditized Machine Learning in Healthcare
Machine Learning 101: 5 Easy Steps for Using it in Healthcare
Michael Mastanduno, PhD, Data Scientist
How Machine Learning in Healthcare Saves Lives
Levi Thatcher, VP, Data Science
How Healthcare AI Makes Machine Learning Accessible to Everyone in Healthcare
Levi Thatcher, VP, Data Science
How Healthcare Text Analytics and Machine Learning Work Together to Improve Patient Outcomes
Mike Dow, Technical Director; Levi Thatcher, VP, Data Science
Machine Learning: The Life-Saving Technology That Will Transform Healthcare
Health Catalyst Technology Overview: catalyst.ai™

22. © 2016 Health Catalyst
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
Other Clinical Quality Improvement Resources
Click to read additional information at www.healthcatalyst.com
Levi did his graduate work at the University of Utah, focusing on atmospheric predictability.
There he used ensemble methods to improve numerical models, in terms of both the lead time
and estimated intensity of hurricane development. At Health Catalyst, Levi started out on the
platform engineering team, creating software improvements to the company’s core ETL offering.
Since he moved internally to lead the data science team, Levi founded healthcare.ai, the first open-
source machine learning project focused on healthcare outcomes. He’s now working to integrate
healthcare.ai into each of Health Catalyst’s products and make healthcare.ai the international center of
collaboration for healthcare machine learning.
Levi Thatcher