Lessons learned over 20 years. This time we focus on technology lessons learned from experience at Intermountain Healthcare, Northwestern Medicine and Cayman Islands Health Authority

1. Part Two - 20 Years in Healthcare
Analytics & Data Warehousing
What did we learn? What’s the future?
Shakeeb Akhter - Director,
Enterprise Data Warehouse
at Northwestern Medicine
Lee Pierce - Healthcare Chief
Data Officer at Sirius and former
Chief Data Officer at
Intermountain Healthcare
Dale Sanders, President of
Technology at Health Catalyst

2. • People
• Processes
• Technology
• What did we do right?
• What did we do wrong?
• What’s the future?
Many thanks to Lee and Shakeeb
Agenda

3. © 2018
Health
Catalyst
I learned what was right by first doing what was wrong.
Luckily, I watched others do what was wrong and learned from them
without suffering their wounds as mine.
Generally Speaking…
3

4. © 2018
Health
Catalyst
What did we do right?
• Hired a balanced combination of social skills, domain skills (clinical and financial),
and technical skills
• Good balance of centralized/decentralized development of these skills… 60/40 split
What would we have done differently?
• Overlooked the cultural issues of ”data”… threatened legacy source systems’
teams… Intermountain the IDN vs Northwestern the AMC
• Too much dependence on matrixed technology resources, esp DBAs and Sys
Admins who were skilled in transaction databases, not analytics
What are we thinking about the future?
• Data science, non-relational technical skills
• A new role: The Digitician
• Reality of the 8% Folly
People - Dale
4

5. © 2018
Health
Catalyst
Data as a Strategic Corporate Asset
This should be
every healthcare
CEO’s strategic
data acquisition
roadmap
5

6. © 2018
Health
Catalyst
What did we do right?
• Hired amazing data professionals - dedicated, smart, committed - treated them right
• Embraced early support from business and clinical leaders that allowed us to start
early: Dr. Brent James, Dr. Homer Warner
• Partnered with & empowered the data analysts to be primary producers of analytics
• Engaged business leaders in Data and Analytics strategy and execution
What would we have done differently?
• More thoughtful and coordinated growth, hiring of data analysts across the business,
earlier job description standardization and smarter growth
• More bold steps to organize analytics as centrally managed, locally deployed
What are we thinking about the future?
• Improved data literacy for leaders = how to ask for analytics help, how to use
insights to improve decision making and change business processes = value
People - Lee
6

7. © 2018
Health
Catalyst
What did we do right?
• Small, highly skilled team at the start resulted in agility and faster product to market
• Multiple tangentially related functions developed within the EDW team that resulted
in self-sufficiency
• Recruited EMR Application Analysts with exposure to Business Intelligence
What would we have done differently?
• Develop additional roles to accommodate growth and scale with the organization
• Recruit clinical and operational SMEs to assist with understanding of data
What are we thinking about the future?
• Develop specialized roles in order to scale for future growth
• Blur the lines between EDW & Analytics via hybrid resources
People - Shakeeb
7

8. © 2018
Health
Catalyst
What did we do right?
• Design and code reviews
• Lightweight data governance – govern to the least extent necessary for the greatest
common good
• Using EHR logs to analyze & understand workflow (the first ‘Meaningful Use’)
• Running the data warehouse like a small business
What would we have done differently?
• Prioritization management, demand vs. capacity
• Managing expectations of data and analytic quality validation
What are we thinking about the future?
• AI/data science changes almost everything... design reviews, data governance,
analytic validation
• Study the DevOps of AI
8
Processes - Dale

9. © 2018
Health
Catalyst
• Enabled by bio-integrated sensors, patients
hold more data about themselves than the
healthcare system
• Their data is constantly being updated and
uploaded to cloud-based AI algorithms
• Those algorithms diagnose the patient’s
condition, calculate a composite health risk
score, and recommend options for
treatment or maintaining health
• The algorithm suggests options for a “best
fit” care provider and the ability to socially
interact with other patients like them
Future of Diagnosis and Treatment
9
• The patient engages with the care provider,
enabled with the output of the AI algorithms

10. © 2018
Health
Catalyst
What did we do right?
• Vision for the use of data – clinical quality improvement
• Focused on real business/clinical use-cases and needs
• Discipline (in many use cases) to implement insights – close the loop
• Practical data governance – project-based, use-case driven, clear business value
What would we have done differently?
• Require analytics return-on-investment planning and measurement for large
analytics projects, document ROI, communicate successes and repeat
• Establish (and enforce) better standard development practices from the beginning
(EDW/ETL specifically), reducing maintenance required after dev.
• Should have been more bold about data governance value and investment
What are we thinking about the future?
• More business discipline to use insights generated with more focus on end value
10
Processes - Lee

11. © 2018
Health
Catalyst
11
Processes - Lee

12. © 2018
Health
Catalyst
What did we do right?
• Single EDW for Research and Operational Analytics
• Small teams leveraged agile principles (informally) to start
• Data Steward Approval + Power User Model
• Report Deployment process
What would we have done differently?
• Avoid waterfall, embrace agile
• Fail faster and more often
• Bring the customer to the table
What are we thinking about the future?
• Dev + Data Ops; Agile Product Teams, source control, automated testing and deployment
• Investing heavily in Data Management; Data Quality and Master Data Management
• Embracing Agile tools & methods
Processes - Shakeeb
12

13. © 2018
Health
Catalyst
The primary goal of Data Ops is to achieve Customer Satisfaction with DataAssets &Analytical
Solutions across the enterprise
• Standardization: Establish standardized tool sets and
processes to increase productivity of data teams
• Cross-Functional Teams: Break-Down silos within data
teams (EDW/Analytics) by establishing cross-functional
teams consisting of Data Architects, Analytics Consultants,
Report Writers, ETL Developers and Data Scientists
• Customer-Focus: Increased alignment with customer
focus and priorities by establishing customer-centric
product teams to serve data needs for operating units and
system functions
• Value-Add: Provide continuous delivery of analytical
insights to enterprise customers in order to establish a
data-driven culture
• Customer Satisfaction: Increase customer satisfaction due
to customer-focused teams and alignment with customer
priorities
Increased
Customer
Satisfaction
Continuous
Delivery of
Analytical Insights
Cross-Functional,
Customer-Centric,
Product Teams
Standardized Tools
and Processes
13
Data Ops - Goals

14. © 2018
Health
Catalyst
Data Ops Product Teams contain all skills required to transform data from its raw form to an
end-user analytical deliverable; report, dashboard, KPI, or analytical application. Possible roles,
and product teams are displayed as a sample below.
Product Team A
NOTE: The above is a draft representation. Specific Product Teams are being defined byAnalytics and EDW teams and will be formed accordingly.
Data
Architect
Analytics
Manager
Sr. Data
Architect
ETL
Developer
Data
Quality
Analyst
Sr.
Analytics
Associate
Analytics
Associate
Product Team B
Data
Architect
Analytics
Manager
Sr. Data
Architect
ETL
Developer
Data
Quality
Analyst
Sr.
Analytics
Associate
Analytics
Associate
• Analytics Manager is accountable for the
customer satisfaction with EDW
deliverables, including self-service data &
reporting applications
• Sr. DataArchitect is accountable for the
efficient & effective integration of data in
EDW and the necessary data structures to
support reporting
• Analytics Manager & Sr. DataArchitect will
collectively set priorities and timelines and
escalate to leadership as needed
• Staff level roles work collaboratively as one
collective team with the customer’s
satisfaction of EDW tools provided as the
primary outcome metric
• Specific # of resources & coverage of
product teams will vary based on expected
demand & resource availability, but FTEs
will be a part of multiple product teams
14
Data Ops – Product Teams

15. © 2018
Health
Catalyst
What did we do right?
• Ignored the enterprise data model in favor of late binding… didn’t need expensive ETL tools
• Took advantage of tried and true SMP architectures
• Ignored the early love affair of Hadoop
• Blended text with discrete data when nobody else was doing it
• Picked Microsoft when nobody else was doing it
What would we have done differently?
• Too much late binding data modeling
• Jumped into massively parallel processing too soon… let the pure technologists talk me into it
• Too much faith in an “enterprise standard” business intelligence tool
What are we thinking about the future?
• Read-only, batch-oriented relational data warehouses are already outdated
• Hybrid transaction & analytic architectures… Lambda and Kappa architectures
Technology - Dale
15

16. External
Data
Sources
EMR
SQL
HL7
X12
FHIR
Flat-files
XML
Data
Integration
Batch Data
Realtime Data
Mirth,
RabbitMq
Catalyst
Data Engine
SQL
Big Data
.NET
DOS
App Cluster
Apps
Microservices
Highly Available
Horizontally Scalable
Angular, D3, .NET, Java, Docker,
Kubernetes, JSON
Datamart Designers & Tools
SAMD, SMD, Atlas, Ops Console, Analytics Portal
Data & Compute Cluster
SQL Server, Hadoop, Spark, ElasticSearch
Transactional data store
SQL, Shared Disk
DOS Marketplace
Apps, Content, AI models
Catalyst AI
Engine
Catalyst.ai
healthcare.ai
.NET, R, Python
Azure
Azure
AI Cluster
Spark, R, Python
SQL
FTP
HL7
FHIR
SQL
HTTP
FHIR
HL7
External
Apps
EMRs
Reports
The Health Catalyst Data Operating System Architecture

17. © 2018
Health
Catalyst
What did we do right?
• Resisted early and on-going pressure to move data to canonical healthcare data
models
• Purchased and implemented Tableau which has been a big value add
• Collaborated with others – HDWA/HDAA, HMA, regional HIMSS events, etc.
• Used a thoughtful approach to make technology spend decisions – Does the
technology have proven value in healthcare? Does it augment current capabilities?
Does it enhance our capability to achieve better healthcare outcomes? Is value >
investment?
• Built an analytics reference architecture to use to rationalize data/analytics and
tools/tech decisions
Technology - Lee
17

18. © 2018
Health
Catalyst
Enterprise Analytics Framework
18
Data Source
Data
Acquisition
Data Storage Data Enrichment BI & Analytics
Streaming
Vended
ETL
Vended Data Stores
Non-relational
Data Lake
Hadoop
Graph
Index
Relational
Relational
Data Discovery
Enterprise
Self-Service
ETL
Statistical
Processing
Machine
Learning
Enterprise
ETL
Analytics Tools
Investigation
Modeling
Data Mining
Analytic Workbench
Surveillance
Data Science
Alerts
Real Time Apps
NLP / Text
Mining
Vended
Reporting
EMR
ERP
Pharmacy
Structured
Medical Devices
Clinical Document
Imaging Data
Semi/Unstructured
Others
HIE
Benchmark
Survey
External
Public Data
Others
Enterprise
ETL
Replication
/ CDC
Data
Processing
Information
Distribution
External
Research
Government
Reporting
Analytic
Applications
Source Data
Mart
Subject Area
Data Mart
Master Data
Nomenclature
Sandbox
Investigative
Departmental
Research
Data Governance / Data Quality
Data Source
Applications
Lab
Scheduling
Others
Enterprise BI
Self-Service BI
Reports
Dashboards
Master Data Management
Security
Self-
Service ETL
Web
Services /
API
EDW
Virtualization
Ad-Hoc Query
Source: Intermountain’s Data/Analytics team

19. © 2018
Health
Catalyst
Technology - Lee
19
What would we have done differently?
• Would not have migrated from Crystal Reports to Cognos – migration cost vs.
additional value realized/not realized
• Would have started earlier to build the “shared/common” data marts to limit
duplication of ETL/data models over the years, better about enforcing internal data
modeling standards
• Would have built a more meaningful working relationship between EMR vendor and
our analytics program and teams, focus on opportunities to deliver value through
analytics together
What are we thinking about the future?
• Machine Learning/AI – will continue to grow in impact and value, needs to be
embedded in business/clinical workflows (still need good data and data mgmt)
• Personalized analytics – n of 1
• Cloud – data warehouse, ETL tools, data governance tools, reporting tools

20. Seven Reasons for Using the Cloud
Support high concurrency querying and pay only for (often spikey) actual usage
Inexpensive to try things & allow for throw-away;OpEx model makes try before you buy attractive
Don’t have to wait for procurement to acquire and IT to set up physical environments
Data can stay closer to customers and where its created and you don’t have to move it
Lower cost, scalable back-up system possible
SaaS model can commoditize most lower level technical maintenance tasks
1) ELASTICITY
2) EXPERIMENTATION
3) AGILITY
4) GRAVITY
5) FOCUS ON BUSINESS
6) CONTINUITY
7)WORKLOAD
BALANCING
Gain excess capacity on demand
www.siriuscom.com

21. © 2018
Health
Catalyst
What did we do right?
• Utilized MSFT BI for Database, Integration, Reporting, and Analytics
• Did not buy into the hype of new architectures and products early (i.e., MPP, Hadoop)
• Modernized the platform once market was more mature; learned from others
What would we have done differently?
• Modernize the platform slightly earlier to enable data discovery prior to ingesting data into EDW
• More focus on technologies to manage data rather than analyze data; Data Governance, Data Quality,
MDM
What are we thinking about the future?
• Continue to integrate data, and expand architecture to enable Self-Service Analytics across the enterprise
• Extracting value from NLP, implementing Predictive and Prescriptive Analytics at enterprise scale
• Leveraging cloud for big data workloads, Dev/Test, Disaster Recovery and ‘Cold’ storage
• Launching a formal Data Quality Program; vital to making ML, AI a reality
Technology - Shakeeb
21

22. © 2018
Health
Catalyst
Current state of a data warehouse
22
Traditional Data Warehousing Approach

23. © 2018
Health
Catalyst
Modernized Data Warehouse Architecture
2
3
Cloud Scale & Performance. Single-Query Model.Data Science.
23

24. © 2018
Health
Catalyst
Healthcare Analytics Summit 18
Sept. 11-13, Salt Lake, Grand America Hotel
TOBY COSGROVE, MD
former CEO and President of
Cleveland Clinic (2004-2017),
who as a cardiac surgeon
performed more than 22,000
operations and holds 30 patents
for medical innovations
KIM GOODSELL
the actualized ‘genomified,’ quantified,
digitalized “patient of the future," her debut at
the 2014 Future of Genomic Medicine
conference made headline news
announcing— “The patient from the future,
here today”
DANIEL KRAFT, MD
a Stanford and Harvard trained physician-
scientist, inventor, entrepreneur, and
innovator, Kraft is the Founder and Chair of
Exponential Medicine, a program that
explores convergent, rapidly developing
technologies and their potential in
biomedicine and healthcare
BRENT JAMES, MD
former Chief Quality Officer at
Intermountain Healthcare - known
internationally for his work in
clinical quality improvement,
patient safety, and the
infrastructure that underlies
successful improvement efforts
PENNY WHEELER, MD
President and Chief Executive
Officer of Allina Health,
returns a second time as one
of the most popular HAS
speakers ever
MARC RANDOLF
Co-founder of Netflix, Marc will
share the Netflixed story: how a
scrappy Silicon Valley startup
brought down Blockbuster and
the lessons that could be
applicable to healthcare
JILL HOGGARD GREEN
PhD, RN, Chief Operating Officer – Mission
Health and President – Mission Hospital,
and recently named to the 2017 Becker’s
Healthcare list of the country’s top Women
Hospital and Health System Leaders to
Know
ROBERT WACHTER, MD
global leader in healthcare safety,
quality, policy, IT; Chair of the
Department of Medicine, University of
California, San Francisco; best-selling
author, “The Digital Doctor: Hope, Hype
and Harm at the Dawn of Medicine’s
Computer Age”
More highlights
4 Digital Innovators (Keynotes)
AI Showcase (10 walkabout case studies)
Digitizing the Patient Showcase (10-12 stations)
28 Educational, Case Study, and Technical Breakouts
24 Analytics Walkabout Projects
More Networking (Introducing “Brain Date”)
CME Accreditation For Clinicians
5-Star Grand America Hotel Experience
96 Total Presentations
National keynotes
Employer
Innovation
Scott
Schreeve
MD, CEO, Crossover Health
Payer
Innovation
Kevin
Sears
Executive Director of Marketing
and Network Services, Cleveland
Clinic
Biosensor
Innovation
John
Rogers
PhD, Founding Director, Center
Bio-Integrated Electronics,
Northwestern University
Pricing
Innovation
Gene
Thompson
Project Director, Health City
Cayman Islands

25. Thank You!