Comparative hospital performance: new data, borrowed methods, more targeted analysis?

1. Comparative hospital performance: new data,
borrowed methods, more targeted analysis?
Professor Jonathan Karnon

2. Why do we measure hospital performance?
1. To ‘sack the Board’
 Aggregate, or aggregated performance measure
 Under direct control of those being assessed
– Costs OR Process indicators
 Not so interested in magnitude of difference
– Identify general poor performance
2. To inform target areas for quality improvement
 Condition-specific
 Not necessarily under direct control
– Costs, outcomes, and processes
 Value of improvement important

3. Task: Improve
population
health
Option 1: Fund
new
technologies
Option 2:
Improve use of
existing
technologies
Show me the money

4. Increasing interest in quality improvement
0
100
200
300
400
500
600
1983 1988 1993 1998 2003 2008 2013
PubMed"qualityimprovement"in
titlepapers
Year

5. Not a new idea
“a programme of work not only to identify causes of variation at
specific local level, but also to prioritise those variations and causes
that have the most important impact on equity, effectiveness,
efficiency and patient health outcomes”?
Variations in health care: The good, the bad and the inexplicable, The King’s Fund 2011

6. Outline
 Evidence of variation;
 The information: costs, outcomes, and processes
 The incentives: potential strategies for using the information

7. Variation in process
 RAND Corp (McGlynn et al, NEJM 2003)
– 30 acute and chronic conditions and preventive care
– 10 to 80% participants receiving recommended care
 CareTrack (Runciman et al, MJA 2012)
– 22 common conditions
– 32 to 86% compliance with appropriate care

8. Variation in costs
Duckett & Breadon, Controlling costly care: a billion-dollar hospital opportunity, Grattan Institute 2014
$ per admission (2010/11)

9. Variation in outcomes
*adjusted by age, sex, indigenous origin and Diagnostic Group Hierarchical Condition Category (HCC) risk score
30-Day Readmission Rates by Area Health Service of Residence
(NSW: 1 July 2005 – 30 June 2008)*

10. Policy/Practice Relevance?
 Appropriate processes in theory ≈ cost-effective care in practice?
– Costs? Timeliness?
 Best outcomes?
– Meaningful outcomes? At what cost?
 Lowest cost?
– With what outcomes?
 Which providers are providing cost-effective care, and how are they
doing it?

11. Using comparative condition-specific health service data
 Systematic review
– Feedback of comparative performance data alone does not work
 Anecdotal
– Service change works best when common recognition of a problem
 Theory
– Costs and post-discharge outcomes data demonstrates problems
 Despite risk adjustment  ‘my patients are sicker’ syndrome
– Process data provides
 Additional rationale for the existence of a problem
 Starting points for identifying solutions
11

12. Case study: ED chest pain presentations
 Four public hospital in South Australia
 Clinical context, underlying diagnosis could be:
– ST Elevated MI
– Non-ST Elevated MI
– Unstable angina
– Non-cardiac chest pain
 Aims:
– Identify benchmark performer(s) on basis of costs and outcomes
– Assess the potential value of improved performance at non-
benchmark hospitals
– Inform targets for investigation – variation in clinical pathways
12

13. The data
 Clinical data extracted from common data warehouse
– key procedures, pathology test results, movement between hospital
departments and wards, etc.
– automated linkages to population-based mortality data.
 Administrative data, linked to index events to identify other
inpatient separation (episodes) at all South Australian
hospitals
– age, gender, and postcode (SEIFA), co-morbidities
13

14. Missing data
 ED capacity
– ED beds, personnel
– Other presentations: rates and severity
 Inpatient capacity
– Bed occupancy rates
 Cardiac
 Non-cardiac
– Staffing ratios
14

15. The dependent variables
 Costs: Bottom-up patient-level costs available for all inpatient
separations
 Outcomes: 30 day/12 month related admission (unstable
angina, MI, or stroke) or mortality
 Process, not quality indicators
– Pr(admission)
– Time to admission
– Pr(PCI | angiogram)
– Inpatient LoS
15

16. The analysis
 Separate multiple regression models fitted
– Cost, outcome, and process variables
 Hospital interaction terms tested
– Identify patient sub-groups driving variation
 Mean covariate values used to generate predicted outputs
– Bootstrapping, stratified by hospital, to represent uncertainty
16

17. Patient characteristics
Hospital 1 Hospital 2 Hospital 3 Hospital 4
Difference
(p value)
No. patients 1997 1527 2368 2058
Age 60.2 62.5 59.1 57.9 <0.001
Male 0.54 0.53 0.53 0.54 0.768
SEIFA decile 5.4 4.0 6.0 2.6 <0.001
Positive troponin test result 0.17 0.13 0.12 0.11 <0.001
Existing circulatory disorder 0.37 0.33 0.37 0.33 0.002
Cancer 0.03 0.03 0.03 0.02 0.189
COPD 0.03 0.03 0.03 0.03 0.706
Renal disease 0.07 0.08 0.09 0.07 0.398
Diabetes 0.07 0.06 0.09 0.10 <0.001
Dementia/Alzheimer's 0.01 0.02 0.01 0.01 0.716
After hours presentation 0.61 0.62 0.59 0.58 0.082
Weekend presentation 0.24 0.25 0.23 0.24 0.636
17
SEIFA – SocioEconomic Indicator For Areas: 1=lowest decile

18. Inpatient costs per presenting patient
Sub-group N Hosp 4
Hosp 1 -
Hosp 4
Hosp 2 - Hosp 4 Hosp 3 - Hosp 4
All patients 7950 $2,868 $42
$630
($307 to $989)
$510
($299 to $713)
Existing circ., Out-
of-hours
1706 $4,539 $229
$1,542
($786 to $2365)
$510
($299 to $713)
No existing circ.,
Out-of-hours
1082 $1,579 -$451
$136
(-$73 to $361)
$510
($299 to $713)
Existing circ., In-
hours
3050 $4,640 $643
$1,265
($513 to $2111)
$510
($299 to $713)
No existing circ.,
In-hours
2112 $1,610 -$6
-$91
(-$318 to $143)
$510
($299 to $713)

19. N
Hosp 1 Hosp 2 Hosp 3 Hosp 4
Pr(event) RR (95%CI)
12m readmission or death
All patients 7950 0.045
1.59
(1.27 - 1.93)
1.06
(0.80 - 1.35)
1.16
(0.91 - 1.47)
Young males 2281 0.043
1.59
(1.27 - 1.94)
1.09
(0.75 - 1.51)
1.16
(0.91 - 1.47)
Old males 1979 0.102
1.54
(1.25 - 1.85)
1.50
(1.18 - 1.93)
1.14
(0.91 - 1.43)
Young
females
1701 0.025
1.61
(1.28 - 1.97)
0.76
(0.47 - 1.20)
1.16
(0.90 - 1.48)
Old females 1989 0.086
1.55
(1.26 - 1.87)
1.09
(0.81 - 1.42)
1.15
(0.91 - 1.44)
12m mortality
All patients 7950 0.02
1.42
(0.93 to 2.05)
0.82
(0.47 to 1.23)
1.05
(0.83 to 1.2)
12m readmission
All patients 7950 0.03
1.72
(1.27 to 2.27)
1.22
(0.84 to 1.66)
1.30
(1.01 to 1.92)

20. C-E Acceptability Planes
value_avoiding_death_000s
0.00
value_avoiding_readmission_000s
50.00
prhosp_3_cost_effective
100.00
0.00
50.00
100.00
0.00
0.17
0.35
value_avoiding_death_000s
0.00
value_avoiding_readmission_000s
50.00
prhosp_1_cost_effective
100.00
0.00
50.00
100.00
0.02
0.39
0.76
value_avoiding_death_000s
0.00
value_avoiding_readmission_000s
50.00
prhosp_4_cost_effective
100.00
0.00
50.00
100.00
0.16
0.56
0.97
value_avoiding_death_000s
0.00
value_avoiding_readmission_000s
50.00
prhosp_2_cost_effective
100.00
0.00
50.00
100.00
0.00
0.01
0.03

21. Pr(Admitted)
N
Hospital 1 Hospital 2 Hospital 3 Hospital 4
Pr(admitted) RR (95%CI)
All patients 7950 0.77
0.87
(0.83 - 0.91)
0.92
(0.89 - 0.96)
0.92
(0.88 - 0.96)
Troponin +ive, Existing
circulatory condition
842 0.97
0.99
(0.98 - 1.01)
0.97
(0.94 - 0.99)
0.97
(0.94 - 0.99)
Troponin +ive, No
existing circulatory
condition
216 0.87
0.88
(0.79 - 0.95)
0.89
(0.78 - 0.97)
0.89
(0.78 - 0.97)
Troponin -ive, Existing
circulatory condition
1946 0.89
0.98
(0.94 - 1.03)
0.97
(0.95 - 0.99)
0.97
(0.94 - 0.99)
Troponin -ive, No
Existing circulatory
condition
4946 0.62
0.71
(0.65 - 0.77)
0.90
(0.84 - 0.96)
0.90
(0.83 - 0.96)
21

22. Pr(PCI | Angiogram)
22
N
Hospital 1 Hospital 2 Hospital 3 Hospital 4
Pr() RR (95%CI)
All patients 7950 0.14
2.40
(1.71 - 3.36)
1.79
(1.31 - 2.44)
1.57
(1.14 - 2.15)
Troponin +ive,
Weekend
279 0.30
2.18
(1.36 - 3.44)
1.07
(0.62 - 1.87)
0.94
(0.54 - 1.53)
Troponin +ive,
Weekday
779 0.22
2.03
(1.44 - 2.79)
1.22
(0.83 - 1.74)
1.72
(1.26 - 2.28)
Troponin -ive,
Weekend
1632 0.17
2.76
(1.48 - 4.74)
1.72
(0.96 - 2.93)
0.93
(0.49 - 1.64)
Troponin -ive,
Weekday
5260 0.12
2.36
(1.56 - 3.41)
2.02
(1.35 - 2.92)
1.90
(1.32 - 2.64)

23. Length of Stay as inpatient
23
N
Hosp4
(hrs)
Hosp1 - Hosp4 Hosp2 - Hosp4 Hosp3 - Hosp4
All patients 5373 37.2
7.4
(4.46 - 10.32)
4.5
(0.79 - 8.38)
10.5
(7.58 - 13.42)
Troponin +ive, Out-
hours presentation
561 56.4
3.3
(-7.28 - 14.39)
18.0
(2.52 - 34.19)
16.3
(5.84 - 27.24)
Troponin +ive, In-
hours presentation
325 51.1
7.0
(-3.44 - 17.45)
13.8
(-1.55 - 29.42)
15.2
(4.33 - 26.11)
Troponin -ive, Out-
hours presentation
2702 33.8
6.5
(3.29 - 9.52)
3.8
(-0.13 - 7.87)
9.9
(6.50 - 13.03)
Troponin -ive, In-
hours presentation
1785 35.3
10.2
(5.88 - 14.53)
-0.4
(-5.02 - 4.19)
8.7
(4.76 - 12.46)

24. How to compare larger numbers of providers?
 Stratify by clinical process and identify best performing strata?
– Hypothesis driven?
– Process mining driven?
 Group hospitals by performance and compare processes?
– Empirical stratification

25. Informing action
 “Publicising the existence of unwarranted variations and their
causes does not guarantee that they will be tackled.”
 “local health organisations… be required to publicly justify and
explain in a consistent way their relative position on key aspects of
health care variation.
 …it may also be necessary to explore the development of harder-
edged, locally focused incentives to encourage action to deal with
unwarranted variation.”
Variations in health care: The good, the bad and the inexplicable, The King’s Fund 2011

26. The incentives
26
1. Sticks
a) Public Reporting
b) Mandated action plans
2. Carrots
a) Pay-for-Performance?
b) External Services Improvement Fund
 Mindful of the NHS Improvement Fund…
 1b + 2b
– Externally identified areas for improvement
– Externally reviewed and supported applications to fund improvement
projects

27. Prioritisation criteria
27
 Expected Value of Removing Variation (EVRV)
– Hospital 2, 1527 patients per annum
– Costs per patient could reduce by $630
– Mortality could decrease by 1% (x $200,000?)
– Readmissions decrease by 2% (x $50,000?)
 Annual EVRV = 1527 x $(630 + 2000 + 1000) = $5.5 million
 x5? x10?

28. Zombies and Paradoxes
• “political paradox of rationing”
• “the appeal for transparency in medical decision-making is
like a zombie, an idea that refuses to die despite its limited
utility”
• Do the benefits of open and explicit quality improvement
outweigh the:
 Financial costs and Political risks of unrealistic expectations?
Oberlander et al, Rationing medical care: rhetoric and reality in the Oregon Health Plan, CMAJ, 2001; 164(11):1583-7

29. Summary
29
 Huge scope for service evaluation and improvement
– Electronic data systems
– Linkage facilities
 Post-improvement evaluation: ICER estimates to inform the
– Design of future improvement processes
– Balance of spending on new technologies and existing services

30. Acknowledgements
 Clarabelle Pham, Andrew Partington, Orla Caffrey, Jason Gordon,
Brenton Hordacre, David Ben-Tovim, Paul Hakendorf, Maria Crotty
 Funders: SA Health, NHMRC, HCF Foundation