The document analyzes the impact of the UK's Quality and Outcomes Framework (QOF), a pay-for-performance scheme for primary care physicians introduced in 2004. It compares changes in quality indicators that were fully incentivized by the QOF, partially incentivized, and non-incentivized. In the short-term (2004/05), fully incentivized indicators showed the largest improvements above expectations, while partially incentivized treatment indicators declined. In the long-term (2006/07), fully incentivized indicators continued improving and non-incentivized/partially incentivized indicators declined below expectations, suggesting the QOF primarily impacted incentivized aspects of care.

1. UK pay-for-performance scheme
Methods
Results
Conclusions
QOF impact on the quality of incentivised and
non-incentivised aspects of primary care
Evangelos Kontopantelis1 Tim Doran*1
Stephen Campbell1 Jose Valderas1 Martin Roland2
Chris Salisbury3 David Reeves1
1National Primary Care Research and Development Centre
University of Manchester
2General Practice and Primary Care Research Unit, University of Cambridge
3Academic Unit of Primary Health Care, University of Bristol
HSRN, 3rd June 2010
Kontopantelis The QOF impact

2. UK pay-for-performance scheme
Methods
Results
Conclusions
Manchester!
pronounced Manchestoh
Where sunglasses last for a lifetime.
20 different accents in a 40 mile
radius (and these are just the local
ones).
If you don’t speak football you die
alone.
Sarcasm is a way of life (Is it really?
Nooooooooo!)
Kontopantelis The QOF impact

3. UK pay-for-performance scheme
Methods
Results
Conclusions
Outline
1 UK pay-for-performance scheme
The idea
Framework details
2 Methods
The design
Data extraction
Analysis
3 Results
Trends
Indicator performance
Short term QOF effect - by indicator group
Long term QOF effect - by indicator group
4 Conclusions
Kontopantelis The QOF impact

4. UK pay-for-performance scheme
Methods
Results
Conclusions
The idea
Framework details
Improving quality of care
a (very) juicy carrot...
A P4P program kicked off in April 2004 with the
introduction of a new GP contract.
General practices are rewarded for achieving a set of
quality targets for patients with chronic conditions.
The aim was to increase overall quality of care and to
reduce variation in quality between practices.
The incentive scheme for payment of GPs was named
Quality and Outcomes Framework (QOF).
Initial investment estimated at £1.8 bn for 3 years
(increasing GP income by up to 25%).
QOF is reviewed at least every two years.
Kontopantelis The QOF impact

5. UK pay-for-performance scheme
Methods
Results
Conclusions
The idea
Framework details
Quality and Outcomes Framework
details for years 1 (2004/05) and 5 (2008/09)
Domains and indicators in year 1 (year 5):
Clinical care for 10 (19) chronic diseases, with 76 (80)
indicators.
Organisation of care, with 56 (36) indicators.
Additional services, with 10 (8) indicators.
Patient experience, with 4 (5) indicators.
Implemented simultaneously in all practices (a control
group was out of the question).
Into the 7th year now (01Mar10/31Apr11); cost for the first
6 years was well above the estimate at £5.8 bn approx.
Kontopantelis The QOF impact

6. UK pay-for-performance scheme
Methods
Results
Conclusions
The idea
Framework details
Performance
incentivised aspects keep improving but what about the non-icentivised ones?
050010001500
Numberofpractices
0 20 40 60 80 100
Percentage of patients
08/09 07/08 06/07 05/06 04/05
Overall, 48+2 (smoking) indicators
Reported achievement Quality scores for all
QOF clinical indicators
have been improving.
Only a small proportion
of all clinical care.
Concerns that quality for
non-incentivised aspects
of care may have been
neglected.
How to measure
performance on
non-incentivised care?
Kontopantelis The QOF impact

7. UK pay-for-performance scheme
Methods
Results
Conclusions
The design
Data extraction
Analysis
Clinical indicators
Two aspects to clinical indicators:
a disease condition (e.g. diabetes, CHD).
a care activity (e.g. influenza vaccination, BP control).
Three indicator classes, in terms of incentivisation:
(A) Condition & process incentivised within QOF (28 ind)
(B) Condition or process incentivised (13 ind)
(C) Neither condition nor process incentivised (7 ind)
Example (Indicators)
(A) DM11: Patients with diabetes in whom the last blood
pressure (within 15m) is 145/85 or less
(B) B4: Patients with peripheral arterial disease who have
a record of total cholesterol in the last 15m
(C) C4: Patients with back pain treated with strong
analgesics (co-dydramol upwards) in the last 15m
Kontopantelis The QOF impact

8. UK pay-for-performance scheme
Methods
Results
Conclusions
The design
Data extraction
Analysis
Research questions
the obvious ones at least!
We aimed to compare the three classes on changes in
quality from pre-QOF (2000/01 - 2002/03) to post-QOF
(2004/05 - 2006/07).
Would class A indicators show most improvement?
Would class B show some ‘halo’ effects since they involve
either a QOF condition or activity?
Has quality for class C indicators declined?
Kontopantelis The QOF impact

9. UK pay-for-performance scheme
Methods
Results
Conclusions
The design
Data extraction
Analysis
The General Practice Research Database
GPRD
Holds event data for more than 270 English practices, from
1999 (545 active practices in Apr10 and 11.2m patients).
Database is broken down to numerous tables, because of
the volume of data.
Final sample was 600,000 patients from 150 nationally
representative practices.
Allows us to:
measure quality of care prior to the introduction of QOF.
construct non-incentivised indicators of quality.
At the heart of the database lies the use of specific codes
by GPs to record all information from a patient’s visit.
Kontopantelis The QOF impact

10. UK pay-for-performance scheme
Methods
Results
Conclusions
The design
Data extraction
Analysis
The process
begins with a stab in the dark...
Size of the tables prohibits looking at codes one by one.
Instead we use search terms to identify potentially relevant
codes in the lookup files and create draft lists.
Clinicians go through the draft lists and select the relevant
codes.
Using the finalised code lists we search for events the main
database files.
For each indicator and year, we used the associated
clinical rules and retrieved events to construct a % score:
(cases where indicator was met / eligible patients)*100%.
Kontopantelis The QOF impact

11. UK pay-for-performance scheme
Methods
Results
Conclusions
The design
Data extraction
Analysis
Issues to tackle
is the analysis ever easy...?
Indicator classes are imbalanced.
Three different types of activities (x3 classes = 9 groups):
clinical processes related to measurement (PM/R).
A:17 B:9 C:0
e.g. blood pressure measurement
clinical processes related to treatment (PT).
A:6 B:4 C:7
e.g. influenza immunisation
intermediate outcome measures (I).
A:5 B:0 C:0
e.g. control of HbA1c to 7.4 or below
Quality of care was already improving (prior to QOF).
The ceiling has been reached for certain ‘easy’ indicators.
Kontopantelis The QOF impact

12. UK pay-for-performance scheme
Methods
Results
Conclusions
The design
Data extraction
Analysis
The approach
The main analysis used logit-transformed scores, due to
the ceiling effect.
Untransformed scores were used in a sensitivity analysis.
The six available indicator groups (of a possible nine) were
compared, on performance above expectation.
FE model selected; controlling for RTTM, denominator,
patient age and gender.
All analyses performed in STATA v10.1
Interrupted Time Series methods employed...
Kontopantelis The QOF impact

13. UK pay-for-performance scheme
Methods
Results
Conclusions
The design
Data extraction
Analysis
The approach
Interrupted Time Series
ITS multi-level multivariate regression analyses, allowed us
to compare the six indicator groups on the basis of two
outcome measures:
0
10
20
30
40
50
60
70
80
90
100
%
0
10
20
30
40
50
60
70
80
90
100
%
Pre-trend Observed Uplift in year 1 Uplift in year 3
The difference between
observed and expected
achievement, in 2004/05.
The difference between
observed and expected
achievement, in 2006/07.
Kontopantelis The QOF impact

14. UK pay-for-performance scheme
Methods
Results
Conclusions
Trends
Indicator performance
Short term QOF effect - by indicator group
Long term QOF effect - by indicator group
Trends by indicator group
untransformed scores
20
30
40
50
60
70
80
90
100
%
2000/01 2001/02 2002/03 2003/04 2004/05 2005/06 2006/07
Year
A−PM/R (17) A−PT (6) A−I (5)
B−PM/R (9) B−PT (4) C−PT (7)
using group means of indicator means (by practice)
in brackets, the number of indicators in each group
Percentage scores
Indicator group performance
Kontopantelis The QOF impact

15. UK pay-for-performance scheme
Methods
Results
Conclusions
Trends
Indicator performance
Short term QOF effect - by indicator group
Long term QOF effect - by indicator group
Trends by indicator group
logit transformed scores
−1.00.01.02.03.0
logitscale
2000/01 2001/02 2002/03 2003/04 2004/05 2005/06 2006/07
Year
A−PM/R (17) A−PT (6) A−I (5)
B−PM/R (9) B−PT (4) C−PT (7)
using group means of indicator means (by practice)
in brackets, the number of indicators in each group
Logit transformed scores
Indicator group performance
Kontopantelis The QOF impact

16. UK pay-for-performance scheme
Methods
Results
Conclusions
Trends
Indicator performance
Short term QOF effect - by indicator group
Long term QOF effect - by indicator group
Observed performance compared to expectation
logit transformed scores
sig positive ‐ sig negative sig positive ‐ sig negative
A-I 5 0 0 4 1 0
A-PM/R 17 0 0 9 3 5
B-PM/R 3 5 1 1 1 7
A-PT 5 1 0 2 3 1
B-PT 0 2 2 0 2 2
C-PT 1 4 2 1 4 2
Difference in 2004/05 Difference in 2006/07
Kontopantelis The QOF impact

17. UK pay-for-performance scheme
Methods
Results
Conclusions
Trends
Indicator performance
Short term QOF effect - by indicator group
Long term QOF effect - by indicator group
Difference in 2004/05
of observed performace compared to expectation
-1.0
-0.5
0.0
0.5
1.0
1.5
logitscale
Indicator group
Uplift 04/05 - FE, logit, controlled
-0.05
0.00
0.05
0.10
0.15
0.20
%
Indicator group
Uplift 04/05 - FE, linear, controlled
All three indicator groups of
class A significantly increased
in level above expectation
post-QOF.
Treatment indicators of class
B significantly decreased in
level below expectation
post-QOF.
Kontopantelis The QOF impact

18. UK pay-for-performance scheme
Methods
Results
Conclusions
Trends
Indicator performance
Short term QOF effect - by indicator group
Long term QOF effect - by indicator group
Difference in 2004/05
comparing ordered means across groups, using the Newman-Keuls procedure
Uplift in
2004/5
Group
Fully
incentivized
measurement
Fully
incentivized
outcome
Fully
incentivized
treatment
Partially
incentivized
measurement
Unincentivized
treatment
Partially
incentivized
treatment
Mean
*
14.5% 8.2% 4.2% 0.8% -0.7% -1.5%
95% confidence interval (14.0, 15.0) (7.3, 9.2) (3.2, 5.3) (-0.2, 1.8) (-1.8, 0.5) (-3.0, -0.2)
P value <0.001 <0.001 <0.001 0.128 0.257 0.03
Difference between means** [---------------------------]
* Group means based on logit-transformed data, back-transformed to percentage scores.
** Neuman-Keuls tests. Means connected by a dashed line were not significantly different (p > 0.05).
Uplift in
2006/7
Group
Fully
incentivized
outcome
Fully
incentivized
measurement
Fully
incentivized
treatment
Unincentivized
treatment
Partially
incentivized
treatment
Partially
incentivized
measurement
Mean * 3.9% 3.9% 2.4% -1.2% -2.8% -5.1%
95% confidence interval (2.9, 4.8) (3.1, 4.6) (1.4, 3.3) (-2.3, -0.2) (-4.2, -1.5) (-6.2, -3.9)
P value <0.001 <0.001 <0.001 0.024 <0.001 <0.001
Kontopantelis The QOF impact

19. UK pay-for-performance scheme
Methods
Results
Conclusions
Trends
Indicator performance
Short term QOF effect - by indicator group
Long term QOF effect - by indicator group
Difference in 2006/07
of observed performace compared to expectation
-1.0
-0.5
0.0
0.5
1.0
1.5
logitscale
Indicator group
Uplift 06/07 - FE, logit, controlled
-0.05
0.00
0.05
0.10
0.15
0.20
%
Indicator group
Uplift 06/07 - FE, linear, controlled
All three indicator groups of
class A significantly increased
in level above expectation
post-QOF.
All partially incentivised and
non-incentivised indicator
groups significantly
decreased in level below
expectation post-QOF.
Kontopantelis The QOF impact

20. UK pay-for-performance scheme
Methods
Results
Conclusions
Trends
Indicator performance
Short term QOF effect - by indicator group
Long term QOF effect - by indicator group
Difference in 2006/07
comparing ordered means across groups, using the Newman-Keuls procedureUplift in
2004/5
Group
Fully
incentivized
measurement
Fully
incentivized
outcome
Fully
incentivized
treatment
Partially
incentivized
measurement
Unincentivized
treatment
Partially
incentivized
treatment
Mean
*
14.5% 8.2% 4.2% 0.8% -0.7% -1.5%
95% confidence interval (14.0, 15.0) (7.3, 9.2) (3.2, 5.3) (-0.2, 1.8) (-1.8, 0.5) (-3.0, -0.2)
P value <0.001 <0.001 <0.001 0.128 0.257 0.03
Difference between means** [---------------------------]
* Group means based on logit-transformed data, back-transformed to percentage scores.
** Neuman-Keuls tests. Means connected by a dashed line were not significantly different (p > 0.05).
Uplift in
2006/7
Group
Fully
incentivized
outcome
Fully
incentivized
measurement
Fully
incentivized
treatment
Unincentivized
treatment
Partially
incentivized
treatment
Partially
incentivized
measurement
Mean * 3.9% 3.9% 2.4% -1.2% -2.8% -5.1%
95% confidence interval (2.9, 4.8) (3.1, 4.6) (1.4, 3.3) (-2.3, -0.2) (-4.2, -1.5) (-6.2, -3.9)
P value <0.001 <0.001 <0.001 0.024 <0.001 <0.001
Difference between means** [------------------------]
* Group means based on logit-transformed data, back-transformed to percentage scores.
** Neuman-Keuls tests. Means connected by a dashed line were not significantly different (p > 0.05).
Kontopantelis The QOF impact

21. UK pay-for-performance scheme
Methods
Results
Conclusions
Summary
Short term:
Overall, all three groups of fully incentivised indicators
exhibited performance above expectation. The uplift varied
from 1.1% to 38.2% with 4 indicators (smoking) having
uplifts of over 30%.
Partially incentivised Measurement/Recording indicators
demonstrated significantly lower than expected gains (on
average).
Long term:
Overall, the three fully incentivised groups continued to
have positive uplifts, although none exceeded 4%.
The three partially incentivised and non-incentivised groups
displayed significantly negative uplifts, on average.
Kontopantelis The QOF impact

22. UK pay-for-performance scheme
Methods
Results
Conclusions
Discussion
Improvement in quality pre-QOF suggests that the quality
initiatives at that time had an effect.
The rate of improvement was not sustained and
improvement appeared to plateau after 04/05.
No such plateau effect for non-QOF indicators with similar
rates of achievement. Practices may have relaxed their
efforts for the incentivized indicators after the first year?
QOF did not generate positive spill-overs to other activities
& appears to have had a negative impact on non-inc ones
for otherwise inc conditions.
Focus on the activities for which incentives applied when
managing particular conditions?
Performed particular activities preferentially for patients for
whom rewards applied?
Kontopantelis The QOF impact

23. UK pay-for-performance scheme
Methods
Results
Conclusions
Something goes around something but
that's as far as I've got...
Comments and questions:
e.kontopantelis@manchester.ac.uk
Kontopantelis The QOF impact