Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

**Scribd will begin operating the SlideShare business on December 1, 2020**
As of this date, Scribd will manage your SlideShare account and any content you may have on SlideShare, and Scribd's General Terms of Use and Privacy Policy will apply. If you wish to opt out, please close your SlideShare account. Learn more.

Successfully reported this slideshow.

Like this presentation? Why not share!

496 views

Published on

No Downloads

Total views

496

On SlideShare

0

From Embeds

0

Number of Embeds

96

Shares

0

Downloads

20

Comments

4

Likes

1

No embeds

No notes for slide

- 1. Introduction Methodology Simulations Application Conclusion Robust Methods for Health-related Quality-of-life Assessment Ian McCarthy Baylor Scott & White Health Center for Clinical Eﬀectiveness Utah Health Services Research Conference April 30, 2014 This project was supported by grant number K99HS022431 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the author and does not necessarily represent the oﬃcial views of the Agency for Healthcare Research and Quality. Robust Methods for Health-related Quality-of-life Assessment
- 2. Introduction Methodology Simulations Application Conclusion Background Cost- and comparative-eﬀectiveness studies becoming increasingly important Require assessment of health-related quality-of-life (HRQoL) outcomes and quality-adjusted life-years (QALYs) Common approach ﬁrst collapses the multi-dimensional HRQoL proﬁle into a one-dimensional QALY (Drummond et al., 2005; Brazier et al., 2002; Brazier & Ratcliﬀe, 2007) EQ-5D SF-6D HUI Robust Methods for Health-related Quality-of-life Assessment
- 3. Introduction Methodology Simulations Application Conclusion Problem Loss of information when reducing HRQoL proﬁle into QALY, with potentially biased and inconsistent marginal eﬀects estimates (Mortimer & Segal, 2008; Devlin et al., 2010; Parkin et al., 2010; Gutacker et al., 2012): 1 Floor and ceiling eﬀects not present in the underlying domains but imposed by the scoring algorithm. 2 Nonlinearities in the relationship between the outcome and independent variables which are diﬃcult to approximate using the summary score. Robust Methods for Health-related Quality-of-life Assessment
- 4. Introduction Methodology Simulations Application Conclusion Current Study 1 Monte Carlo study showing the bias of the estimated coeﬃcients when relying solely on QALYs or some other summary score based on several ordered outcome variables. 2 Propose new two-step methodology that ﬁrst estimates coeﬃcients in each HRQoL domain and then transforms the coeﬃcients and marginal eﬀects into the QALY domain based on predicted values from the ﬁrst-stage regressions. Robust Methods for Health-related Quality-of-life Assessment
- 5. Introduction Methodology Simulations Application Conclusion Estimating QALYs Marginal Eﬀects: Standard Approach Marginal Eﬀects: Proposed Methodology The SF-6D Developed by John Brazier and other, the SF-6D is formed from a subset of questions from the SF-36 or SF-12 and is a common HRQoL outcome intended to provide a general measure of a patient’s health status (Brazier et al., 2002; Brazier & Ratcliﬀe, 2007). Six dimensions/domains of health: (Physical functioning, role limitations, social functioning, pain, mental health, and vitality) Each domain characterized numerically with a range of integers. Best value is 1, and worst value ranges from 4 to 6. Scoring algorithm developed in Brazier et al. (2002) and Brazier & Ratcliﬀe (2007) for calculating a population-based index score from the SF-6D questionnaire Robust Methods for Health-related Quality-of-life Assessment
- 6. Introduction Methodology Simulations Application Conclusion Estimating QALYs Marginal Eﬀects: Standard Approach Marginal Eﬀects: Proposed Methodology Scoring the SF-6D Physical Functioning (PF) PF=2 or PF=3 -0.035 PF=4 -0.044 PF=5 -0.056 PF=6 -0.117 Role Limitations (RL) RL=2 or RL=3 or RL=4 -0.053 Social Functioning (SF) SF=2 -0.057 SF=3 -0.059 SF=4 -0.072 SF=5 -0.087 Pain (P) P=2 or P=3 -0.042 P=4 -0.065 P=5 -0.102 P=6 -0.171 Mental Health (MH) MH=2 or MH=3 -0.042 MH=4 -0.100 MH=5 -0.118 Vitality (V) V=2 or V=3 or V=4 -0.071 V=5 -0.092 Combination of Domains “Most Severe” -0.061 Robust Methods for Health-related Quality-of-life Assessment
- 7. Introduction Methodology Simulations Application Conclusion Estimating QALYs Marginal Eﬀects: Standard Approach Marginal Eﬀects: Proposed Methodology Focus on QALYs By far the most common methodology for estimating coeﬃcients and ultimately marginal eﬀects is to ﬁrst reduce the multi-dimensional health proﬁle to a one-dimensional QALY (Austin et al., 2000; Austin, 2002; Richardson & Manca, 2004; Manca et al., 2005; Basu & Manca, 2012) Recent literature on how best to accommodate distributional features somewhat speciﬁc to QALYs (Austin, 2002; Basu & Manca, 2012), including a censored least absolute deviation model and a Beta MLE approach Robust Methods for Health-related Quality-of-life Assessment
- 8. Introduction Methodology Simulations Application Conclusion Estimating QALYs Marginal Eﬀects: Standard Approach Marginal Eﬀects: Proposed Methodology First Stage Regression 1 Estimate an ordered probit model separately for each domain, d = 1, ..., 6, with the follow-up HRQoL response (yid,t1 ) modeled as a function of person-speciﬁc variables (xi ), baseline HRQoL response (yid,t0 ), and treatment status (Ti ). 2 Form predicted probabilities of every possible response, j, in each domain, d, denoted ˆpd j . The regression results provide a predicted (marginal) probability for each of 31 possible outcomes for each person. Robust Methods for Health-related Quality-of-life Assessment
- 9. Introduction Methodology Simulations Application Conclusion Estimating QALYs Marginal Eﬀects: Standard Approach Marginal Eﬀects: Proposed Methodology “Most Severe” Category 1 Deﬁned as any one of the following (Brazier et al., 2002): 4 or more in the physical functioning, social functioning, mental health, or vitality domains; 3 or more in the role limitation domain; or 5 or more in the pain domain 2 Since the probabilities, Pd ij , are potentially correlated across domains, the probability of a “most severe” health status can be calculated following the principle of inclusion and exclusion for probability: P (A1 ∪ A2 ∪ ... ∪ AN) = P (A1) + ... + P (AN) + N n=2 (−1)n+1 P (∩ n events) . Robust Methods for Health-related Quality-of-life Assessment
- 10. Introduction Methodology Simulations Application Conclusion Estimating QALYs Marginal Eﬀects: Standard Approach Marginal Eﬀects: Proposed Methodology Estimate QALYs QALY i = 1 − 0.035 × ˆPPF i2 + ˆPPF i3 − 0.044 × ˆPPF i4 − 0.056 × ˆPPF i5 − 0.117 × ˆPPF i6 − 0.053 × ˆPRL i2 + ˆPRL i3 + ˆPRL i4 − 0.057 × ˆPSF i2 − 0.059 × ˆPSF i3 − 0.072 × ˆPSF i4 − 0.087 × ˆPSF i5 − 0.042 × ˆPPain i2 + ˆPPain i3 − 0.065 × ˆPPain i4 − 0.102 × ˆPPain i5 − 0.171 × ˆPPain i6 − 0.042 × ˆPMH i2 + ˆPMH i3 − 0.100 × ˆPMH i4 − 0.118 × ˆPMH i5 − 0.071 × ˆPV i2 + ˆPV i3 + ˆPV i4 − 0.092 × ˆPV i5 − 0.061 × ˆP (Most Severe) . Robust Methods for Health-related Quality-of-life Assessment
- 11. Introduction Methodology Simulations Application Conclusion Marginal Eﬀects on QALYs Treatment Eﬀects with Selection Data Generating Processes The D × 1 vector of latent HRQoL values, y∗ i , is simulated as follows: y∗ i = γ + βxi + εi , where ε ∼ N (0D×1, ID×D) , x ∼ U[0, 1], γ = ID×1, and β = 1.5 × ID×1. Discrete HRQoL values are generated based on the value of the latent value, y∗ id , relative to the Jd × 1 vector of threshold values in each domain. Robust Methods for Health-related Quality-of-life Assessment
- 12. Introduction Methodology Simulations Application Conclusion Marginal Eﬀects on QALYs Treatment Eﬀects with Selection Simulated QALY Distributions 01020304050 Frequency .4 .6 .8 1 SF-6D Index Score 010203040 Frequency .2 .4 .6 .8 1 SF-6D Index Score 01020304050 Frequency .4 .6 .8 1 SF-6D Index Score 020406080 Frequency .3 .4 .5 .6 .7 .8 SF-6D Index Score 050100150200 Frequency .4 .6 .8 1 SF-6D Index Score Robust Methods for Health-related Quality-of-life Assessment
- 13. Introduction Methodology Simulations Application Conclusion Marginal Eﬀects on QALYs Treatment Eﬀects with Selection Monte Carlo Results Model Incremental Eﬀect St. Dev. Mean % Bias Lower % Bias Upper % Bias RMSE DGP 1: True Eﬀect 0.070 0.002 Two-stage Approach 0.070 0.003 -0.73% -11.85% 11.64% 0.0827 OLS 0.073 0.004 3.79% -8.89% 17.18% 0.0828 Beta MLE 0.077 0.004 9.49% -4.84% 25.44% 0.0830 Beta QMLE 0.075 0.004 6.27% -6.66% 19.96% 0.0829 DGP 2: True Eﬀect 0.093 0.003 Two-stage Approach 0.092 0.005 -0.64% -12.62% 11.48% 0.1041 OLS 0.089 0.005 -3.84% -15.36% 8.39% 0.1043 Beta MLE 0.142 0.010 52.57% 28.34% 76.59% 0.1115 Beta QMLE 0.102 0.006 10.14% -4.26% 25.24% 0.1043 DGP 3: True Eﬀect 0.076 0.003 Two-stage Approach 0.075 0.005 -1.34% -15.60% 15.21% 0.0916 OLS 0.065 0.004 -15.02% -29.91% -1.40% 0.0923 Beta MLE 0.075 0.008 -1.01% -23.44% 23.44% 0.0935 Beta QMLE 0.086 0.006 12.71% -5.97% 32.68% 0.0917 Robust Methods for Health-related Quality-of-life Assessment
- 14. Introduction Methodology Simulations Application Conclusion Marginal Eﬀects on QALYs Treatment Eﬀects with Selection Monte Carlo Results Model Incremental Eﬀect St. Dev. Mean % Bias Lower % Bias Upper % Bias RMSE DGP 4: True Eﬀect 0.075 0.002 Two-stage Approach 0.075 0.003 -0.22% -10.58% 11.14% 0.0966 OLS 0.083 0.004 10.32% -2.40% 24.52% 0.0968 Beta MLE 0.083 0.005 10.71% -2.67% 25.71% 0.0969 Beta QMLE 0.082 0.004 9.20% -3.23% 22.88% 0.0968 DGP 5: True Eﬀect 0.062 0.002 Two-stage Approach 0.061 0.003 -0.28% -11.20% 11.19% 0.0916 OLS 0.072 0.004 16.70% 2.21% 32.65% 0.0920 Beta MLE 0.070 0.004 13.03% -1.05% 28.53% 0.0919 Beta QMLE 0.070 0.004 13.46% -0.26% 28.56% 0.0919 Robust Methods for Health-related Quality-of-life Assessment
- 15. Introduction Methodology Simulations Application Conclusion Marginal Eﬀects on QALYs Treatment Eﬀects with Selection General Case of Selection on Observable Variables -.0050.005.01.015 DeviationfromTrueEffect 0 10 20 30 40 50 Degree of Selection -.01-.0050.005.01 DeviationfromTrueEffect 0 10 20 30 40 50 Degree of Selection OLS 2SE DGP 1: Homogeneous Eﬀects across Domains -.0050.005.01.015.02 DeviationfromTrueEffect 0 10 20 30 40 50 Degree of Selection -.01-.0050.005.01 DeviationfromTrueEffect 0 10 20 30 40 50 Degree of Selection OLS 2SE DGP 2: Heterogeneous Eﬀects across Domains Robust Methods for Health-related Quality-of-life Assessment
- 16. Introduction Methodology Simulations Application Conclusion Marginal Eﬀects on QALYs Treatment Eﬀects with Selection With Simulated Data based on the SF-6D Random Treatment Assignment Selection on Observed Variables Model Treatment Eﬀect St. Dev. RMSE Treatment Eﬀect St. Dev. RMSE DGP 1: δ = 1.5 × I6×1 True Eﬀect 0.142 0.005 0.142 0.005 2SE 0.143 0.006 0.054 0.143 0.007 0.054 OLS 0.143 0.007 0.066 0.151 0.010 0.068 Beta MLE 0.169 0.012 0.082 0.174 0.021 0.080 Beta QMLE 0.143 0.007 0.067 0.146 0.011 0.066 DGP 2: δ = 3 × I6×1 True Eﬀect 0.264 0.007 0.264 0.007 2SE 0.264 0.007 0.046 0.263 0.009 0.046 OLS 0.265 0.008 0.077 0.284 0.010 0.091 Beta MLE 0.296 0.010 0.075 0.378 0.018 0.067 Beta QMLE 0.264 0.008 0.061 0.320 0.013 0.056 DGP 3: δ = [2, 1, 0.5, 2.5, 0, 1] True Eﬀect 0.104 0.004 0.104 0.004 2SE 0.104 0.005 0.055 0.104 0.007 0.055 OLS 0.104 0.006 0.063 0.088 0.009 0.064 Beta MLE 0.117 0.012 0.087 0.083 0.023 0.083 Beta QMLE 0.104 0.006 0.070 0.079 0.011 0.070 Robust Methods for Health-related Quality-of-life Assessment
- 17. Introduction Methodology Simulations Application Conclusion Marginal Eﬀects on QALYs Treatment Eﬀects with Selection With Simulated Data based on the SF-6D Random Treatment Assignment Selection on Observed Variables Model Treatment Eﬀect St. Dev. RMSE Treatment Eﬀect St. Dev. RMSE DGP 4: interaction terms with δ = 1.5 × I6×1 True Eﬀect 0.122 0.006 0.122 0.006 2SE 0.122 0.007 0.048 0.122 0.010 0.048 OLS 0.122 0.008 0.084 0.137 0.014 0.094 Beta MLE 0.133 0.011 0.096 0.234 0.023 0.085 Beta QMLE 0.122 0.008 0.074 0.165 0.015 0.073 DGP 5: interaction terms with δ = 3 × I6×1 True Eﬀect 0.220 0.007 0.220 0.007 2SE 0.220 0.007 0.043 0.220 0.010 0.043 OLS 0.220 0.008 0.096 0.266 0.014 0.132 Beta MLE 0.231 0.011 0.081 0.332 0.022 0.080 Beta QMLE 0.220 0.008 0.068 0.272 0.015 0.065 DGP 6: interaction terms with δ = [2, 1, 0.5, 2.5, 0, 1] True Eﬀect 0.102 0.005 0.102 0.005 2SE 0.102 0.006 0.047 0.102 0.009 0.047 OLS 0.102 0.007 0.078 0.098 0.013 0.079 Beta MLE 0.114 0.012 0.109 0.210 0.024 0.090 Beta QMLE 0.102 0.008 0.081 0.137 0.015 0.081 Robust Methods for Health-related Quality-of-life Assessment
- 18. Introduction Methodology Simulations Application Conclusion Data Summary Results Data Data collected prospectively on adult scoliosis patients from over 10 participating members of the International Spine Study Group (ISSG). Variable Mean Standard Deviation Age 56.76 14.51 BMI 26.59 5.84 Baseline SF-6D 0.61 0.12 Follow-up SF-6D 0.66 0.12 Count Percent Operative 193 53% Female 309 85% Robust Methods for Health-related Quality-of-life Assessment
- 19. Introduction Methodology Simulations Application Conclusion Data Summary Results Summary Statistics Baseline Follow-up Count Percent Count Percent Physical Functioning Domain PF=1 0 0% 0 0% PF=2 35 10% 54 15% PF=3 117 32% 121 33% PF=4 96 27% 83 23% PF=5 100 28% 95 26% PF=6 14 4% 9 2% Role Limitations Domain RL=1 41 11% 53 15% RL=2 115 32% 144 40% RL=3 10 3% 11 3% RL=4 196 54% 154 42% Social Functioning Domain SF=1 110 30% 156 43% SF=2 72 20% 77 21% SF=3 99 27% 86 24% SF=4 56 15% 30 8% SF=5 25 7% 13 4% Robust Methods for Health-related Quality-of-life Assessment
- 20. Introduction Methodology Simulations Application Conclusion Data Summary Results Summary Statistics Baseline Follow-up Count Percent Count Percent Pain Domain P=1 5 1% 19 5% P=2 34 9% 47 13% P=3 79 22% 123 34% P=4 85 23% 88 24% P=5 109 30% 66 18% P=6 50 14% 19 5% Mental Health Domain MH=1 76 21% 130 36% MH=2 127 35% 132 36% MH=3 89 25% 61 17% MH=4 53 15% 32 9% MH=5 17 5% 7 2% Vitality Domain V=1 13 4% 15 4% V=2 73 20% 123 34% V=3 107 30% 108 30% V=4 94 26% 74 20% V=5 75 21% 42 12% Robust Methods for Health-related Quality-of-life Assessment
- 21. Introduction Methodology Simulations Application Conclusion Data Summary Results Average Treatment Effect of Surgery OLS Beta Beta 2SE MLE QMLE Outcome: QALY QALY QALY PF RL SF P MH V Surgery 0.03*** 0.17*** 0.15*** -0.06 -0.06 0.14 0.54*** 0.28** 0.26** (0.01) (0.05) (0.05) (0.12) (0.12) (0.12) (0.12) (0.12) (0.12) Age 0.00* 0.00 0.00* -0.00 -0.01 0.00 0.01** 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) Female -0.02 -0.10 -0.09 -0.12 -0.27 0.07 -0.09 -0.47*** -0.31* (0.01) (0.07) (0.07) (0.16) (0.17) (0.17) (0.16) (0.18) (0.17) BMI -0.00 -0.00 -0.00 -0.00 -0.02** 0.01 -0.02 -0.00 0.00 (0.00) (0.00) (0.00) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01) ATE on QALY 0.033*** 0.038*** 0.032*** 0.029*** (0.011) (0.011) (0.011) (0.010) RMSE 0.098 0.111 0.098 0.097 Robust Methods for Health-related Quality-of-life Assessment
- 22. Introduction Methodology Simulations Application Conclusion Intuition Collapsing multi-dimensional proﬁle into a single summary measure introduces ﬂoor/ceiling eﬀects and nonlinearities that are diﬃcult to accommodate in a single equation framework. With selection into treatment (whether on observables or unobservables), standard methods relying only on QALYs provide biased estimates of true treatment eﬀect. An alternative approach is to estimate coeﬃcients based on the full health proﬁle and then re-interpret eﬀects in the QALY domain based on predicted probabilities in the ﬁrst-stage regressions. Robust Methods for Health-related Quality-of-life Assessment
- 23. Introduction Methodology Simulations Application Conclusion Thank You Robust Methods for Health-related Quality-of-life Assessment
- 24. Introduction Methodology Simulations Application Conclusion Bibliography I Austin, P.C. 2002. A comparison of methods for analyzing health-related quality-of-life measures. Value in Health, 5(4), 329–337. Austin, P.C., Escobar, M., & Kopec, J.A. 2000. The use of the Tobit model for analyzing measures of health status. Quality of Life Research, 9(8), 901–910. Basu, A., & Manca, A. 2012. Regression Estimators for Generic Health-Related Quality of Life and Quality-Adjusted Life Years. Medical Decision Making, 32(1), 56–69. Brazier, J., & Ratcliﬀe, J. 2007. Measuring and valuing health beneﬁts for economic evaluation. Oxford University Press, USA. Brazier, J., Roberts, J., & Deverill, M. 2002. The estimation of a preference-based measure of health from the SF-36. Journal of health economics, 21(2), 271–292. Devlin, N.J., Parkin, D., & Browne, J. 2010. Patient-reported outcome measures in the NHS: new methods for analysing and reporting EQ-5D data. Health economics, 19(8), 886–905. Drummond, M.F., Sculpher, M.J., & Torrance, G.W. 2005. Methods for the economic evaluation of health care programmes. Oxford University Press, USA. Gutacker, N., Bojke, C., Daidone, S., Devlin, N., & Street, A. 2012. Analysing Hospital Variation in Health Outcome at the Level of EQ-5D Dimensions. Manca, A., Hawkins, N., & Sculpher, M.J. 2005. Estimating mean QALYs in trial-based cost-eﬀectiveness analysis: the importance of controlling for baseline utility. Health economics, 14(5), 487–496. Mortimer, D., & Segal, L. 2008. Comparing the incomparable? A systematic review of competing techniques for converting descriptive measures of health status into QALY-weights. Medical decision making, 28(1), 66. Parkin, D., Rice, N., & Devlin, N. 2010. Statistical analysis of EQ-5D proﬁles: does the use of value sets bias inference? Medical Decision Making, 30(5), 556–565. Richardson, G., & Manca, A. 2004. Calculation of quality adjusted life years in the published literature: a review of methodology and transparency. Health economics, 13(12), 1203–1210. Robust Methods for Health-related Quality-of-life Assessment

No public clipboards found for this slide

Login to see the comments