Statistical Methods to Address Selection Bias in Economic Evaluations that Use Patient-Level Observational Data

This thesis compares statistical methods for addressing selection bias in cost-effectiveness analyses (CEA) that use observational data. The thesis has four objectives: (1) to critically appraise currently recommended statistical methods, (2) to consider alternative statistical methods for CEA, (3) to compare propensity score (PS) approaches and Genetic Matching (GM) for estimating subgroup-effects in CEA, and (4) to compare methods that combine regression with PS approaches, for CEA. I developed a new checklist for critically appraising statistical methods for addressing selection bias in CEA, and applied it in a systematic review of published CEA. Most studies used regression or matching methods, and did not assess their underlying assumptions, such as the correct specification of the PS or the endpoint regression model. I identified methods that can make less restrictive assumptions: GM, a multivariate matching method that can directly balance covariates, double-robust (DR) methods, regression-adjusted matching, and machine learning estimation of the PS and the endpoint regression. I compared these methods across a range of typical CEA circumstances, using simulations and case studies. In the first case study, where cost-effectiveness estimates for subgroups were of interest, I found that the cost-effectiveness results differed according to the statistical approach. The accompanying simulation study found that GM was relatively robust to the misspecification of the PS, and provided the least biased and most precise estimates of cost-effectiveness for each subgroup. The second simulation study considered DR methods and regression-adjusted matching for estimating overall cost-effectiveness and found that regression-adjusted matching was relatively robust to misspecification of the PS and the regression model. The third study extended these approaches with machine learning estimation of the PS and the endpoint regression, and found that bias due to misspecification could be further reduced. This thesis concludes that those approaches that relax the assumption that the statistical model for addressing selection bias is correctly specified, can give more accurate and precise estimates of cost-effectiveness than previously recommended methods. Findings from this thesis can improve the quality of CEA that use patient-level observational data, to help future studies provide a sounder basis for policy making

Reflection on modern methods: constructing directed acyclic graphs (DAGs) with domain experts for health services research

Directed acyclic graphs (DAGs) are a useful tool to represent, in a graphical format, researchers’ assumptions about the causal structure among variables while providing a rationale for the choice of confounding variables to adjust for. With origins in the field of probabilistic graphical modelling, DAGs are yet to be widely adopted in applied health research, where causal assumptions are frequently made for the purpose of evaluating health services initiatives. In this context, there is still limited practical guidance on how to construct and use DAGs. Some progress has recently been made in terms of building DAGs based on studies from the literature, but an area that has received less attention is how to create DAGs from information provided by domain experts, an approach of particular importance when there is limited published information about the intervention under study. This approach offers the opportunity for findings to be more robust and relevant to patients, carers and the public, and more likely to inform policy and clinical practice. This article draws lessons from a stakeholder workshop involving patients, health care professionals, researchers, commissioners and representatives from industry, whose objective was to draw DAGs for a complex intervention—online consultation, i.e. written exchange between the patient and health care professional using an online system—in the context of the English National Health Service. We provide some initial, practical guidance to those interested in engaging with domain experts to develop DAGs

Impact of Provider Incentives on Quality and Value of Health Care

The use of financial incentives to improve quality in health care has become widespread. Yet evidence on the effectiveness of incentives suggests that they have generally had limited impact on the value of care and have not led to better patient outcomes. Lessons from social psychology and behavioral economics indicate that incentive programs in health care have not been effectively designed to achieve their intended impact. In the United States, Medicare's Hospital Readmission Reduction Program and Hospital Value- Based Purchasing Program, created under the Affordable Care Act (ACA), provide evidence on how variations in the design of incentive programs correspond with differences in effect. As financial incentives continue to be used as a tool to increase the value and quality of health care, improving the design of programs will be crucial to ensure their success. Expected final online publication date for the Annual Review of Public Health Volume 38 is March 20, 2017. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates

Validation of the SF-36 in patients with endometriosis.

OBJECTIVES: Endometriosis presents with significant pain as the most common symptom. Generic health measures can allow comparisons across diseases or populations. However, the Medical Outcomes Study Short Form 36 (SF-36) has not been validated for this disease. The goal of this study was to validate the SF-36 (version 2) for endometriosis. METHODS: Using data from two clinical trials (N = 252 and 198) of treatment for endometriosis, a full complement of psychometric analyses was performed. Additional instruments included a pain visual analog scale (VAS); a physician-completed questionnaire based on patient interview (modified Biberoglu and Behrman--B&B); clinical global impression of change (CGI-C); and patient satisfaction with treatment. RESULTS: Bodily pain (BP) and the Physical Component Summary Score (PCS) were correlated with the pain VAS at baseline and over time and the B&B at baseline and end of study. In addition, those who had the greatest change in BP and PCS also reported the greatest change on CGI-C and patient satisfaction with treatment. Other subscales showed smaller, but significant, correlations with change in the pain VAS, CGI-C, and patient satisfaction with treatment. CONCLUSIONS: The SF-36--particularly BP and the PCS--appears to be a valid and responsive measure for endometriosis and its treatment

Regression-adjusted matching and double-robust methods for estimating average treatment effects

Regression, propensity score (PS) and double-robust (DR) methods can reduce selection bias when estimating average treatment effects (ATEs). Economic evaluations of health care interventions exemplify complex data structures, in that the covariate–endpoint relationships tend to be highly non-linear, with highly skewed cost and health outcome endpoints. When either the regression or PS model is correct, DR methods can provide unbiased, efficient estimates of ATEs, but generally the specification of both models is unknown. Regression-adjusted matching can also protect against bias from model misspecification, but has not been compared to DR methods. This paper compares regression-adjusted matching to selected DR methods (weighted regression and augmented inverse probability of treatment weighting) as well as to regression and PS methods for addressing selection bias in cost-effectiveness analyses (CEA). We contrast the methods in a CEA of a pharmaceutical intervention, where there are extreme estimated PSs, hence unstable inverse probability of treatment (IPT) weights. The case study motivates a simulation which considers settings with functional form misspecification in the PS and endpoint regression models (e.g. cost model with log instead of identity link), stable and unstable PS weights. We find that in the realistic setting of unstable IPT weights and misspecifications to the PS and regression models, regression-adjusted matching reports less bias than DR methods. We conclude that regression-adjusted matching is a relatively robust method for estimating ATEs in applications with complex data structures exemplified by CEA

[Accepted Manuscript] Estimating causal effects: considering three alternatives to difference-in-differences estimation.

Difference-in-differences (DiD) estimators provide unbiased treatment effect estimates when, in the absence of treatment, the average outcomes for the treated and control groups would have followed parallel trends over time. This assumption is implausible in many settings. An alternative assumption is that the potential outcomes are independent of treatment status, conditional on past outcomes. This paper considers three methods that share this assumption: the synthetic control method, a lagged dependent variable (LDV) regression approach, and matching on past outcomes. Our motivating empirical study is an evaluation of a hospital pay-for-performance scheme in England, the best practice tariffs programme. The conclusions of the original DiD analysis are sensitive to the choice of approach. We conduct a Monte Carlo simulation study that investigates these methods' performance. While DiD produces unbiased estimates when the parallel trends assumption holds, the alternative approaches provide less biased estimates of treatment effects when it is violated. In these cases, the LDV approach produces the most efficient and least biased estimates

Methods for estimating subgroup effects in cost-effectiveness analyses that use observational data

Decision makers require cost-effectiveness estimates for patient subgroups. In nonrandomized studies, propensity score (PS) matching and inverse probability of treatment weighting (IPTW) can address overt selection bias, but only if they balance observed covariates between treatment groups. Genetic matching (GM) matches on the PS and individual covariates using an automated search algorithm to directly balance baseline covariates. This article compares these methods for estimating subgroup effects in cost-effectiveness analyses (CEA). The motivating case study is a CEA of a pharmaceutical intervention, drotrecogin alfa (DrotAA), for patient subgroups with severe sepsis (n = 2726). Here, GM reported better covariate balance than PS matching and IPTW. For the subgroup at a high level of baseline risk, the probability that DrotAA was cost-effective ranged from 30% (IPTW) to 90% (PS matching and GM), at a threshold of £20 000 per quality-adjusted life-year. We then compared the methods in a simulation study, in which initially the PS was correctly specified and then misspecified, for example, by ignoring the subgroup-specific treatment assignment. Relative performance was assessed as bias and root mean squared error (RMSE) in the estimated incremental net benefits. When the PS was correctly specified and inverse probability weights were stable, each method performed well; IPTW reported the lowest RMSE. When the subgroup-specific treatment assignment was ignored, PS matching and IPTW reported covariate imbalance and bias; GM reported better balance, less bias, and more precise estimates. We conclude that if the PS is correctly specified and the weights for IPTW are stable, each method can provide unbiased cost-effectiveness estimates. However, unlike IPTW and PS matching, GM is relatively robust to PS misspecification