Data-adaptive doubly robust instrumental variable methods for treatment effect heterogeneity

We consider the estimation of the average treatment effect in the treated as a function of baseline covariates, where there is a valid (conditional) instrument. We describe two doubly-robust (DR) estimators: a g-estimator and a targeted minimum loss-based estimator (TMLE). These estimators can be viewed as generalisations of the two-stage least squares (TSLS) method to semiparametric models that make weaker assumptions. We exploit recent theoretical results and use data-adaptive estimation of the nuisance parameters for the g-estimator. A simulation study is used to compare standard TSLS with the two DR estimators’ finite-sample performance when using (1) parametric or (2) data-adaptive estimation of the nuisance parameters. Data-adaptive DR estimators have lower bias and improved coverage, when compared to incorrectly specified parametric DR estimators and TSLS. When the parametric model for the treatment effect curve is correctly specified, the g-estimator outperforms all others, but when this model is misspecified, TMLE performs best, while TSLS can result in large biases and zero coverage. The methods are also applied to the COPERS (COping with persistent Pain, Effectiveness Research in Selfmanagement) trial to make inferences about the causal effect of treatment actually received, and the extent to which this is modified by depression at baseline

Overview of parametric survival analysis for health-economic applications.

Health economic models rely on data from trials to project the risk of events (e.g., death) over time beyond the span of the available data. Parametric survival analysis methods can be applied to identify an appropriate statistical model for the observed data, which can then be extrapolated to derive a complete time-to-event curve. This paper describes the properties of the most commonly used statistical distributions as a basis for these models and describes an objective process of identifying the most suitable parametric distribution in a given dataset. The approach can be applied with both individual-patient data as well as with survival probabilities derived from published Kaplan-Meier curves. Both are illustrated with analyses of overall survival from the Sorafenib Hepatocellular Carcinoma Assessment Randomised Protocol trial

Machine Learning for Staggered Difference-in-Differences and Dynamic Treatment Effect Heterogeneity

We combine two recently proposed nonparametric difference-in-differences methods, extending them to enable the examination of treatment effect heterogeneity in the staggered adoption setting using machine learning. The proposed method, machine learning difference-in-differences (MLDID), allows for estimation of time-varying conditional average treatment effects on the treated, which can be used to conduct detailed inference on drivers of treatment effect heterogeneity. We perform simulations to evaluate the performance of MLDID and find that it accurately identifies the true predictors of treatment effect heterogeneity. We then use MLDID to evaluate the heterogeneous impacts of Brazil's Family Health Program on infant mortality, and find those in poverty and urban locations experienced the impact of the policy more quickly than other subgroups

Who benefits from health insurance? : Uncovering heterogeneous policy impacts using causal machine learning.

To be able to target health policies more efficiently, policymakers require knowledge about which individuals benefit most from a particular programme. While traditional approaches for subgroup analyses are constrained only to consider a small number of arbitrarily set, pre-defined subgroups, recently proposed causal machine learning (CML) approaches help explore treatment-effect heterogeneity in a more flexible yet principled way. This paper illustrates one such approach – ‘causal forests’ – in evaluating the effect of mothers’ health insurance enrolment in Indonesia. Contrasting two health insurance schemes (subsidised and contributory) to no insurance, we find beneficial average impacts of enrolment in contributory health insurance on maternal health care utilisation and infant mortality. For subsidised health insurance, however, both effects were smaller and not statistically significant. The causal forest algorithm identified significant heterogeneity in the impacts of the contributory insurance scheme: disadvantaged mothers (i.e. with lower wealth quintiles, lower educated, or in rural areas) benefit the most in terms of increased health care utilisation. No significant heterogeneity was found for the subsidised scheme, even though this programme targeted vulnerable populations. Our study demonstrates the power of CML approaches to uncover the heterogeneity in programme impacts, hence providing policymakers with valuable information for programme design

Data-adaptive doubly robust instrumental variable methods for treatment effect heterogeneity

We consider the estimation of the average treatment effect in the treated as a function of baseline covariates, where there is a valid (conditional) instrument. We describe two doubly-robust (DR) estimators: a g-estimator and a targeted minimum loss-based estimator (TMLE). These estimators can be viewed as generalisations of the two-stage least squares (TSLS) method to semiparametric models that make weaker assumptions. We exploit recent theoretical results and use data-adaptive estimation of the nuisance parameters for the g-estimator. A simulation study is used to compare standard TSLS with the two DR estimators’ finite-sample performance when using (1) parametric or (2) data-adaptive estimation of the nuisance parameters. Data-adaptive DR estimators have lower bias and improved coverage, when compared to incorrectly specified parametric DR estimators and TSLS. When the parametric model for the treatment effect curve is correctly specified, the g-estimator outperforms all others, but when this model is misspecified, TMLE performs best, while TSLS can result in large biases and zero coverage. The methods are also applied to the COPERS (COping with persistent Pain, Effectiveness Research in Selfmanagement) trial to make inferences about the causal effect of treatment actually received, and the extent to which this is modified by depression at baseline.We consider the estimation of the average treatment effect in the treated as a function of baseline covariates, where there is a valid (conditional) instrument. We describe two doubly-robust (DR) estimators: a g-estimator and a targeted minimum loss-based estimator (TMLE). These estimators can be viewed as generalisations of the two-stage least squares (TSLS) method to semiparametric models that make weaker assumptions. We exploit recent theoretical results and use data-adaptive estimation of the nuisance parameters for the g-estimator. A simulation study is used to compare standard TSLS with the two DR estimators’ finite-sample performance when using (1) parametric or (2) data-adaptive estimation of the nuisance parameters. Data-adaptive DR estimators have lower bias and improved coverage, when compared to incorrectly specified parametric DR estimators and TSLS. When the parametric model for the treatment effect curve is correctly specified, the g-estimator outperforms all others, but when this model is misspecified, TMLE performs best, while TSLS can result in large biases and zero coverage. The methods are also applied to the COPERS (COping with persistent Pain, Effectiveness Research in Selfmanagement) trial to make inferences about the causal effect of treatment actually received, and the extent to which this is modified by depression at baseline

Estimating the comparative effectiveness of feeding interventions in the paediatric intensive care unit : a demonstration of longitudinal targeted maximum likelihood estimation

Longitudinal data sources offer new opportunities for the evaluation of sequential interventions. To adjust for time-dependent confounding in these settings, longitudinal targeted maximum likelihood based estimation (TMLE), a doubly robust method that can be coupled with machine learning, has been proposed. This paper provides a tutorial in applying longitudinal TMLE, in contrast to inverse probability of treatment weighting and g-computation based on iterative conditional expectations. We apply these methods to estimate the causal effect of nutritional interventions on clinical outcomes among critically ill children in a United Kingdom study (Control of Hyperglycemia in Paediatric Intensive Care, 2008–2011). We estimate the probability of a child’s being discharged alive from the pediatric intensive care unit by a given day, under a range of static and dynamic feeding regimes. We find that before adjustment, patients who follow the static regime “never feed” are discharged by the end of the fifth day with a probability of 0.88 (95% confidence interval: 0.87, 0.90), while for the patients who follow the regime “feed from day 3,” the probability of discharge is 0.64 (95% confidence interval: 0.62, 0.66). After adjustment for time-dependent confounding, most of this difference disappears, and the statistical methods produce similar results. TMLE offers a flexible estimation approach; hence, we provide practical guidance on implementation to encourage its wider use

A comparison of methods for health policy evaluation with controlled pre-post designs.

OBJECTIVE: To compare interactive fixed effects (IFE) and generalized synthetic control (GSC) methods to methods prevalent in health policy evaluation and re-evaluate the impact of the hip fracture best practice tariffs introduced for hospitals in England in 2010. DATA SOURCES: Simulations and Hospital Episode Statistics. STUDY DESIGN: Best practice tariffs aimed to incentivize providers to deliver care in line with guidelines. Under the scheme, 62 providers received an additional payment for each hip fracture admission, while 49 providers did not. We estimate the impact using difference-in-differences (DiD), synthetic control (SC), IFE, and GSC methods. We contrast the estimation methods' performance in a Monte Carlo simulation study. PRINCIPAL FINDINGS: Unlike DiD, SC, and IFE methods, the GSC method provided reliable estimates across a range of simulation scenarios and was preferred for this case study. The introduction of best practice tariffs led to a 5.9 (confidence interval: 2.0 to 9.9) percentage point increase in the proportion of patients having surgery within 48 hours and a statistically insignificant 0.6 (confidence interval: -1.4 to 0.4) percentage point reduction in 30-day mortality. CONCLUSIONS: The GSC approach is an attractive method for health policy evaluation. We cannot be confident that best practice tariffs were effective

A framework for conducting economic evaluations alongside natural experiments

Internationally, policy makers are increasingly focussed on reducing the detrimental consequences and rising costs associated with unhealthy diets, inactivity, smoking, alcohol and other risk factors on the health of their populations. This has led to an increase in the demand for evidence-based, cost-effective Population Health Interventions (PHIs) to reverse this trend. Given that research designs such as randomised controlled trials (RCTs) are often not suited to the evaluation of PHIs, Natural Experiments (NEs) are now frequently being used as a design to evaluate such complex, preventive PHIs. However, current guidance for economic evaluation focusses on RCT designs and therefore does not address the specific challenges of NE designs. Using such guidance can lead to sub-optimal design, data collection and analysis for NEs, leading to bias in the estimated effectiveness and cost-effectiveness of the PHI. As a consequence, there is a growing recognition of the need to identify a robust methodological framework for the design and conducting of economic evaluations alongside such NEs. This paper outlines the challenges inherent to the design and conduct of economic evaluations of PHIs alongside NEs, providing a comprehensive framework and outlining a research agenda in this area