Non-compliance and missing data in health economic evaluation

Health economic evaluations face the issues of non-compliance and missing data. Here, non-compliance is defined as non-adherence to a specific treatment, and occurs within randomised controlled trials (RCTs) when participants depart from their random assignment. Missing data arises if, for example, there is loss to follow-up, survey non-response, or the information available from routine data sources is incomplete. Appropriate statistical methods for handling non-compliance and missing data have been developed, but they have rarely been applied in health economics studies. Here, we illustrate the issues and outline some of the appropriate methods to handle these with an application to a health economic evaluation that uses data from an RCT. In an RCT the random assignment can be used as an instrument for treatment receipt, to obtain consistent estimates of the complier average causal effect, provided the underlying assumptions are met. Instrumental variable methods can accommodate essential features of the health economic context such as the correlation between individuals' costs and outcomes in cost-effectiveness studies. Methodological guidance for handling missing data encourages approaches such as multiple imputation or inverse probability weighting, that assume the data are Missing At Random, but also sensitivity analyses that recognise the data may be missing according to the true, unobserved values, that is, Missing Not at Random. Future studies should subject the assumptions behind methods for handling non-compliance and missing data to thorough sensitivity analyses. Modern machine learning methods can help reduce reliance on correct model specification. Further research is required to develop flexible methods for handling more complex forms of non-compliance and missing data.Comment: 41 page

Estimating cluster-level local average treatment effects in cluster randomised trials with non-adherence

Non-adherence to assigned treatment is a common issue in cluster randomised trials (CRTs). In these settings, the efficacy estimand may be also of interest. Many methodological contributions in recent years have advocated using instrumental variables to identify and estimate the local average treatment effect (LATE). However, the clustered nature of randomisation in CRTs adds to the complexity of such analyses. In this paper, we show that under certain assumptions, the LATE can be estimated via two-stage least squares (TSLS) using cluster-level summaries of outcomes and treatment received. Implementation needs to account for this, as well as the possible heteroscedasticity, to obtain valid inferences. We use simulations to assess the performance of TSLS of cluster-level summaries under cluster-level or individual-level non-adherence, with and without weighting and robust standard errors. We also explore the impact of adjusting for cluster-level covariates and of appropriate degrees of freedom correction for inference. We find that TSLS estimation using cluster-level summaries provides estimates with small to negligible bias and coverage close to nominal level, provided small sample degrees of freedom correction is used for inference, with appropriate use of robust standard errors. We illustrate the methods by re-analysing a CRT in UK primary health settings.Comment: 21 pages, 6 Figure

Reporting non-adherence in cluster randomised trials: A systematic review.

BACKGROUND: Treatment non-adherence in randomised trials refers to situations where some participants do not receive their allocated treatment as intended. For cluster randomised trials, where the unit of randomisation is a group of participants, non-adherence may occur at the cluster or individual level. When non-adherence occurs, randomisation no longer guarantees that the relationship between treatment receipt and outcome is unconfounded, and the power to detect the treatment effects in intention-to-treat analysis may be reduced. Thus, recording adherence and estimating the causal treatment effect adequately are of interest for clinical trials. OBJECTIVES: To assess the extent of reporting of non-adherence issues in published cluster trials and to establish which methods are currently being used for addressing non-adherence, if any, and whether clustering is accounted for in these. METHODS: We systematically reviewed 132 cluster trials published in English in 2011 previously identified through a search in PubMed. RESULTS: One-hundred and twenty three cluster trials were included in this systematic review. Non-adherence was reported in 56 cluster trials. Among these, 19 reported a treatment efficacy estimate: per protocol in 15 and as treated in 4. No study discussed the assumptions made by these methods, their plausibility or the sensitivity of the results to deviations from these assumptions. LIMITATIONS: The year of publication of the cluster trials included in this review (2011) could be considered a limitation of this study; however, no new guidelines regarding the reporting and the handling of non-adherence for cluster trials have been published since. In addition, a single reviewer undertook the data extraction. To mitigate this, a second reviewer conducted a validation of the extraction process on 15 randomly selected reports. Agreement was satisfactory (93%). CONCLUSION: Despite the recommendations of the Consolidated Standards of Reporting Trials statement extension to cluster randomised trials, treatment adherence is under-reported. Among the trials providing adherence information, there was substantial variation in how adherence was defined, handled and reported. Researchers should discuss the assumptions required for the results to be interpreted causally and whether these are scientifically plausible in their studies. Sensitivity analyses to study the robustness of the results to departures from these assumptions should be performed

Data-adaptive doubly robust instrumental variable methods for treatment effect heterogeneity

We consider the estimation of the average treatment effect in the treated as a function of baseline covariates, where there is a valid (conditional) instrument. We describe two doubly-robust (DR) estimators: a g-estimator and a targeted minimum loss-based estimator (TMLE). These estimators can be viewed as generalisations of the two-stage least squares (TSLS) method to semiparametric models that make weaker assumptions. We exploit recent theoretical results and use data-adaptive estimation of the nuisance parameters for the g-estimator. A simulation study is used to compare standard TSLS with the two DR estimators’ finite-sample performance when using (1) parametric or (2) data-adaptive estimation of the nuisance parameters. Data-adaptive DR estimators have lower bias and improved coverage, when compared to incorrectly specified parametric DR estimators and TSLS. When the parametric model for the treatment effect curve is correctly specified, the g-estimator outperforms all others, but when this model is misspecified, TMLE performs best, while TSLS can result in large biases and zero coverage. The methods are also applied to the COPERS (COping with persistent Pain, Effectiveness Research in Selfmanagement) trial to make inferences about the causal effect of treatment actually received, and the extent to which this is modified by depression at baseline

Consent processes in cluster-randomised trials in residential facilities for older adults : a systematic review of reporting practices and proposed guidelines

Objective: To assess the quality of reported consent processes of cluster-randomised trials conducted in residential facilities for older people and to explore whether the focus on improving the general conduct and reporting of cluster-randomised trials influenced the quality of conduct and reporting of ethical processes in these trials. Design: Systematic review of cluster-randomised trials reports, published up to the end of 2010. Data sources: National Library of Medicine (Medline) via PubMed, hand-searches of BMJ, Journal of the American Medical Association, BMC Health Services Research, Age and Ageing and Journal of the American Geriatrics Society, reference search in Web of Knowledge and consultation with experts. Eligibility for selecting studies: Published cluster-randomised trials where the unit of randomisation is a part or the whole of a residential facility for older people, without language or year of publication restrictions. Results: We included 73 trials. Authors reported ethical approval in 59, obtaining individual consent in 51, and using proxies for this consent in 37, but the process to assess residents’ capacity to consent was clearly reported in only eight. We rated only six trials high for the quality of consent processes. We considered that individual informed consent could have been waived legitimately in 14 of 22 trials not reporting obtaining consent. The proportions reporting ethical approval and quality of consent processes were higher in recent trials. Conclusions: Recently published international recommendations regarding ethical conduct in cluster-randomised trials are much needed. In relation to consent processes when cognitively impaired individuals are included in these trials, we provide a six-point checklist and recommend the minimum information to be reported. Those who lack capacity in trials with complex designs should be afforded the same care in relation to consent as competent adults in trials with simpler designs

Who benefits from health insurance? : Uncovering heterogeneous policy impacts using causal machine learning.

To be able to target health policies more efficiently, policymakers require knowledge about which individuals benefit most from a particular programme. While traditional approaches for subgroup analyses are constrained only to consider a small number of arbitrarily set, pre-defined subgroups, recently proposed causal machine learning (CML) approaches help explore treatment-effect heterogeneity in a more flexible yet principled way. This paper illustrates one such approach – ‘causal forests’ – in evaluating the effect of mothers’ health insurance enrolment in Indonesia. Contrasting two health insurance schemes (subsidised and contributory) to no insurance, we find beneficial average impacts of enrolment in contributory health insurance on maternal health care utilisation and infant mortality. For subsidised health insurance, however, both effects were smaller and not statistically significant. The causal forest algorithm identified significant heterogeneity in the impacts of the contributory insurance scheme: disadvantaged mothers (i.e. with lower wealth quintiles, lower educated, or in rural areas) benefit the most in terms of increased health care utilisation. No significant heterogeneity was found for the subsidised scheme, even though this programme targeted vulnerable populations. Our study demonstrates the power of CML approaches to uncover the heterogeneity in programme impacts, hence providing policymakers with valuable information for programme design