Has COVID-19 been the making of Open Science?

One outcome of the COVID-19 pandemic has been to put discussions about open research methods and practices, such as preprints, into the mainstream. Drawing on an recent analysis of the extent to which Open Science principles have been adopted during the COVID-19 pandemic, Lonni Besançon, Corentin Segalas, Clémence Leyrat, argue that while the pandemic has accelerated certain forms of Open Science, much work remains to be done to ensure that these principles are engaged with optimally

MatchThem:: Matching and Weighting after Multiple Imputation

Balancing the distributions of the confounders across the exposure levels in an observational study through matching or weighting is an accepted method to control for confounding due to these variables when estimating the association between an exposure and outcome and to reduce the degree of dependence on certain modeling assumptions. Despite the increasing popularity in practice, these procedures cannot be immediately applied to datasets with missing values. Multiple imputation of the missing data is a popular approach to account for missing values while preserving the number of units in the dataset and accounting for the uncertainty in the missing values. However, to the best of our knowledge, there is no comprehensive matching and weighting software that can be easily implemented with multiply imputed datasets. In this paper, we review this problem and suggest a framework to map out the matching and weighting multiply imputed datasets to 5 actions as well as the best practices to assess balance in these datasets after matching and weighting. We also illustrate these approaches using a companion package for R, MatchThem.Comment: 23 Pages, 3 Figure

Propensity score to detect baseline imbalance in cluster randomized trials: the role of the c-statistic.

BACKGROUND: Despite randomization, baseline imbalance and confounding bias may occur in cluster randomized trials (CRTs). Covariate imbalance may jeopardize the validity of statistical inferences if they occur on prognostic factors. Thus, the diagnosis of a such imbalance is essential to adjust statistical analysis if required. METHODS: We developed a tool based on the c-statistic of the propensity score (PS) model to detect global baseline covariate imbalance in CRTs and assess the risk of confounding bias. We performed a simulation study to assess the performance of the proposed tool and applied this method to analyze the data from 2 published CRTs. RESULTS: The proposed method had good performance for large sample sizes (n =500 per arm) and when the number of unbalanced covariates was not too small as compared with the total number of baseline covariates (≥40% of unbalanced covariates). We also provide a strategy for pre selection of the covariates needed to be included in the PS model to enhance imbalance detection. CONCLUSION: The proposed tool could be useful in deciding whether covariate adjustment is required before performing statistical analyses of CRTs

Immortal-time bias in older vs younger age groups: a simulation study with application to a population-based cohort of patients with colon cancer

BACKGROUND: In observational studies, the risk of immortal-time bias (ITB) increases with the likelihood of early death, itself increasing with age. We investigated how age impacts the magnitude of ITB when estimating the effect of surgery on 1-year overall survival (OS) in patients with Stage IV colon cancer aged 50–74 and 75–84 in England. METHODS: Using simulations, we compared estimates from a time-fixed exposure model to three statistical methods addressing ITB: time-varying exposure, delayed entry and landmark methods. We then estimated the effect of surgery on OS using a population-based cohort of patients from the CORECT-R resource and conducted the analysis using the emulated target trial framework. RESULTS: In simulations, the magnitude of ITB was larger among older patients when their probability of early death increased or treatment was delayed. The bias was corrected using the methods addressing ITB. When applied to CORECT-R data, these methods yielded a smaller effect of surgery than the time-fixed exposure approach but effects were similar in both age groups. CONCLUSION: ITB must be addressed in all longitudinal studies, particularly, when investigating the effect of exposure on an outcome in different groups of people (e.g., age groups) with different distributions of exposure and outcomes

Common Methods for Handling Missing Data in Marginal Structural Models: What Works and Why.

Marginal structural models (MSMs) are commonly used to estimate causal intervention effects in longitudinal nonrandomized studies. A common challenge when using MSMs to analyze observational studies is incomplete confounder data, where a poorly informed analysis method will lead to biased estimates of intervention effects. Despite a number of approaches described in the literature for handling missing data in MSMs, there is little guidance on what works in practice and why. We reviewed existing missing-data methods for MSMs and discussed the plausibility of their underlying assumptions. We also performed realistic simulations to quantify the bias of 5 methods used in practice: complete-case analysis, last observation carried forward, the missingness pattern approach, multiple imputation, and inverse-probability-of-missingness weighting. We considered 3 mechanisms for nonmonotone missing data encountered in research based on electronic health record data. Further illustration of the strengths and limitations of these analysis methods is provided through an application using a cohort of persons with sleep apnea: the research database of the French Observatoire Sommeil de la Fédération de Pneumologie. We recommend careful consideration of 1) the reasons for missingness, 2) whether missingness modifies the existing relationships among observed data, and 3) the scientific context and data source, to inform the choice of the appropriate method(s) for handling partially observed confounders in MSMs

Applying the estimands framework to non-inferiority trials: guidance on choice of hypothetical estimands for non-adherence and comparison of estimation methods

A common concern in non-inferiority (NI) trials is that non adherence due, for example, to poor study conduct can make treatment arms artificially similar. Because intention to treat analyses can be anti-conservative in this situation, per protocol analyses are sometimes recommended. However, such advice does not consider the estimands framework, nor the risk of bias from per protocol analyses. We therefore sought to update the above guidance using the estimands framework, and compare estimators to improve on the performance of per protocol analyses. We argue the main threat to validity of NI trials is the occurrence of trial specific intercurrent events (IEs), that is, IEs which occur in a trial setting, but would not occur in practice. To guard against erroneous conclusions of non inferiority, we suggest an estimand using a hypothetical strategy for trial specific IEs should be employed, with handling of other non trial specific IEs chosen based on clinical considerations. We provide an overview of estimators that could be used to estimate a hypothetical estimand, including inverse probability weighting (IPW), and two instrumental variable approaches (one using an informative Bayesian prior on the effect of standard treatment, and one using a treatment by covariate interaction as an instrument). We compare them, using simulation in the setting of all or nothing compliance in two active treatment arms, and conclude both IPW and the instrumental variable method using a Bayesian prior are potentially useful approaches, with the choice between them depending on which assumptions are most plausible for a given trial

Does internet-accessed STI (e-STI) testing increase testing uptake for chlamydia and other STIs among a young population who have never tested? Secondary analyses of data from a randomised controlled trial.

OBJECTIVES: To assess the effectiveness of an internet-accessed STI (e-STI) testing and results service on testing uptake among young adults (16-30 years) who have never tested for STIs in London, England. METHODS: We conducted secondary analyses on data from a randomised controlled trial. In the trial, participants were randomly allocated to receive a text message with the web link of an e-STI testing and results service (intervention group) or a text message with the link of a website listing the locations, contact details and websites of seven local sexual health clinics (control group). We analysed a subsample of 528 trial participants who reported never testing for STIs at baseline. Outcomes were self-reported STI testing at 6 weeks, verified by patient record checks, and time from randomisation to completion of an STI test. RESULTS: Uptake of STI testing among 'never testers' almost doubled. At 6 weeks, 45.3% of the intervention completed at least one test (chlamydia, gonorrhoea, syphilis and HIV), compared with 24.1% of the control (relative risk [RR] 1.88, 95% CI 1.47 to 2.40, p<0.001). For chlamydia and gonorrhoea testing combined, uptake was 44.3% in the intervention versus 24.1% in controls (RR 1.84, 95% CI 1.44 to 2.36, p<0.001). The intervention reduced time to any STI test (restricted mean survival time: 29.0 days vs 36.3 days, p<0.001) at a time horizon of 42 days. CONCLUSIONS : e-STI testing increased uptake of STI testing and reduced time to test among a young population of 'never testers' recruited in community settings. Although encouraging, questions remain on how best to manage the additional demand generated by e-STI testing in a challenging funding environment. Larger studies are required to assess the effects later in the cascade of care, including STI diagnoses and cases treated

Peer Review of Grant Applications: A Simple Method to Identify Proposals with Discordant Reviews

Grant proposals submitted for funding are usually selected by a peer-review rating process. Some proposals may result in discordant peer-review ratings and therefore require discussion by the selection committee members. The issue is which peer-review ratings are considered as discordant. We propose a simple method to identify such proposals. Our approach is based on the intraclass correlation coefficient, which is usually used in assessing agreement in studies with continuous ratings

G-computation and doubly robust standardisation for continuous-time data: A comparison with inverse probability weighting.

In time-to-event settings, g-computation and doubly robust estimators are based on discrete-time data. However, many biological processes are evolving continuously over time. In this paper, we extend the g-computation and the doubly robust standardisation procedures to a continuous-time context. We compare their performance to the well-known inverse-probability-weighting estimator for the estimation of the hazard ratio and restricted mean survival times difference, using a simulation study. Under a correct model specification, all methods are unbiased, but g-computation and the doubly robust standardisation are more efficient than inverse-probability-weighting. We also analyse two real-world datasets to illustrate the practical implementation of these approaches. We have updated the R package RISCA to facilitate the use of these methods and their dissemination

Reflection on modern methods: trial emulation in the presence of immortal-time bias. Assessing the benefit of major surgery for elderly lung cancer patients using observational data.

Acquiring real-world evidence is crucial to support health policy, but observational studies are prone to serious biases. An approach was recently proposed to overcome confounding and immortal-time biases within the emulated trial framework. This tutorial provides a step-by-step description of the design and analysis of emulated trials, as well as R and Stata code, to facilitate its use in practice. The steps consist in: (i) specifying the target trial and inclusion criteria; (ii) cloning patients; (iii) defining censoring and survival times; (iv) estimating the weights to account for informative censoring introduced by design; and (v) analysing these data. These steps are illustrated with observational data to assess the benefit of surgery among 70-89-year-old patients diagnosed with early-stage lung cancer. Because of the severe unbalance of the patient characteristics between treatment arms (surgery yes/no), a naïve Kaplan-Meier survival analysis of the initial cohort severely overestimated the benefit of surgery on 1-year survival (22% difference), as did a survival analysis of the cloned dataset when informative censoring was ignored (17% difference). By contrast, the estimated weights adequately removed the covariate imbalance. The weighted analysis still showed evidence of a benefit, though smaller (11% difference), of surgery among older lung cancer patients on 1-year survival. Complementing the CERBOT tool, this tutorial explains how to proceed to conduct emulated trials using observational data in the presence of immortal-time bias. The strength of this approach is its transparency and its principles that are easily understandable by non-specialists