Confound-leakage: confound removal in machine learning leads to leakage

BACKGROUND: Machine learning (ML) approaches are a crucial component of modern data analysis in many fields, including epidemiology and medicine. Nonlinear ML methods often achieve accurate predictions, for instance, in personalized medicine, as they are capable of modeling complex relationships between features and the target. Problematically, ML models and their predictions can be biased by confounding information present in the features. To remove this spurious signal, researchers often employ featurewise linear confound regression (CR). While this is considered a standard approach for dealing with confounding, possible pitfalls of using CR in ML pipelines are not fully understood. RESULTS: We provide new evidence that, contrary to general expectations, linear confound regression can increase the risk of confounding when combined with nonlinear ML approaches. Using a simple framework that uses the target as a confound, we show that information leaked via CR can increase null or moderate effects to near-perfect prediction. By shuffling the features, we provide evidence that this increase is indeed due to confound-leakage and not due to revealing of information. We then demonstrate the danger of confound-leakage in a real-world clinical application where the accuracy of predicting attention-deficit/hyperactivity disorder is overestimated using speech-derived features when using depression as a confound. CONCLUSIONS: Mishandling or even amplifying confounding effects when building ML models due to confound-leakage, as shown, can lead to untrustworthy, biased, and unfair predictions. Our expose of the confound-leakage pitfall and provided guidelines for dealing with it can help create more robust and trustworthy ML models

Use of routinely collected health data in randomised clinical trials: comparison of trial-specific death data in the BOSS trial with NHS Digital data

Background: A promising approach to reduce the increasing costs of clinical trials is the use of routinely collected health data as participant data. However, the quality of this data could limit its usability as trial participant data. Methods: The BOSS trial is a randomised controlled trial comparing regular endoscopies versus endoscopies at need in patients with Barrett’s oesophagus with primary endpoint death. Data on death and cancer collected every 2 years after randomisation (trial-specific data) were compared to data received annually (all patients on one date) from the routinely collected health data source National Health Service (NHS) Digital. We investigated completeness, agreement and timeliness and looked at the implications for the primary trial outcome. Completeness and agreement were assessed by evaluating the number of reported and missing cases and any disparities between reported dates. Timeliness was considered by graphing the year a death was first reported in the trial-specific data against that for NHS Digital data. Implications on the primary trial outcome, overall survival, of using one of the data sources alone were investigated using Kaplan-Meier graphs. To assess the utility of cause of death and cancer diagnoses, oesophageal cancer cases were compared. Results: NHS Digital datasets included more deaths and often reported them sooner than the trial-specific data. The number reported as being from oesophageal cancer was similar in both datasets. Due to time lag in reporting and missing cases, the event rate appeared higher using the NHS Digital data. Conclusion: NHS Digital death data is useful for calculating overall survival where trial-specific follow-up is only every 2 years from randomisation and the follow-up requires patient response. The cancer data was not a large enough sample to assess usability. We suggest that this assessment of registry data is done for more phase III RCTs and for more registry data to get a more complete picture of when RCHD would be useful in phase III RCT. Trial registration: ISRCTN54190466 (BOSS) 1 Oct 2009

Practicalities in running early-phase trials using the time-to-event continual reassessment method (TiTE-CRM) for interventions with long toxicity periods using two radiotherapy oncology trials as examples

BACKGROUND: Awareness of model-based designs for dose-finding studies such as the Continual Reassessment Method (CRM) is now becoming more commonplace amongst clinicians, statisticians and trial management staff. In some settings toxicities can occur a long time after treatment has finished, resulting in extremely long, interrupted, CRM design trials. The Time-to-Event CRM (TiTE-CRM), a modification to the original CRM, accounts for the timing of late-onset toxicities and results in shorter trial duration. In this article, we discuss how to design and deliver a trial using this method, from the grant application stage through to dissemination, using two radiotherapy trials as examples. METHODS: The TiTE-CRM encapsulates the dose-toxicity relationship with a statistical model. The model incorporates observed toxicities and uses a weight to account for the proportion of completed follow-up of participants without toxicity. This model uses all available data to determine the next participant's dose and subsequently declare the maximum tolerated dose. We focus on two trials designed by the authors to illustrate practical issues when designing, setting up, and running such studies. RESULTS: In setting up a TiTE-CRM trial, model parameters need to be defined and the time element involved might cause complications, therefore looking at operating characteristics through simulations is essential. At the grant application stage, we suggest resources to fund statisticians' time before funding is awarded and make recommendations for the level of detail to include in funding applications. While running the trial, close contact of all involved staff is required as a dose decision is made each time a participant is recruited. We suggest ways of capturing data in a timely manner and give example code in R for design and delivery of the trial. Finally, we touch upon dissemination issues while the trial is running and upon completion. CONCLUSION: Model-based designs can be complex. We hope this paper will help clinical trial teams to demystify the conduct of TiTE-CRM trials and be a starting point for using this methodology in practice

Why are feasibility studies accessing routinely collected health data? A systematic review

BACKGROUND: Feasibility trials are often undertaken to determine whether a larger randomised controlled trial (RCT) is achievable. In a recent review, 15 feasibility trials accessed routinely collected health data (RCHD) from UK national databases and registries. This paper looks at attributes of these trials and the reasons why they accessed RCHD. METHODS: We extracted data from all publicly available sources for the 15 feasibility studies found in a previous review of trials successfully accessing RCHD in the UK between 2013–2018 for the purpose of informing or supplementing participant data. We extracted trial characteristics, the registry accessed, and the way the RCHD was used. RESULTS: The 15 feasibility RCTs were conducted in a variety of disease areas, and were generally small (median sample size 100, range 41–4061) and individually randomised (60%, 9/15). The primary trial outcome was predominantly administrative (non-clinical) (80%, 12/15) such as feasibility of patient recruitment. They were more likely to recruit from secondary care (67%, 10/15) settings than primary (33%, 5/15). NHS Digital was the most commonly accessed registry (33% (5/15)) with SAIL databank (20% (3/15)), electronic Data Research and Innovation Service (eDRIS) and Paediatric Intensive Care Audit Network (PICANET) (each 13% 2/15) also being accessed. Where the information was clear, the trials used RCHD for data collection during the trial (47%, 7/15), follow-up after the trial (27%, 4/15) and recruitment (13%, 2/15). CONCLUSIONS: Between 2013 and 2018, 15 feasibility trials successfully accessed UK RCHD. Feasibility trials would benefit, as with other trials, from guidance on reporting the use of RCHD in protocols and publications

e-Consent in UK academic-led clinical trials: current practice, challenges and the need for more evidence

BACKGROUND: During the COVID-19 pandemic, in-person healthcare visits were reduced. Consequently, trial teams needed to consider implementing remote methods for conducting clinical trials, including e-Consent. Although some clinical trials may have implemented e-Consent prior to the pandemic, anecdotes of uptake for this method increased within academic-led trials. When the increased use of this process emerged, representatives from several large academic clinical trial groups within the UK collaborated to discuss ways in which trialists can learn from one another when implementing e-Consent. METHODS: A survey of UKCRC-registered Clinical Trials Units (CTUs) was undertaken in April–June 2021 to understand the implementation of and their views on the use of e-Consent and experiences from the perspectives of systems programmers and quality assurance staff on the use of e-Consent. CTUs not using e-Consent were asked to provide any reasons/barriers (including no suitable trials) and any plans for implementing it in the future. Two events for trialists and patient and public involvement (PPI) representatives were then held to disseminate findings, foster discussion, share experiences and aid in the identification of areas that the academic CTU community felt required more research. RESULTS: Thirty-four (64%) of 53 CTUs responded to the survey, with good geographical representation across the UK. Twenty-one (62%) of the responding CTUs had implemented e-Consent in at least one of their trials, across different types of trials, including CTIMPs (Clinical Trial of Investigational Medicinal Product), ATIMPs (Advanced Therapy Medicinal Products) and non-CTIMPs. One hundred ninety-seven participants attended the two workshops for wide-ranging discussions. CONCLUSION: e-Consent is increasingly used in academic-led trials, yet uncertainties remain amongst trialists, patients and members of the public. Uncertainties include a lack of formal, practical guidance and a lack of evidence to demonstrate optimal or appropriate methods to use. We strongly encourage trialists to continue to share their own experiences of the implementation of e-Consent

The emerging structure of the Extended Evolutionary Synthesis: where does Evo-Devo fit in?

The Extended Evolutionary Synthesis (EES) debate is gaining ground in contemporary evolutionary biology. In parallel, a number of philosophical standpoints have emerged in an attempt to clarify what exactly is represented by the EES. For Massimo Pigliucci, we are in the wake of the newest instantiation of a persisting Kuhnian paradigm; in contrast, Telmo Pievani has contended that the transition to an EES could be best represented as a progressive reformation of a prior Lakatosian scientific research program, with the extension of its Neo-Darwinian core and the addition of a brand-new protective belt of assumptions and auxiliary hypotheses. Here, we argue that those philosophical vantage points are not the only ways to interpret what current proposals to ‘extend’ the Modern Synthesis-derived ‘standard evolutionary theory’ (SET) entail in terms of theoretical change in evolutionary biology. We specifically propose the image of the emergent EES as a vast network of models and interweaved representations that, instantiated in diverse practices, are connected and related in multiple ways. Under that assumption, the EES could be articulated around a paraconsistent network of evolutionary theories (including some elements of the SET), as well as models, practices and representation systems of contemporary evolutionary biology, with edges and nodes that change their position and centrality as a consequence of the co-construction and stabilization of facts and historical discussions revolving around the epistemic goals of this area of the life sciences. We then critically examine the purported structure of the EES—published by Laland and collaborators in 2015—in light of our own network-based proposal. Finally, we consider which epistemic units of Evo-Devo are present or still missing from the EES, in preparation for further analyses of the topic of explanatory integration in this conceptual framework

Magnetic resonance imaging of abnormal ventricular septal motion in heart diseases: a pictorial review

The purpose of this article is to illustrate the usefulness of MR imaging in the clinical evaluation of congenital and acquired cardiac diseases characterised by ventricular septal wall motion abnormality. Recognition of the features of abnormal ventricular septal motion in MR images is important to evaluate the haemodynamic status in patients with congenital and acquired heart diseases in routine clinical practice