Confound-leakage: confound removal in machine learning leads to leakage

BACKGROUND: Machine learning (ML) approaches are a crucial component of modern data analysis in many fields, including epidemiology and medicine. Nonlinear ML methods often achieve accurate predictions, for instance, in personalized medicine, as they are capable of modeling complex relationships between features and the target. Problematically, ML models and their predictions can be biased by confounding information present in the features. To remove this spurious signal, researchers often employ featurewise linear confound regression (CR). While this is considered a standard approach for dealing with confounding, possible pitfalls of using CR in ML pipelines are not fully understood. RESULTS: We provide new evidence that, contrary to general expectations, linear confound regression can increase the risk of confounding when combined with nonlinear ML approaches. Using a simple framework that uses the target as a confound, we show that information leaked via CR can increase null or moderate effects to near-perfect prediction. By shuffling the features, we provide evidence that this increase is indeed due to confound-leakage and not due to revealing of information. We then demonstrate the danger of confound-leakage in a real-world clinical application where the accuracy of predicting attention-deficit/hyperactivity disorder is overestimated using speech-derived features when using depression as a confound. CONCLUSIONS: Mishandling or even amplifying confounding effects when building ML models due to confound-leakage, as shown, can lead to untrustworthy, biased, and unfair predictions. Our expose of the confound-leakage pitfall and provided guidelines for dealing with it can help create more robust and trustworthy ML models

Evidence for similar structural brain anomalies in youth and adult attention-deficit/hyperactivity disorder: a machine learning analysis

Attention-deficit/hyperactivity disorder (ADHD) affects 5% of children world-wide. Of these, two-thirds continue to have impairing symptoms of ADHD into adulthood. Although a large literature implicates structural brain differences of the disorder, it is not clear if adults with ADHD have similar neuroanatomical differences as those seen in children with recent reports from the large ENIGMA-ADHD consortium finding structural differences for children but not for adults. This paper uses deep learning neural network classification models to determine if there are neuroanatomical changes in the brains of children with ADHD that are also observed for adult ADHD, and vice versa. We found that structural MRI data can significantly separate ADHD from control participants for both children and adults. Consistent with the prior reports from ENIGMA-ADHD, prediction performance and effect sizes were better for the child than the adult samples. The model trained on adult samples significantly predicted ADHD in the child sample, suggesting that our model learned anatomical features that are common to ADHD in childhood and adulthood. These results support the continuity of ADHD’s brain differences from childhood to adulthood. In addition, our work demonstrates a novel use of neural network classification models to test hypotheses about developmental continuity