Systematic Overestimation of Machine Learning Performance in Neuroimaging Studies of Depression

We currently observe a disconcerting phenomenon in machine learning studies in psychiatry: While we would expect larger samples to yield better results due to the availability of more data, larger machine learning studies consistently show much weaker performance than the numerous small-scale studies. Here, we systematically investigated this effect focusing on one of the most heavily studied questions in the field, namely the classification of patients suffering from Major Depressive Disorder (MDD) and healthy controls. Drawing upon a balanced sample of $N = 1,868$ MDD patients and healthy controls from our recent international Predictive Analytics Competition (PAC), we first trained and tested a classification model on the full dataset which yielded an accuracy of 61%. Next, we mimicked the process by which researchers would draw samples of various sizes ($N=4$ to $N=150$) from the population and showed a strong risk of overestimation. Specifically, for small sample sizes ($N=20$), we observe accuracies of up to 95%. For medium sample sizes ($N=100$) accuracies up to 75% were found. Importantly, further investigation showed that sufficiently large test sets effectively protect against performance overestimation whereas larger datasets per se do not. While these results question the validity of a substantial part of the current literature, we outline the relatively low-cost remedy of larger test sets

Effects of polygenic risk for major mental disorders and cross-disorder on cortical complexity

Background MRI-derived cortical folding measures are an indicator of largely genetically driven early developmental processes. However, the effects of genetic risk for major mental disorders on early brain development are not well understood. Methods We extracted cortical complexity values from structural MRI data of 580 healthy participants using the CAT12 toolbox. Polygenic risk scores (PRS) for schizophrenia, bipolar disorder, major depression, and cross-disorder (incorporating cumulative genetic risk for depression, schizophrenia, bipolar disorder, autism spectrum disorder, and attention-deficit hyperactivity disorder) were computed and used in separate general linear models with cortical complexity as the regressand. In brain regions that showed a significant association between polygenic risk for mental disorders and cortical complexity, volume of interest (VOI)/region of interest (ROI) analyses were conducted to investigate additional changes in their volume and cortical thickness. Results The PRS for depression was associated with cortical complexity in the right orbitofrontal cortex (right hemisphere: p = 0.006). A subsequent VOI/ROI analysis showed no association between polygenic risk for depression and either grey matter volume or cortical thickness. We found no associations between cortical complexity and polygenic risk for either schizophrenia, bipolar disorder or psychiatric cross-disorder when correcting for multiple testing. Conclusions Changes in cortical complexity associated with polygenic risk for depression might facilitate well-established volume changes in orbitofrontal cortices in depression. Despite the absence of psychopathology, changed cortical complexity that parallels polygenic risk for depression might also change reward systems, which are also structurally affected in patients with depressive syndrome

Severity of current depression and remission status are associated with structural connectome alterations in major depressive disorder

Major depressive disorder (MDD) is associated to affected brain wiring. Little is known whether these changes are stable over time and hence might represent a biological predisposition, or whether these are state markers of current disease severity and recovery after a depressive episode. Human white matter network ("connectome") analysis via network science is a suitable tool to investigate the association between affected brain connectivity and MDD. This study examines structural connectome topology in 464 MDD patients (mean age: 36.6 years) and 432 healthy controls (35.6 years). MDD patients were stratified categorially by current disease status (acute vs. partial remission vs. full remission) based on DSM-IV criteria. Current symptom severity was assessed continuously via the Hamilton Depression Rating Scale (HAMD). Connectome matrices were created via a combination of T1-weighted magnetic resonance imaging (MRI) and tractography methods based on diffusion-weighted imaging. Global tract-based metrics were not found to show significant differences between disease status groups, suggesting conserved global brain connectivity in MDD. In contrast, reduced global fractional anisotropy (FA) was observed specifically in acute depressed patients compared to fully remitted patients and healthy controls. Within the MDD patients, FA in a subnetwork including frontal, temporal, insular, and parietal nodes was negatively associated with HAMD, an effect remaining when correcting for lifetime disease severity. Therefore, our findings provide new evidence of MDD to be associated with structural, yet dynamic, state-dependent connectome alterations, which covary with current disease severity and remission status after a depressive episode

Volume of the human hippocampus and clinical response following electroconvulsive therapy

BACKGROUND: Hippocampal enlargements are commonly reported after electroconvulsive therapy (ECT). To clarify mechanisms, we examined if ECT-induced hippocampal volume change relates to dose (number of ECT sessions and electrode placement) and acts as a biomarker of clinical outcome. METHODS: Longitudinal neuroimaging and clinical data from 10 independent sites participating in the Global ECT-Magnetic Resonance Imaging Research Collaboration (GEMRIC) were obtained for mega-analysis. Hippocampal volumes were extracted from structural magnetic resonance images, acquired before and after patients (n = 281) experiencing a major depressive episode completed an ECT treatment series using right unilateral and bilateral stimulation. Untreated nondepressed control subjects (n = 95) were scanned twice. RESULTS: The linear component of hippocampal volume change was 0.28% (SE 0.08) per ECT session (p < .001). Volume change varied by electrode placement in the left hippocampus (bilateral, 3.3 +/- 2.2%, d = 1.5; right unilateral, 1.6 +/- 2.1%, d = 0.8; p < .0001) but not the right hippocampus (bilateral, 3.0 +/- 1.7%, d = 1.8; right unilateral, 2.7 +/- 2.0%, d = 1.4; p = .36). Volume change for electrode placement per ECT session varied similarly by hemisphere. Individuals with greater treatment-related volume increases had poorer outcomes (Montgomery-Asberg Depression Rating Scale change -1.0 [SE 0.35], per 1% volume increase, p = .005), although the effects were not significant after controlling for ECT number (slope -0.69 [SE 0.38], p = .069). CONCLUSIONS: The number of ECT sessions and electrode placement impacts the extent and laterality of hippocampal enlargement, but volume change is not positively associated with clinical outcome. The results suggest that the high efficacy of ECT is not explained by hippocampal enlargement, which alone might not serve as a viable biomarker for treatment outcome

In vivo hippocampal subfield volumes in bipolar disorder—A mega-analysis from The Enhancing Neuro Imaging Genetics through Meta-Analysis Bipolar Disorder Working Group

The hippocampus consists of anatomically and functionally distinct subfields that may be differentially involved in the pathophysiology of bipolar disorder (BD). Here we, the Enhancing NeuroImaging Genetics through Meta‐Analysis Bipolar Disorder workinggroup, study hippocampal subfield volumetry in BD. T1‐weighted magnetic resonance imaging scans from 4,698 individuals (BD = 1,472, healthy controls [HC] = 3,226) from 23 sites worldwide were processed with FreeSurfer. We used linear mixed‐effects models and mega‐analysis to investigate differences in hippocampal subfield volumes between BD and HC, followed by analyses of clinical characteristics and medication use. BD showed significantly smaller volumes of the whole hippocampus (Cohen's d = −0.20), cornu ammonis (CA)1 (d = −0.18), CA2/3 (d = −0.11), CA4 (d = −0.19), molecular layer (d = −0.21), granule cell layer of dentate gyrus (d = −0.21), hippocampal tail (d = −0.10), subiculum (d = −0.15), presubiculum (d = −0.18), and hippocampal amygdala transition area (d = −0.17) compared to HC. Lithium users did not show volume differences compared to HC, while non‐users did. Antipsychotics or antiepileptic use was associated with smaller volumes. In this largest study of hippocampal subfields in BD to date, we show widespread reductions in nine of 12 subfields studied. The associations were modulated by medication use and specifically the lack of differences between lithium users and HC supports a possible protective role of lithium in BD

Apolipoprotein E Homozygous ε4 Allele Status: A Deteriorating Effect on Visuospatial Working Memory and Global Brain Structure

Theoretical background: The Apolipoprotein E (APOE) ε4 genotype is known to be one of the strongest single-gene predictors for Alzheimer disease, which is characterized by widespread brain structural degeneration progressing along with cognitive impairment. The ε4 allele status has been associated with brain structural alterations and lower cognitive ability in non-demented subjects. However, it remains unclear to what extent the visuospatial cognitive domain is affected, from what age onward changes are detectable and if alterations may interact with cognitive deficits in major depressive disorder (MDD). The current work investigated the effect of APOE ε4 homozygosity on visuospatial working memory (vWM) capacity, and on hippocampal morphometry. Furthermore, potential moderating roles of age and MDD were assessed.Methods: A sample of n = 31 homozygous ε4 carriers was contrasted with n = 31 non-ε4 carriers in a cross-sectional design. The sample consisted of non-demented, young to mid-age participants (mean age = 34.47; SD = 13.48; 51.6% female). Among them were n = 12 homozygous ε4 carriers and n = 12 non-ε4 carriers suffering from MDD (39%). VWM was assessed using the Corsi block-tapping task. Region of interest analyses of hippocampal gray matter density and volume were conducted using voxel-based morphometry (CAT12), and Freesurfer, respectively.Results: Homozygous ε4 carriers showed significantly lower Corsi span capacity than non-ε4 carriers did, and Corsi span capacity was associated with higher gray matter density of the hippocampus. APOE group differences in hippocampal volume could be detected but were no longer present when controlling for total intracranial volume. Hippocampal gray matter density did not differ between APOE groups. We did not find any interaction effects of age and MDD diagnosis on hippocampal morphometry.Conclusion: Our results point toward a negative association of homozygous ε4 allele status with vWM capacity already during mid-adulthood, which emerges independently of MDD diagnosis and age. APOE genotype seems to be associated with global brain structural rather than hippocampus specific alterations in young- to mid-age participants

Electroconvulsive therapy and cognitive performance from the Global ECT MRI Research Collaboration

The Global ECT MRI Research Collaboration (GEMRIC) has collected clinical and neuroimaging data of patients treated with electroconvulsive therapy (ECT) from around the world. Results to date have focused on neuroimaging correlates of antidepressant response. GEMRIC sites have also collected longitudinal cognitive data. Here, we summarize the existing GEMRIC cognitive data and provide recommendations for prospective data collection for future ECT-imaging investigations. We describe the criteria for selection of cognitive measures for mega-analyses: Trail Making Test Parts A (TMT-A) and B (TMT-B), verbal fluency category (VFC), verbal fluency letter (VFL), and percent retention from verbal learning and memory tests. We performed longitudinal data analysis focused on the pre-/post-ECT assessments with healthy comparison (HC) subjects at similar timepoints and assessed associations between demographic and ECT parameters with cognitive changes. The study found an interaction between electrode placement and treatment number for VFC (F(1,107) = 4.14, p = 0.04). Higher treatment was associated with decreased VFC performance with right unilateral electrode placement. Percent retention showed a main effect for group, with post-hoc analysis indicating decreased cognitive performance among the HC group. However, there were no significant effects of group or group interactions observed for TMT-A, TMT-B, or VFL. We assessed the current GEMRIC cognitive data and acknowledge the limitations associated with this data set including the limited number of neuropsychological domains assessed. Aside from the VFC and treatment number relationship, we did not observe ECT-mediated neurocognitive effects in this investigation. We provide prospective cognitive recommendations for future ECT-imaging investigations focused on strong psychometrics and minimal burden to subjects

The genetic architecture of the human cerebral cortex

The cerebral cortex underlies our complex cognitive capabilities, yet little is known about the specific genetic loci that influence human cortical structure. To identify genetic variants that affect cortical structure, we conducted a genome-wide association meta-analysis of brain magnetic resonance imaging data from 51,665 individuals. We analyzed the surface area and average thickness of the whole cortex and 34 regions with known functional specializations. We identified 199 significant loci and found significant enrichment for loci influencing total surface area within regulatory elements that are active during prenatal cortical development, supporting the radial unit hypothesis. Loci that affect regional surface area cluster near genes in Wnt signaling pathways, which influence progenitor expansion and areal identity. Variation in cortical structure is genetically correlated with cognitive function, Parkinson's disease, insomnia, depression, neuroticism, and attention deficit hyperactivity disorder

Electroconvulsive therapy-induced volumetric brain changes converge on a common causal circuit in depression

Neurostimulation is a mainstream treatment option for major depression. Neuromodulation techniques apply repetitive magnetic or electrical stimulation to some neural target but significantly differ in their invasiveness, spatial selectivity, mechanism of action, and efficacy. Despite these differences, recent analyses of transcranial magnetic stimulation (TMS) and deep brain stimulation (DBS)-treated individuals converged on a common neural network that might have a causal role in treatment response. We set out to investigate if the neuronal underpinnings of electroconvulsive therapy (ECT) are similarly associated with this causal depression network (CDN). Our aim here is to provide a comprehensive analysis in three cohorts of patients segregated by electrode placement (N = 246 with right unilateral, 79 with bitemporal, and 61 with mixed) who underwent ECT. We conducted a data-driven, unsupervised multivariate neuroimaging analysis Principal Component Analysis (PCA) of the cortical and subcortical volume changes and electric field (EF) distribution to explore changes within the CDN associated with antidepressant outcomes. Despite the different treatment modalities (ECT vs TMS and DBS) and methodological approaches (structural vs functional networks), we found a highly similar pattern of change within the CDN in the three cohorts of patients (spatial similarity across 85 regions: r = 0.65, 0.58, 0.40, df = 83). Most importantly, the expression of this pattern correlated with clinical outcomes (t = -2.35, p = 0.019). This evidence further supports that treatment interventions converge on a CDN in depression. Optimizing modulation of this network could serve to improve the outcome of neurostimulation in depression

The genetic architecture of the human cerebral cortex

The cerebral cortex underlies our complex cognitive capabilities, yet little is known about the specific genetic loci that influence human cortical structure. To identify genetic variants that affect cortical structure, we conducted a genome-wide association meta-analysis of brain magnetic resonance imaging data from 51,665 individuals. We analyzed the surface area and average thickness of the whole cortex and 34 regions with known functional specializations. We identified 199 significant loci and found significant enrichment for loci influencing total surface area within regulatory elements that are active during prenatal cortical development, supporting the radial unit hypothesis. Loci that affect regional surface area cluster near genes in Wnt signaling pathways, which influence progenitor expansion and areal identity. Variation in cortical structure is genetically correlated with cognitive function, Parkinson's disease, insomnia, depression, neuroticism, and attention deficit hyperactivity disorder