Multi-Site Benchmark Classification of Major Depressive Disorder Using Machine Learning on Cortical and Subcortical Measures

Machine learning (ML) techniques have gained popularity in the neuroimaging field due to their potential for classifying neuropsychiatric disorders. However, the diagnostic predictive power of the existing algorithms has been limited by small sample sizes, lack of representativeness, data leakage, and/or overfitting. Here, we overcome these limitations with the largest multi-site sample size to date (N = 5365) to provide a generalizable ML classification benchmark of major depressive disorder (MDD) using shallow linear and non-linear models. Leveraging brain measures from standardized ENIGMA analysis pipelines in FreeSurfer, we were able to classify MDD versus healthy controls (HC) with a balanced accuracy of around 62%. But after harmonizing the data, e.g., using ComBat, the balanced accuracy dropped to approximately 52%. Accuracy results close to random chance levels were also observed in stratified groups according to age of onset, antidepressant use, number of episodes and sex. Future studies incorporating higher dimensional brain imaging/phenotype features, and/or using more advanced machine and deep learning methods may yield more encouraging prospects

DenseNet and Support Vector Machine classifications of major depressive disorder using vertex-wise cortical features

Major depressive disorder (MDD) is a complex psychiatric disorder that affects the lives of hundreds of millions of individuals around the globe. Even today, researchers debate if morphological alterations in the brain are linked to MDD, likely due to the heterogeneity of this disorder. The application of deep learning tools to neuroimaging data, capable of capturing complex non-linear patterns, has the potential to provide diagnostic and predictive biomarkers for MDD. However, previous attempts to demarcate MDD patients and healthy controls (HC) based on segmented cortical features via linear machine learning approaches have reported low accuracies. In this study, we used globally representative data from the ENIGMA-MDD working group containing an extensive sample of people with MDD (N=2,772) and HC (N=4,240), which allows a comprehensive analysis with generalizable results. Based on the hypothesis that integration of vertex-wise cortical features can improve classification performance, we evaluated the classification of a DenseNet and a Support Vector Machine (SVM), with the expectation that the former would outperform the latter. As we analyzed a multi-site sample, we additionally applied the ComBat harmonization tool to remove potential nuisance effects of site. We found that both classifiers exhibited close to chance performance (balanced accuracy DenseNet: 51%; SVM: 53%), when estimated on unseen sites. Slightly higher classification performance (balanced accuracy DenseNet: 58%; SVM: 55%) was found when the cross-validation folds contained subjects from all sites, indicating site effect. In conclusion, the integration of vertex-wise morphometric features and the use of the non-linear classifier did not lead to the differentiability between MDD and HC. Our results support the notion that MDD classification on this combination of features and classifiers is unfeasible

Multi-site benchmark classification of major depressive disorder using machine learning on cortical and subcortical measures

Machine learning (ML) techniques have gained popularity in the neuroimaging field due to their potential for classifying neuropsychiatric disorders. However, the diagnostic predictive power of the existing algorithms has been limited by small sample sizes, lack of representativeness, data leakage, and/or overfitting. Here, we overcome these limitations with the largest multi-site sample size to date (N = 5365) to provide a generalizable ML classification benchmark of major depressive disorder (MDD) using shallow linear and non-linear models. Leveraging brain measures from standardized ENIGMA analysis pipelines in FreeSurfer, we were able to classify MDD versus healthy controls (HC) with a balanced accuracy of around 62%. But after harmonizing the data, e.g., using ComBat, the balanced accuracy dropped to approximately 52%. Accuracy results close to random chance levels were also observed in stratified groups according to age of onset, antidepressant use, number of episodes and sex. Future studies incorporating higher dimensional brain imaging/phenotype features, and/or using more advanced machine and deep learning methods may yield more encouraging prospects

A large-scale ENIGMA multisite replication study of brain age in depression

International audienceBackgroundSeveral studies have evaluated whether depressed persons have older appearing brains than their nondepressed peers. However, the estimated neuroimaging-derived “brain age gap” has varied from study to study, likely driven by differences in training and testing sample (size), age range, and used modality/features. To validate our previously developed ENIGMA brain age model and the identified brain age gap, we aim to replicate the presence and effect size estimate previously found in the largest study in depression to date (N = 2126 controls & N = 2675 cases; +1.08 years [SE 0.22], Cohen's d = 0.14, 95% CI: 0.08–0.20), in independent cohorts that were not part of the original study.MethodsA previously trained brain age model (www.photon-ai.com/enigma_brainage) based on 77 FreeSurfer brain regions of interest was used to obtain unbiased brain age predictions in 751 controls and 766 persons with depression (18–75 years) from 13 new cohorts collected from 20 different scanners. Meta-regressions were used to examine potential moderating effects of basic cohort characteristics (e.g., clinical and scan technical) on the brain age gap.ResultsOur ENIGMA MDD brain age model generalized reasonably well to controls from the new cohorts (predicted age vs. age: r = 0.73, R2 = 0.47, MAE = 7.50 years), although the performance varied from cohort to cohort. In these new cohorts, on average, depressed persons showed a significantly higher brain age gap of +1 year (SE 0.35) (Cohen's d = 0.15, 95% CI: 0.05–0.25) compared with controls, highly similar to our previous finding. Significant moderating effects of FreeSurfer version 6.0 (d = 0.41, p = 0.007) and Philips scanner vendor (d = 0.50, p 3400 patients and >2800 controls worldwide show reliable but subtle effects of brain aging in adult depression. Future studies are needed to identify factors that may further explain the brain age gap variance between cohorts

Structural brain alterations associated with suicidal thoughts and behaviors in young people: results from 21 international studies from the ENIGMA Suicidal Thoughts and Behaviours consortium.

Funder: MQ Brighter Futures Award MQBFC/2 and the U.S. National Institute of Mental Health under Award Number R01MH117601. National Suicide Prevention Research Fund, managed by Suicide Prevention AustraliaFunder: MQ Brighter Futures Award MQBFC/2. Interdisziplinäres Zentrum für Klinische Forschung, UKJFunder: Italian Ministry of Health grant RC17-18-19-20-21/AFunder: Instituto de Salud Carlos III through the projects PI14/00639, PI14/00918 and PI17/01056 (Co-funded by European Regional Development Fund/European Social Fund "Investing in your future") and Fundación Instituto de Investigación Marqués de Valdecilla (NCT0235832 and NCT02534363)Funder: National Institute of Mental Health (K23MH090421), the National Alliance for Research on Schizophrenia and Depression, the University of Minnesota Graduate School, the Minnesota Medical Foundation, and the Biotechnology Research Center (P41 RR008079 to the Center for Magnetic Resonance Research), University of Minnesota, and the Deborah E. Powell Center for Women’s Health Seed Grant, University of MinnesotaFunder: Medical Leader Foundation of Yunnan Province (L2019011) and Famous Doctors Project of Yunnan Province Plan (YNWR-MY-2018-041)Funder: CJ Martin Fellowship (NHMRC app 1161356). “Investissements d’avenir” ANR-10-IAIHU-06Funder: German Research Foundation (DFG, grant FOR2107-DA1151/5-1 and DA1151/5-2 to UD, and DFG grants HA7070/2-2, HA7070/3, HA7070/4 to TH)Funder: Australian National Health and Medical Research Council of Australia (NHMRC) Project Grants 1024570 NHMRC Career Development Fellowships (1061757)Funder: Medical Faculty Münster, Innovative Medizinische Forschung (Grant IMF KO 1218 06)Funder: Australian National Medical and Health Research Council (Program Grant 1037196 and Investigator Grant 1177991 to PBM, Project Grant 1066177 to JMF), the Lansdowne Foundation, Good Talk and the Keith Pettigrew Family Bequest (PM) Janette Mary O’Neil Research Fellowship. IHG is supported in part by R37MH101495Funder: Australian National Health and Medical Research Council of Australia (NHMRC) Project Grants 1064643 (principal investigator, BJH) NHMRC Career Development Fellowships (1124472)Funder: National Institute of Mental Health (K01MH106805). Klingenstein Third Generation Foundation, the National Institute of Mental Health (K01MH117442), the Stanford Maternal Child Health Research Institute, and the Stanford Center for Cognitive and Neurobiological Imaging. TCH receives partial support from the Ray and Dagmar Dolby Family FundFunder: German Research Foundation (DFG, grant FOR2107-JA 1890/7-1 and JA 1890/7-2 to AJ, and DFG, grant FOR2107-KI588/14-1 and FOR2107-KI588/14-2 to TK)Funder: NIAAA (K01AA027573, R21AA027884) and the American Foundation for Suicide PreventionFunder: National Institute of Mental Health (R01-MH103291)Funder: National Center for Complementary and Integrative Health (NCCIH) R21AT009173 and R61AT009864 National Center for Advancing Translational Sciences (CTSI), National Institutes of Health, through UCSF-CTSI UL1TR001872 American Foundation for Suicide Prevention (AFSP) SRG-1-141-18 UCSF Research Evaluation and Allocation Committee (REAC) and J. Jacobson Fund to TTY; by the National Institute of Mental Health (NIMH) R01MH085734 and the Brain and Behavior Research Foundation (formerly NARSAD)Funder: MQ Brighter Futures Award MQBFC/2 R61MH111929RC1MH088366, R01MH070902, R01MH069747, American Foundation for Suicide Prevention, International Bipolar Foundation, Brain and Behavior Research Foundation, For the Love of Travis Foundation and Women’s Health Research at YaleFunder: MQ Brighter Futures Award MQBFC/2 Social Safety and Resilience programme of Leiden UniversityFunder: MQ Brighter Futures Award MQBFC/2 National Institute of Mental Health under Award Number R01MH117601 NHMRC Career Development Fellowship (1140764)Identifying brain alterations associated with suicidal thoughts and behaviors (STBs) in young people is critical to understanding their development and improving early intervention and prevention. The ENIGMA Suicidal Thoughts and Behaviours (ENIGMA-STB) consortium analyzed neuroimaging data harmonized across sites to examine brain morphology associated with STBs in youth. We performed analyses in three separate stages, in samples ranging from most to least homogeneous in terms of suicide assessment instrument and mental disorder. First, in a sample of 577 young people with mood disorders, in which STBs were assessed with the Columbia Suicide Severity Rating Scale (C-SSRS). Second, in a sample of young people with mood disorders, in which STB were assessed using different instruments, MRI metrics were compared among healthy controls without STBs (HC; N = 519), clinical controls with a mood disorder but without STBs (CC; N = 246) and young people with current suicidal ideation (N = 223). In separate analyses, MRI metrics were compared among HCs (N = 253), CCs (N = 217), and suicide attempters (N = 64). Third, in a larger transdiagnostic sample with various assessment instruments (HC = 606; CC = 419; Ideation = 289; HC = 253; CC = 432; Attempt=91). In the homogeneous C-SSRS sample, surface area of the frontal pole was lower in young people with mood disorders and a history of actual suicide attempts (N = 163) than those without a lifetime suicide attempt (N = 323; FDR-p = 0.035, Cohen's d = 0.34). No associations with suicidal ideation were found. When examining more heterogeneous samples, we did not observe significant associations. Lower frontal pole surface area may represent a vulnerability for a (non-interrupted and non-aborted) suicide attempt; however, more research is needed to understand the nature of its relationship to suicide risk

Concurrent Validity and Reliability of Suicide Risk Assessment Instruments: A Meta-Analysis of 20 Instruments Across 27 International Cohorts

Objective: A major limitation of current suicide research is the lack of power to identify robust correlates of suicidal thoughts or behavior. Variation in suicide risk assessment instruments used across cohorts may represent a limitation to pooling data in international consortia. Method: Here, we examine this issue through two approaches: (a) an extensive literature search on the reliability and concurrent validity of the most commonly used instruments and (b) by pooling data (N ∼ 6,000 participants) from cohorts from the Enhancing NeuroImaging Genetics Through Meta-Analysis (ENIGMA) Major Depressive Disorder and ENIGMA–Suicidal Thoughts and Behaviour working groups, to assess the concurrent validity of instruments currently used for assessing suicidal thoughts or behavior. Results: We observed moderate-to-high correlations between measures, consistent with the wide range (κ range: 0.15–0.97; r range: 0.21–0.94) reported in the literature. Two common multi-item instruments, the Columbia Suicide Severity Rating Scale and the Beck Scale for Suicidal Ideation were highly correlated with each other (r = 0.83). Sensitivity analyses identified sources of heterogeneity such as the time frame of the instrument and whether it relies on self-report or a clinical interview. Finally, construct-specific analyses suggest that suicide ideation items from common psychiatric questionnaires are most concordant with the suicide ideation construct of multi-item instruments. Conclusions: Our findings suggest that multi-item instruments provide valuable information on different aspects of suicidal thoughts or behavior but share a modest core factor with single suicidal ideation items. Retrospective, multisite collaborations including distinct instruments should be feasible provided they harmonize across instruments or focus on specific constructs of suicidality

Neurostructural subgroup in 4291 individuals with schizophrenia identified using the subtype and stage inference algorithm

Abstract Machine learning can be used to define subtypes of psychiatric conditions based on shared biological foundations of mental disorders. Here we analyzed cross-sectional brain images from 4,222 individuals with schizophrenia and 7038 healthy subjects pooled across 41 international cohorts from the ENIGMA, non-ENIGMA cohorts and public datasets. Using the Subtype and Stage Inference (SuStaIn) algorithm, we identify two distinct neurostructural subgroups by mapping the spatial and temporal ‘trajectory’ of gray matter change in schizophrenia. Subgroup 1 was characterized by an early cortical-predominant loss with enlarged striatum, whereas subgroup 2 displayed an early subcortical-predominant loss in the hippocampus, striatum and other subcortical regions. We confirmed the reproducibility of the two neurostructural subtypes across various sample sites, including Europe, North America and East Asia. This imaging-based taxonomy holds the potential to identify individuals with shared neurobiological attributes, thereby suggesting the viability of redefining existing disorder constructs based on biological factors