Systematic Overestimation of Machine Learning Performance in Neuroimaging Studies of Depression

We currently observe a disconcerting phenomenon in machine learning studies in psychiatry: While we would expect larger samples to yield better results due to the availability of more data, larger machine learning studies consistently show much weaker performance than the numerous small-scale studies. Here, we systematically investigated this effect focusing on one of the most heavily studied questions in the field, namely the classification of patients suffering from Major Depressive Disorder (MDD) and healthy controls. Drawing upon a balanced sample of $N = 1,868$ MDD patients and healthy controls from our recent international Predictive Analytics Competition (PAC), we first trained and tested a classification model on the full dataset which yielded an accuracy of 61%. Next, we mimicked the process by which researchers would draw samples of various sizes ($N=4$ to $N=150$) from the population and showed a strong risk of overestimation. Specifically, for small sample sizes ($N=20$), we observe accuracies of up to 95%. For medium sample sizes ($N=100$) accuracies up to 75% were found. Importantly, further investigation showed that sufficiently large test sets effectively protect against performance overestimation whereas larger datasets per se do not. While these results question the validity of a substantial part of the current literature, we outline the relatively low-cost remedy of larger test sets

Development and Validation of a Risk Score for Chronic Kidney Disease in HIV Infection Using Prospective Cohort Data from the D:A:D Study

Ristola M. on työryhmien DAD Study Grp ; Royal Free Hosp Clin Cohort ; INSIGHT Study Grp ; SMART Study Grp ; ESPRIT Study Grp jäsen.Background Chronic kidney disease (CKD) is a major health issue for HIV-positive individuals, associated with increased morbidity and mortality. Development and implementation of a risk score model for CKD would allow comparison of the risks and benefits of adding potentially nephrotoxic antiretrovirals to a treatment regimen and would identify those at greatest risk of CKD. The aims of this study were to develop a simple, externally validated, and widely applicable long-term risk score model for CKD in HIV-positive individuals that can guide decision making in clinical practice. Methods and Findings A total of 17,954 HIV-positive individuals from the Data Collection on Adverse Events of Anti-HIV Drugs (D:A:D) study with >= 3 estimated glomerular filtration rate (eGFR) values after 1 January 2004 were included. Baseline was defined as the first eGFR > 60 ml/min/1.73 m2 after 1 January 2004; individuals with exposure to tenofovir, atazanavir, atazanavir/ritonavir, lopinavir/ritonavir, other boosted protease inhibitors before baseline were excluded. CKD was defined as confirmed (>3 mo apart) eGFR In the D:A:D study, 641 individuals developed CKD during 103,185 person-years of follow-up (PYFU; incidence 6.2/1,000 PYFU, 95% CI 5.7-6.7; median follow-up 6.1 y, range 0.3-9.1 y). Older age, intravenous drug use, hepatitis C coinfection, lower baseline eGFR, female gender, lower CD4 count nadir, hypertension, diabetes, and cardiovascular disease (CVD) predicted CKD. The adjusted incidence rate ratios of these nine categorical variables were scaled and summed to create the risk score. The median risk score at baseline was -2 (interquartile range -4 to 2). There was a 1: 393 chance of developing CKD in the next 5 y in the low risk group (risk score = 5, 505 events), respectively. Number needed to harm (NNTH) at 5 y when starting unboosted atazanavir or lopinavir/ritonavir among those with a low risk score was 1,702 (95% CI 1,166-3,367); NNTH was 202 (95% CI 159-278) and 21 (95% CI 19-23), respectively, for those with a medium and high risk score. NNTH was 739 (95% CI 506-1462), 88 (95% CI 69-121), and 9 (95% CI 8-10) for those with a low, medium, and high risk score, respectively, starting tenofovir, atazanavir/ritonavir, or another boosted protease inhibitor. The Royal Free Hospital Clinic Cohort included 2,548 individuals, of whom 94 individuals developed CKD (3.7%) during 18,376 PYFU (median follow-up 7.4 y, range 0.3-12.7 y). Of 2,013 individuals included from the SMART/ESPRIT control arms, 32 individuals developed CKD (1.6%) during 8,452 PYFU (median follow-up 4.1 y, range 0.6-8.1 y). External validation showed that the risk score predicted well in these cohorts. Limitations of this study included limited data on race and no information on proteinuria. Conclusions Both traditional and HIV-related risk factors were predictive of CKD. These factors were used to develop a risk score for CKD in HIV infection, externally validated, that has direct clinical relevance for patients and clinicians to weigh the benefits of certain antiretrovirals against the risk of CKD and to identify those at greatest risk of CKD.Peer reviewe

Biological sex classification with structural MRI data shows increased misclassification in transgender women

Transgender individuals (TIs) show brain-structural alterations that differ from their biological sex as well as their perceived gender. To substantiate evidence that the brain structure of TIs differs from male and female, we use a combined multivariate and univariate approach. Gray matter segments resulting from voxel-based morphometry preprocessing of N = 1753 cisgender (CG) healthy participants were used to train (N = 1402) and validate (20% holdout N = 351) a support-vector machine classifying the biological sex. As a second validation, we classified N = 1104 patients with depression. A third validation was performed using the matched CG sample of the transgender women (TW) application sample. Subsequently, the classifier was applied to N = 26 TW. Finally, we compared brain volumes of CG-men, women, and TW-pre/post treatment cross-sex hormone treatment (CHT) in a univariate analysis controlling for sexual orientation, age, and total brain volume. The application of our biological sex classifier to the transgender sample resulted in a significantly lower true positive rate (TPR-male = 56.0%). The TPR did not differ between CG-individuals with (TPR-male = 86.9%) and without depression (TPR-male = 88.5%). The univariate analysis of the transgender application-sample revealed that TW-pre/post treatment show brain-structural differences from CG-women and CG-men in the putamen and insula, as well as the whole-brain analysis. Our results support the hypothesis that brain structure in TW differs from brain structure of their biological sex (male) as well as their perceived gender (female). This finding substantiates evidence that TIs show specific brain-structural alterations leading to a different pattern of brain structure than CG-individuals. (c) The Author(s), under exclusive licence to American College of Neuropsychopharmacology 202

FreeSurfer‐based segmentation of hippocampal subfields: A review of methods and applications, with a novel quality control procedure for ENIGMA studies and other collaborative efforts

Structural hippocampal abnormalities are common in many neurological and psychiatric disorders, and variation in hippocampal measures is related to cognitive performance and other complex phenotypes such as stress sensitivity. Hippocampal subregions are increasingly studied, as automated algorithms have become available for mapping and volume quantification. In the context of the Enhancing Neuro Imaging Genetics through Meta Analysis Consortium, several Disease Working Groups are using the FreeSurfer software to analyze hippocampal subregion (subfield) volumes in patients with neurological and psychiatric conditions along with data from matched controls. In this overview, we explain the algorithm's principles, summarize measurement reliability studies, and demonstrate two additional aspects (subfield autocorrelation and volume/reliability correlation) with illustrative data. We then explain the rationale for a standardized hippocampal subfield segmentation quality control (QC) procedure for improved pipeline harmonization. To guide researchers to make optimal use of the algorithm, we discuss how global size and age effects can be modeled, how QC steps can be incorporated and how subfields may be aggregated into composite volumes. This discussion is based on a synopsis of 162 published neuroimaging studies (01/2013-12/2019) that applied the FreeSurfer hippocampal subfield segmentation in a broad range of domains including cognition and healthy aging, brain development and neurodegeneration, affective disorders, psychosis, stress regulation, neurotoxicity, epilepsy, inflammatory disease, childhood adversity and posttraumatic stress disorder, and candidate and whole genome (epi-)genetics. Finally, we highlight points where FreeSurfer-based hippocampal subfield studies may be optimized

Brain structural abnormalities in obesity: relation to age, genetic risk, and common psychiatric disorders: Evidence through univariate and multivariate mega-analysis including 6420 participants from the ENIGMA MDD working group

Emerging evidence suggests that obesity impacts brain physiology at multiple levels. Here we aimed to clarify the relationship between obesity and brain structure using structural MRI (n = 6420) and genetic data (n = 3907) from the ENIGMA Major Depressive Disorder (MDD) working group. Obesity (BMI > 30) was significantly associated with cortical and subcortical abnormalities in both mass-univariate and multivariate pattern recognition analyses independent of MDD diagnosis. The most pronounced effects were found for associations between obesity and lower temporo-frontal cortical thickness (maximum Cohen´s d (left fusiform gyrus) = −0.33). The observed regional distribution and effect size of cortical thickness reductions in obesity revealed considerable similarities with corresponding patterns of lower cortical thickness in previously published studies of neuropsychiatric disorders. A higher polygenic risk score for obesity significantly correlated with lower occipital surface area. In addition, a significant age-by-obesity interaction on cortical thickness emerged driven by lower thickness in older participants. Our findings suggest a neurobiological interaction between obesity and brain structure under physiological and pathological brain conditions

Brain structural abnormalities in obesity: relation to age, genetic risk, and common psychiatric disorders : Evidence through univariate and multivariate mega-analysis including 6420 participants from the ENIGMA MDD working group

Emerging evidence suggests that obesity impacts brain physiology at multiple levels. Here we aimed to clarify the relationship between obesity and brain structure using structural MRI (n = 6420) and genetic data (n = 3907) from the ENIGMA Major Depressive Disorder (MDD) working group. Obesity (BMI > 30) was significantly associated with cortical and subcortical abnormalities in both mass-univariate and multivariate pattern recognition analyses independent of MDD diagnosis. The most pronounced effects were found for associations between obesity and lower temporo-frontal cortical thickness (maximum Cohen´s d (left fusiform gyrus) = -0.33). The observed regional distribution and effect size of cortical thickness reductions in obesity revealed considerable similarities with corresponding patterns of lower cortical thickness in previously published studies of neuropsychiatric disorders. A higher polygenic risk score for obesity significantly correlated with lower occipital surface area. In addition, a significant age-by-obesity interaction on cortical thickness emerged driven by lower thickness in older participants. Our findings suggest a neurobiological interaction between obesity and brain structure under physiological and pathological brain conditions

Connecting the study of wild influenza with the potential for pandemic disease

Continuing outbreaks of pathogenic (H5N1) and pandemic (SOIVH1N1) influenza have underscored the need to understand the origin, characteristics, and evolution of novel influenza A virus (IAV) variants that pose a threat to human health. In the last 4–5 years, focus has been placed on the organization of large-scale surveillance programs to examine the phylogenetics of avian influenza virus (AIV) and host–virus relationships in domestic and wild animals. Here we review the current gaps in wild animal and environmental surveillance and the current understanding of genetic signatures in potentially pandemic strains.National Institute of Allergy and Infectious Diseases (U.S.) (Contract HHSN266200700010C)Massachusetts Institute of Technolog