Harnessing spatial homogeneity of neuroimaging data: patch individual filter layers for CNNs

Neuroimaging data, e.g. obtained from magnetic resonance imaging (MRI), is comparably homogeneous due to (1) the uniform structure of the brain and (2) additional efforts to spatially normalize the data to a standard template using linear and non-linear transformations. Convolutional neural networks (CNNs), in contrast, have been specifically designed for highly heterogeneous data, such as natural images, by sliding convolutional filters over different positions in an image. Here, we suggest a new CNN architecture that combines the idea of hierarchical abstraction in neural networks with a prior on the spatial homogeneity of neuroimaging data: Whereas early layers are trained globally using standard convolutional layers, we introduce for higher, more abstract layers patch individual filters (PIF). By learning filters in individual image regions (patches) without sharing weights, PIF layers can learn abstract features faster and with fewer samples. We thoroughly evaluated PIF layers for three different tasks and data sets, namely sex classification on UK Biobank data, Alzheimer's disease detection on ADNI data and multiple sclerosis detection on private hospital data. We demonstrate that CNNs using PIF layers result in higher accuracies, especially in low sample size settings, and need fewer training epochs for convergence. To the best of our knowledge, this is the first study which introduces a prior on brain MRI for CNN learning

Patch individual filter layers in CNNs to harness the spatial homogeneity of neuroimaging data

Convolutional neural networks (CNNs)-as a type of deep learning-have been specifically designed for highly heterogeneous data, such as natural images. Neuroimaging data, however, is comparably homogeneous due to (1) the uniform structure of the brain and (2) additional efforts to spatially normalize the data to a standard template using linear and non-linear transformations. To harness spatial homogeneity of neuroimaging data, we suggest here a new CNN architecture that combines the idea of hierarchical abstraction in CNNs with a prior on the spatial homogeneity of neuroimaging data. Whereas early layers are trained globally using standard convolutional layers, we introduce patch individual filters (PIF) for higher, more abstract layers. By learning filters in individual latent space patches without sharing weights, PIF layers can learn abstract features faster and specific to regions. We thoroughly evaluated PIF layers for three different tasks and data sets, namely sex classification on UK Biobank data, Alzheimer's disease detection on ADNI data and multiple sclerosis detection on private hospital data, and compared it with two baseline models, a standard CNN and a patch-based CNN. We obtained two main results: First, CNNs using PIF layers converge consistently faster, measured in run time in seconds and number of iterations than both baseline models. Second, both the standard CNN and the PIF model outperformed the patch-based CNN in terms of balanced accuracy and receiver operating characteristic area under the curve (ROC AUC) with a maximal balanced accuracy (ROC AUC) of 94.21% (99.10%) for the sex classification task (PIF model), and 81.24% and 80.48% (88.89% and 87.35%) respectively for the Alzheimer's disease and multiple sclerosis detection tasks (standard CNN model). In conclusion, we demonstrated that CNNs using PIF layers result in faster convergence while obtaining the same predictive performance as a standard CNN. To the best of our knowledge, this is the first study that introduces a prior in form of an inductive bias to harness spatial homogeneity of neuroimaging data

Registration of 3D fetal neurosonography and MRI.

We propose a method for registration of 3D fetal brain ultrasound with a reconstructed magnetic resonance fetal brain volume. This method, for the first time, allows the alignment of models of the fetal brain built from magnetic resonance images with 3D fetal brain ultrasound, opening possibilities to develop new, prior information based image analysis methods for 3D fetal neurosonography. The reconstructed magnetic resonance volume is first segmented using a probabilistic atlas and a pseudo ultrasound image volume is simulated from the segmentation. This pseudo ultrasound image is then affinely aligned with clinical ultrasound fetal brain volumes using a robust block-matching approach that can deal with intensity artefacts and missing features in the ultrasound images. A qualitative and quantitative evaluation demonstrates good performance of the method for our application, in comparison with other tested approaches. The intensity average of 27 ultrasound images co-aligned with the pseudo ultrasound template shows good correlation with anatomy of the fetal brain as seen in the reconstructed magnetic resonance image

The Third International Workshop on Biomedical Image Registration - WBIR 2006

Editorial

Theory-based Explorations of Associations between Human Brain Structure and Intelligence from Childhood to Early Adulthood

Intelligence is often defined as the ability of an agent to learn, adapt to its environment, and solve novel challenges. However, despite over 100 years of theoretical development (e.g., general intelligence), widespread explanatory power (up to 50% of variance in cognitive scores), and the ability of intelligence measures to predict important life outcomes such as educational achievement and mortality, the exact configuration and neural correlates of cognitive ability remain poorly understood. This dissertation aims to make progress in this pursuit by exploring how human brain structure and intelligence correlate and co-develop with each other from childhood to early adulthood (ages 5 – 22 years). This endeavour is undertaken in three large cohorts (N range: 337 – 2072), guided by theory (e.g., crystallised and fluid intelligence), and implemented using rigorous, cutting-edge quantitative methods (i.e., structural equation modelling and network science). The results of this research provide robust evidence that the brain-behaviour relationships in intelligence are complex (i.e., consists of many independent yet interacting parts) and change nonlinearly during development. The first study sought to elucidate the factorial structure and white matter substrates of child and adolescent intelligence using two cross-sectional, developmental samples (CALM: N = 551 (N = 165 neuroimaging), age range: 5 – 18 years; NKI-Rockland: N = 337 (N = 65 neuroimaging), age range: 6 – 18 years). In both samples, it was found (using structural equation modelling (SEM)) that cognitive ability is best modelled as two separable yet related constructs, crystallised and fluid intelligence, which became more distinct (i.e., less correlated) across development, in line with the age differentiation hypothesis. Further analyses revealed that white matter microstructure, most prominently of the superior longitudinal fasciculus, was strongly associated with crystallised (gc) and fluid (gf) abilities. Finally, SEM trees, which combines traditional SEM with decision trees, provided evidence for developmental reorganisation of gc and gf and their white matter substrates such that the relationships among these factors dropped between ages 7 – 8 years before increasing around age 10. Together, these results suggested that shortly before puberty marks a pivotal phase of change in the neurocognitive architecture of intelligence. The second study builds upon the first by again examining the neurocognitive structure of intelligence, this time from a network perspective. The network or mutualism theory of intelligence presupposes direct (statistical) interactions among cognitive abilities (e.g., maths, memory, and vocabulary) throughout development. Therefore, this project used network analytic methods (specifically graphical LASSO) to simultaneously model brain-behaviour relationships essential for general intelligence in a large (behavioural, N = 805; cortical volume, N = 246; fractional anisotropy, N = 165), developmental (ages 5 – 18 years) cohort of struggling learners (CALM). Results indicated that both the single-layer (cognitive or neural nodes) and multilayer (combined cognitive and neural variables) networks consisted of mostly positive, small partial correlations, providing further support for the mutualism/network theory of cognitive ability. Moreover, using community detection (i.e., the Walktrap algorithm) and calculating node centrality (absolute strength and bridge strength), convergent evidence suggested that subsets of both cognitive and neural nodes play an intermediary role ‘between’ brain and behaviour. Overall, these findings suggest specific behavioural and neural variables that may have greater influence among (or might be more influenced by) other nodes within general intelligence. The final study investigated the longitudinal relationships between human cortical grey matter structure and measures of decision-making, risk-related behaviours, and spatial working memory from adolescence to early adulthood (ages 14 – 22 years). In the IMAGEN study (maximum N across time points/waves = 2072), latent growth curve models were used to estimate the baseline and longitudinal associations between behavioural measures and cortical surface area, thickness, and volume. Univariate models (only behavioural or neural measures) revealed that performance in decision-making, risk-related behaviours, and spatial working memory, as well as brain structure changed nonlinearly from mid-adolescence (age 14) to early adulthood (age 22). Furthermore, bivariate models (combined behavioural and neural measures) provided evidence for adaptive reorganisation (behaviour intercept predicts changes in brain structure) but not structural scaffolding (brain structure intercept predicts changes in behaviour). Furthermore, findings suggested that there were no correlated changes between behavioural and brain structure slopes (rates of change from mid-adolescence to early adulthood). This dissertation concludes by summarising the core results, addressing key limitations, and discussing avenues for future research. Taken together, this thesis hopes to convince cognitive neuroscientists that to understand cognitive ability and its neural determinants, they (we) must work more diligently toward building coherent, rigorous, and testable neurocognitive theories of intelligence—particularly under the conceptual and analytic paradigm of complex systems.The Cambridge Commonwealth, European & International Trus

Explainable deep learning classifiers for disease detection based on structural brain MRI data

In dieser Doktorarbeit wird die Frage untersucht, wie erfolgreich deep learning bei der Diagnostik von neurodegenerativen Erkrankungen unterstützen kann. In 5 experimentellen Studien wird die Anwendung von Convolutional Neural Networks (CNNs) auf Daten der Magnetresonanztomographie (MRT) untersucht. Ein Schwerpunkt wird dabei auf die Erklärbarkeit der eigentlich intransparenten Modelle gelegt. Mit Hilfe von Methoden der erklärbaren künstlichen Intelligenz (KI) werden Heatmaps erstellt, die die Relevanz einzelner Bildbereiche für das Modell darstellen. Die 5 Studien dieser Dissertation zeigen das Potenzial von CNNs zur Krankheitserkennung auf neurologischen MRT, insbesondere bei der Kombination mit Methoden der erklärbaren KI. Mehrere Herausforderungen wurden in den Studien aufgezeigt und Lösungsansätze in den Experimenten evaluiert. Über alle Studien hinweg haben CNNs gute Klassifikationsgenauigkeiten erzielt und konnten durch den Vergleich von Heatmaps zur klinischen Literatur validiert werden. Weiterhin wurde eine neue CNN Architektur entwickelt, spezialisiert auf die räumlichen Eigenschaften von Gehirn MRT Bildern.Deep learning and especially convolutional neural networks (CNNs) have a high potential of being implemented into clinical decision support software for tasks such as diagnosis and prediction of disease courses. This thesis has studied the application of CNNs on structural MRI data for diagnosing neurological diseases. Specifically, multiple sclerosis and Alzheimer’s disease were used as classification targets due to their high prevalence, data availability and apparent biomarkers in structural MRI data. The classification task is challenging since pathology can be highly individual and difficult for human experts to detect and due to small sample sizes, which are caused by the high acquisition cost and sensitivity of medical imaging data. A roadblock in adopting CNNs to clinical practice is their lack of interpretability. Therefore, after optimizing the machine learning models for predictive performance (e.g. balanced accuracy), we have employed explainability methods to study the reliability and validity of the trained models. The deep learning models achieved good predictive performance of over 87% balanced accuracy on all tasks and the explainability heatmaps showed coherence with known clinical biomarkers for both disorders. Explainability methods were compared quantitatively using brain atlases and shortcomings regarding their robustness were revealed. Further investigations showed clear benefits of transfer-learning and image registration on the model performance. Lastly, a new CNN layer type was introduced, which incorporates a prior on the spatial homogeneity of neuro-MRI data. CNNs excel when used on natural images which possess spatial heterogeneity, and even though MRI data and natural images share computational similarities, the composition and orientation of neuro-MRI is very distinct. The introduced patch-individual filter (PIF) layer breaks the assumption of spatial invariance of CNNs and reduces convergence time on different data sets without reducing predictive performance. The presented work highlights many challenges that CNNs for disease diagnosis face on MRI data and defines as well as tests strategies to overcome those