One-shot Joint Extraction, Registration and Segmentation of Neuroimaging Data

Brain extraction, registration and segmentation are indispensable preprocessing steps in neuroimaging studies. The aim is to extract the brain from raw imaging scans (i.e., extraction step), align it with a target brain image (i.e., registration step) and label the anatomical brain regions (i.e., segmentation step). Conventional studies typically focus on developing separate methods for the extraction, registration and segmentation tasks in a supervised setting. The performance of these methods is largely contingent on the quantity of training samples and the extent of visual inspections carried out by experts for error correction. Nevertheless, collecting voxel-level labels and performing manual quality control on high-dimensional neuroimages (e.g., 3D MRI) are expensive and time-consuming in many medical studies. In this paper, we study the problem of one-shot joint extraction, registration and segmentation in neuroimaging data, which exploits only one labeled template image (a.k.a. atlas) and a few unlabeled raw images for training. We propose a unified end-to-end framework, called JERS, to jointly optimize the extraction, registration and segmentation tasks, allowing feedback among them. Specifically, we use a group of extraction, registration and segmentation modules to learn the extraction mask, transformation and segmentation mask, where modules are interconnected and mutually reinforced by self-supervision. Empirical results on real-world datasets demonstrate that our proposed method performs exceptionally in the extraction, registration and segmentation tasks. Our code and data can be found at https://github.com/Anonymous4545/JERSComment: Published as a research track paper at KDD 2023. Code: https://github.com/Anonymous4545/JER

Superior longitudinal fasciculus microstructure and its functional triple-network mechanisms in depressive rumination

Depressive rumination, which involves a repetitive focus on one's distress, is associated with function connectivity disturbances of Default-Mode, Salience, and Executive-Control networks, comprising the so-called "triple-network" of attention. Missing, however, is a multimodal account of rumination that neuroanatomically explains the perseveration of these dysfunctional networks as a stable human trait. Using diffusion and functional Magnetic Resonance Imaging, we explored multimodal relationships between rumination severity, white-matter microstructure, and resting-state functional connectivity in N=39 depressed adults, and then directly replicated our findings in a demographically-matched, independent sample (N=39). Among the fully-replicated results, three core findings emerged. First, rumination severity is associated with both disintegrated and desegregated functional connectivity of the triple-network. Second, global microstructural inefficiency of the right Superior Longitudinal Fasciculus (SLF) provides a neuroanatomical connectivity basis for rumination and accounts for anywhere between 25-37% of the variance in rumination (Discovery: p corr<0.01; Replication: p corr<0.01; MSE=0.05). Finally, microstructure of the right SLF and auxiliary white-matter is strongly associated with functional connectivity biomarkers of rumination, both within and between components of the triple-network (Discovery: R²=0.36, p corr<0.05; Replication: R²=0.25, p corr<0.05; MSE=0.04-0.06). By cross-validating discovery with replication, our findings advance a reproducible microstructural-functional brain connectivity model of depressive rumination that unifies neurodevelopmental and neurocognitive perspectives.Psycholog

HA-HI: Synergising fMRI and DTI through Hierarchical Alignments and Hierarchical Interactions for Mild Cognitive Impairment Diagnosis

Early diagnosis of mild cognitive impairment (MCI) and subjective cognitive decline (SCD) utilizing multi-modal magnetic resonance imaging (MRI) is a pivotal area of research. While various regional and connectivity features from functional MRI (fMRI) and diffusion tensor imaging (DTI) have been employed to develop diagnosis models, most studies integrate these features without adequately addressing their alignment and interactions. This limits the potential to fully exploit the synergistic contributions of combined features and modalities. To solve this gap, our study introduces a novel Hierarchical Alignments and Hierarchical Interactions (HA-HI) method for MCI and SCD classification, leveraging the combined strengths of fMRI and DTI. HA-HI efficiently learns significant MCI- or SCD- related regional and connectivity features by aligning various feature types and hierarchically maximizing their interactions. Furthermore, to enhance the interpretability of our approach, we have developed the Synergistic Activation Map (SAM) technique, revealing the critical brain regions and connections that are indicative of MCI/SCD. Comprehensive evaluations on the ADNI dataset and our self-collected data demonstrate that HA-HI outperforms other existing methods in diagnosing MCI and SCD, making it a potentially vital and interpretable tool for early detection. The implementation of this method is publicly accessible at https://github.com/ICI-BCI/Dual-MRI-HA-HI.git

Deep learning in medical image registration: introduction and survey

Image registration (IR) is a process that deforms images to align them with respect to a reference space, making it easier for medical practitioners to examine various medical images in a standardized reference frame, such as having the same rotation and scale. This document introduces image registration using a simple numeric example. It provides a definition of image registration along with a space-oriented symbolic representation. This review covers various aspects of image transformations, including affine, deformable, invertible, and bidirectional transformations, as well as medical image registration algorithms such as Voxelmorph, Demons, SyN, Iterative Closest Point, and SynthMorph. It also explores atlas-based registration and multistage image registration techniques, including coarse-fine and pyramid approaches. Furthermore, this survey paper discusses medical image registration taxonomies, datasets, evaluation measures, such as correlation-based metrics, segmentation-based metrics, processing time, and model size. It also explores applications in image-guided surgery, motion tracking, and tumor diagnosis. Finally, the document addresses future research directions, including the further development of transformers

Whole brain resting-state analysis reveals decreased functional connectivity in major depression

Recently, both increases and decreases in resting-state functional connectivity have been found in major depression. However, these studies only assessed functional connectivity within a specific network or between a few regions of interest, while comorbidity and use of medication was not always controlled for. Therefore, the aim of the current study was to investigate whole-brain functional connectivity, unbiased by a priori definition of regions or networks of interest, in medication-free depressive patients without comorbidity. We analyzed resting-state fMRI data of 19 medication-free patients with a recent diagnosis of major depression (within six months before inclusion) and no comorbidity, and 19 age- and gender-matched controls. Independent component analysis was employed on the concatenated data sets of all participants. Thirteen functionally relevant networks were identified, describing the entire study sample. Next, individual representations of the networks were created using a dual regression method. Statistical inference was subsequently done on these spatial maps using voxelwise permutation tests. Abnormal functional connectivity was found within three resting-state networks in depression: 1) decreased bilateral amygdala and left anterior insula connectivity in an affective network, 2) reduced connectivity of the left frontal pole in a network associated with attention and working memory, and 3) decreased bilateral lingual gyrus connectivity within ventromedial visual regions. None of these effects were associated with symptom severity or grey matter density. We found abnormal resting-state functional connectivity not previously associated with major depression, which might relate to abnormal affect regulation and mild cognitive deficits, both associated with the symptomatology of the disorder

Towards Deeper Understanding in Neuroimaging

Neuroimaging is a growing domain of research, with advances in machine learning having tremendous potential to expand understanding in neuroscience and improve public health. Deep neural networks have recently and rapidly achieved historic success in numerous domains, and as a consequence have completely redefined the landscape of automated learners, giving promise of significant advances in numerous domains of research. Despite recent advances and advantages over traditional machine learning methods, deep neural networks have yet to have permeated significantly into neuroscience studies, particularly as a tool for discovery. This dissertation presents well-established and novel tools for unsupervised learning which aid in feature discovery, with relevant applications to neuroimaging. Through our works within, this dissertation presents strong evidence that deep learning is a viable and important tool for neuroimaging studies

A Survey on Deep Learning in Medical Image Registration: New Technologies, Uncertainty, Evaluation Metrics, and Beyond

Over the past decade, deep learning technologies have greatly advanced the field of medical image registration. The initial developments, such as ResNet-based and U-Net-based networks, laid the groundwork for deep learning-driven image registration. Subsequent progress has been made in various aspects of deep learning-based registration, including similarity measures, deformation regularizations, and uncertainty estimation. These advancements have not only enriched the field of deformable image registration but have also facilitated its application in a wide range of tasks, including atlas construction, multi-atlas segmentation, motion estimation, and 2D-3D registration. In this paper, we present a comprehensive overview of the most recent advancements in deep learning-based image registration. We begin with a concise introduction to the core concepts of deep learning-based image registration. Then, we delve into innovative network architectures, loss functions specific to registration, and methods for estimating registration uncertainty. Additionally, this paper explores appropriate evaluation metrics for assessing the performance of deep learning models in registration tasks. Finally, we highlight the practical applications of these novel techniques in medical imaging and discuss the future prospects of deep learning-based image registration

A Plug-and-Play Image Registration Network

Deformable image registration (DIR) is an active research topic in biomedical imaging. There is a growing interest in developing DIR methods based on deep learning (DL). A traditional DL approach to DIR is based on training a convolutional neural network (CNN) to estimate the registration field between two input images. While conceptually simple, this approach comes with a limitation that it exclusively relies on a pre-trained CNN without explicitly enforcing fidelity between the registered image and the reference. We present plug-and-play image registration network (PIRATE) as a new DIR method that addresses this issue by integrating an explicit data-fidelity penalty and a CNN prior. PIRATE pre-trains a CNN denoiser on the registration field and "plugs" it into an iterative method as a regularizer. We additionally present PIRATE+ that fine-tunes the CNN prior in PIRATE using deep equilibrium models (DEQ). PIRATE+ interprets the fixed-point iteration of PIRATE as a network with effectively infinite layers and then trains the resulting network end-to-end, enabling it to learn more task-specific information and boosting its performance. Our numerical results on OASIS and CANDI datasets show that our methods achieve state-of-the-art performance on DIR

Deep learning approaches to multimodal MRI brain age estimation

Brain ageing remains an intricate, multifaceted process, marked not just by chronological time but by a myriad of structural, functional, and microstructural changes that often lead to discrepancies between actual age and the age inferred from neuroimaging. Machine learning methods, and especially Convolutional Neural Networks (CNNs), have proven adept in capturing patterns relating to ageing induced changes in the brain. The differences between the predicted and chronological ages, referred to as brain age deltas, have emerged as useful biomarkers for exploring those factors which promote accelerated ageing or resilience, such as pathologies or lifestyle factors. However, previous studies relied overwhelmingly on structural neuroimaging for predictions, overlooking rich details inherent in other MRI modalities, such as potentially informative functional and microstructural changes. This research, utilising the extensive UK Biobank dataset, reveals that 57 different maps spanning structural, susceptibility-weighted, diffusion, and functional MRI modalities can not only predict an individual's chronological age, but also encode unique ageing-related details. Through the use of both 3D CNNs and the novel 3D Shifted Window (SWIN) Transformers, this work uncovered associations between brain age deltas and 191 different non-imaging derived phenotypes (nIDPs), offering a valuable insight into factors influencing brain ageing. Moreover, this work found that ensembling data from multiple maps results in higher prediction accuracies. After a thorough comparison of both linear and non-linear multi-modal ensembling methods, including deep fusion networks, it was found that linear methods, such as ElasticNet, generally outperform their more complex non-linear counterparts. In addition, while ensembling was found to strengthen age prediction accuracies, it was found to weaken nIDP associations in certain circumstances where ensembled maps might have opposing sensitivities to a particular nIDP, thus reinforcing the need for guided selections of the ensemble components. Finally, while both CNNs and SWINs show comparable brain age prediction precision, SWIN networks stand out for their robustness against data corruption, while also proving a degree of inherent explainability. Overall, the results presented herein demonstrate that other 3D maps and modalities, which have not been considered previously for the task of brain age prediction, encode different information about the ageing brain. This research lays the foundation for further explorations into how different factors, such as off-target drug effects, impact brain ageing. It also ushers in possibilities for enhanced clinical trial design, diagnostic approaches, and therapeutic monitoring grounded in refined brain age prediction models