Predicting Infant Brain Connectivity with Federated Multi-Trajectory GNNs using Scarce Data

The understanding of the convoluted evolution of infant brain networks during the first postnatal year is pivotal for identifying the dynamics of early brain connectivity development. Existing deep learning solutions suffer from three major limitations. First, they cannot generalize to multi-trajectory prediction tasks, where each graph trajectory corresponds to a particular imaging modality or connectivity type (e.g., T1-w MRI). Second, existing models require extensive training datasets to achieve satisfactory performance which are often challenging to obtain. Third, they do not efficiently utilize incomplete time series data. To address these limitations, we introduce FedGmTE-Net++, a federated graph-based multi-trajectory evolution network. Using the power of federation, we aggregate local learnings among diverse hospitals with limited datasets. As a result, we enhance the performance of each hospital's local generative model, while preserving data privacy. The three key innovations of FedGmTE-Net++ are: (i) presenting the first federated learning framework specifically designed for brain multi-trajectory evolution prediction in a data-scarce environment, (ii) incorporating an auxiliary regularizer in the local objective function to exploit all the longitudinal brain connectivity within the evolution trajectory and maximize data utilization, (iii) introducing a two-step imputation process, comprising a preliminary KNN-based precompletion followed by an imputation refinement step that employs regressors to improve similarity scores and refine imputations. Our comprehensive experimental results showed the outperformance of FedGmTE-Net++ in brain multi-trajectory prediction from a single baseline graph in comparison with benchmark methods

Applications of Deep Learning Techniques for Automated Multiple Sclerosis Detection Using Magnetic Resonance Imaging: A Review

Multiple Sclerosis (MS) is a type of brain disease which causes visual, sensory, and motor problems for people with a detrimental effect on the functioning of the nervous system. In order to diagnose MS, multiple screening methods have been proposed so far; among them, magnetic resonance imaging (MRI) has received considerable attention among physicians. MRI modalities provide physicians with fundamental information about the structure and function of the brain, which is crucial for the rapid diagnosis of MS lesions. Diagnosing MS using MRI is time-consuming, tedious, and prone to manual errors. Research on the implementation of computer aided diagnosis system (CADS) based on artificial intelligence (AI) to diagnose MS involves conventional machine learning and deep learning (DL) methods. In conventional machine learning, feature extraction, feature selection, and classification steps are carried out by using trial and error; on the contrary, these steps in DL are based on deep layers whose values are automatically learn. In this paper, a complete review of automated MS diagnosis methods performed using DL techniques with MRI neuroimaging modalities is provided. Initially, the steps involved in various CADS proposed using MRI modalities and DL techniques for MS diagnosis are investigated. The important preprocessing techniques employed in various works are analyzed. Most of the published papers on MS diagnosis using MRI modalities and DL are presented. The most significant challenges facing and future direction of automated diagnosis of MS using MRI modalities and DL techniques are also provided

A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

뇌의 미·거시적 특성을 이용한 자기공명영상의 딥러닝 이미지 간 변환 기법

학위논문(석사) -- 서울대학교대학원 : 자연과학대학 뇌인지과학과, 2023. 8. 차지욱.뇌는 여러 뇌 영역의 고도로 국소화된 기능과 신경 연결을 통한 영역의 통합으로 구성된다. 이러한 뇌의 신경 연결은 끊임없이 변화하는 환경에 효과적으로 대응하기 위해 시스템 및 시냅스 수준에서 지속적으로 변화한다. 이러한 동적 상호작용을 가능하게 하는 중요한 요소 중 하나는 인간 두뇌의 거시적 및 미시적 규모에서의 구조적 가소성이다. 뇌의 거시적 및 미세적 구조는 서로 다르지만 보완적인 정보를 전달하기 때문에 두 구조를 모두 고려하는 것은 뇌의 구조적 가소성과 인지 작업 중 연결성을 이해하는데 매우 중요하다. 하지만 기존의 연구는 이를 효과적으로 고려하지 못했다. 본 연구에선 인간 뇌의 거시적 및 미세적 구조를 통합하여 기존에 알지 못했던 인간 뇌의 의미와 새로운 표현형을 얻기 위해 구조 MRI (sMRI)에서 고품질 확산 텐서 이미징 (DTI) 및 트랙토그래피를 생성하도록 설계된 새로운 딥러닝 프레임워크인 Macro2Micro를 제시한다. 본 연구는 거시적 구조로부터 미시적 구조 정보를 유추할 수 있다는 가설을 전제로 하여, 초기에 한 가지 영상 기법만 획득하더라도 질병 진단 및 연구에 유익한 추가적인 영상 기법을 생성한다. 신경 영상 영역에서 전례가 없는 이 접근 방식은 멀티모달 이미지 번역의 이점을 활용하여 상당한 시간과 비용을 절감한다. Macro2Micro는 3D T1을 입력으로 사용하여 2D T1 슬라이스를 생성한 다음 적대적 생성망(GAN)을 통해 처리되어 2D DTI (FA) 슬라이스와 2D 트랙토그래피 이미지를 생성한다. 이 프로세스의 핵심 요소는 이미지 특성을 주파수 대역에 따라 분리하는 옥타브 합성곱을 사용하는 것이다. 이 프레임워크는 청소년 뇌 인지 발달(ABCD) 데이터를 사용하여 훈련되었으며, 이미지 픽셀 손실, 지각 손실, GAN 손실 및 뇌 중심 패치 GAN 손실을 통해 훈련 손실이 정의되었다. 본 연구의 결과는 정량적 및 정성적으로 뛰어난 성능을 보였으며 단순히 이미지 분포만을 학습한 것이 아닌 뇌의 구조적 및 미시적 구조의 생물학적인 특성까지 학습했다는 점에서 의의가 있다. 이미지 변환 모델을 데이터 증대 방법으로 잠재적으로 적용하면 데이터 불균형 및 희소성 문제를 해결할 수 있다. 이 연구는 발전하는 질병 모델링에서 다중 모드 이미징, 특히 T1, DTI 및 트랙토그래피의 조합 사용의 잠재력을 제시한다.The brain consists of the highly localized functions of several brain regions and the integration of these regions through neural connections. These brain neural connections are constantly changing at the systemic and synaptic levels to effectively respond to the ever-changing environment. One of the key factors enabling these dynamic interactions is the structural plasticity of the human brain at the macro and micro scale. Because the brain's macro- and micro-structures convey different but complementary information, considering both structures is critical to understanding the brain's structural plasticity and connectivity during cognitive tasks. However, previous studies have not effectively considered this issue. In this study, a novel deep learning framework, Macro2Micro, is proposed to generate high-quality Diffusion Tensor Imaging (DTI) and tractography from structural MRI (sMRI). The study is premised on the hypothesis that micro-scale structural information can be inferred from macro-scale structures, enabling the generation of different imaging modalities beneficial for disease diagnosis and research, even when only one modality is initially obtained. This approach, unprecedented in the realm of neuroimaging, leverages the benefits of cross-modality image translation, offering significant time and cost savings. The Macro2Micro framework utilizes 3D T1 to generate 2D T1 slices as input, which are then processed through a Generative Adversarial Network (GAN) to produce 2D DTI (FA) slices and subsequently 2D tractography. The key element of this process is the use of Octave Convolutions, which facilitate the analysis of connections between various scale MR modalities. The framework was trained using the Adolescent Brain Cognitive Development (ABCD) dataset, with training losses evaluated through Image Pixel loss, Perceptual loss, GAN loss, and brain-focused patch GAN loss. The results not only showed superior performance compared to other algorithms quantitatively and qualitatively but also have significant meaning in neuroscience in that they learned not only the image distribution but also the biological characteristics of the structural and microscopic structures of the brain. The potential application of this image translation model as a data augmentation method could address issues of data imbalance and scarcity. This research underscores the potential of multimodal imaging, specifically the combined use of T1, DTI, and tractography, in advancing disease modeling.Chapter 1. INTRODUCTION 1 Chapter 2. RELATED WORK 7 Chapter 3. METHOD 15 3.1. Architecture Overview 3.2. Octave Convolution 3.3. Networks 3.4. Training Losses 3.5. Image Quality Metrics 3.6. Comparison of generated and real FA images in low-dimensional representation 3.7. Prediction of biological and cognitive variables using predicted FA images 3.8. Prediction of Tractography from FA images 3.9. Experimental Settings Chapter 4. RESULTS 26 4.1. Qualitative Evaluation 4.2. Quantitative Evaluation 4.3. Generated FA images by Macro2Micro can efficiently predict sex, ADHD, and intelligence 4.4 Ablation Studies 4.5. Effectiveness of Macro2Micro along the distance from the center of the brain 4.6. FA Image Translation to Tractography Chapter 5. DISCUSSION AND CONCLUSIONS 40 Bibliography 45 국문초록 52석

Learning from Complex Neuroimaging Datasets

Advancements in Magnetic Resonance Imaging (MRI) allowed for the early diagnosis of neurodevelopmental disorders and neurodegenerative diseases. Neuroanatomical abnormalities in the cerebral cortex are often investigated by examining group-level differences of brain morphometric measures extracted from highly-sampled cortical surfaces. However, group-level differences do not allow for individual-level outcome prediction critical for the application to clinical practice. Despite the success of MRI-based deep learning frameworks, critical issues have been identified: (1) extracting accurate and reliable local features from the cortical surface, (2) determining a parsimonious subset of cortical features for correct disease diagnosis, (3) learning directly from a non-Euclidean high-dimensional feature space, (4) improving the robustness of multi-task multi-modal models, and (5) identifying anomalies in imbalanced and heterogeneous settings. This dissertation describes novel methodological contributions to tackle the challenges above. First, I introduce a Laplacian-based method for quantifying local Extra-Axial Cerebrospinal Fluid (EA-CSF) from structural MRI. Next, I describe a deep learning approach for combining local EA-CSF with other morphometric cortical measures for early disease detection. Then, I propose a data-driven approach for extending convolutional learning to non-Euclidean manifolds such as cortical surfaces. I also present a unified framework for robust multi-task learning from imaging and non-imaging information. Finally, I propose a semi-supervised generative approach for the detection of samples from untrained classes in imbalanced and heterogeneous developmental datasets. The proposed methodological contributions are evaluated by applying them to the early detection of Autism Spectrum Disorder (ASD) in the first year of the infant’s life. Also, the aging human brain is examined in the context of studying different stages of Alzheimer’s Disease (AD).Doctor of Philosoph

Is attention all you need in medical image analysis? A review

Medical imaging is a key component in clinical diagnosis, treatment planning and clinical trial design, accounting for almost 90% of all healthcare data. CNNs achieved performance gains in medical image analysis (MIA) over the last years. CNNs can efficiently model local pixel interactions and be trained on small-scale MI data. The main disadvantage of typical CNN models is that they ignore global pixel relationships within images, which limits their generalisation ability to understand out-of-distribution data with different 'global' information. The recent progress of Artificial Intelligence gave rise to Transformers, which can learn global relationships from data. However, full Transformer models need to be trained on large-scale data and involve tremendous computational complexity. Attention and Transformer compartments (Transf/Attention) which can well maintain properties for modelling global relationships, have been proposed as lighter alternatives of full Transformers. Recently, there is an increasing trend to co-pollinate complementary local-global properties from CNN and Transf/Attention architectures, which led to a new era of hybrid models. The past years have witnessed substantial growth in hybrid CNN-Transf/Attention models across diverse MIA problems. In this systematic review, we survey existing hybrid CNN-Transf/Attention models, review and unravel key architectural designs, analyse breakthroughs, and evaluate current and future opportunities as well as challenges. We also introduced a comprehensive analysis framework on generalisation opportunities of scientific and clinical impact, based on which new data-driven domain generalisation and adaptation methods can be stimulated