Decoding Task-Based fMRI Data Using Graph Neural Networks, Considering Individual Differences

Functional magnetic resonance imaging (fMRI) is a non-invasive technology that provides high spatial resolution in determining the human brain\u27s responses and measures regional brain activity through metabolic changes in blood oxygen consumption associated with neural activity. Task fMRI provides an opportunity to analyze the working mechanisms of the human brain during specific task performance. Over the past several years, a variety of computational methods have been proposed to decode task fMRI data that can identify brain regions associated with different task stimulations. Despite the advances made by these methods, several limitations exist due to graph representations and graph embeddings transferred from task fMRI signals. In the present study, we proposed an end-to-end graph convolutional network by combining the convolutional neural network with graph representation, with three convolutional layers to classify task fMRI data from the Human Connectome Project (302 participants, 22–35 years of age). One goal of this dissertation was to improve classification performance. We applied four of the most widely used node embedding algorithms—NetMF, RandNE, Node2Vec, and Walklets—to automatically extract the structural properties of the nodes in the brain functional graph, then evaluated the performance of the classification model. The empirical results indicated that the proposed GCN framework accurately identified the brain\u27s state in task fMRI data and achieved comparable macro F1 scores of 0.978 and 0.976 with the NetMF and RandNE embedding methods, respectively. Another goal of the dissertation was to assess the effects of individual differences (i.e., gender and fluid intelligence) on classification performance. We tested the proposed GCN framework on sub-datasets divided according to gender and fluid intelligence. Experimental results indicated significant differences in the classification predictions of gender, but not high/low fluid intelligence fMRI data. Our experiments yielded promising results and demonstrated the superior ability of our GCN in modeling task fMRI data

Spatio-temporal Deep Learning Architectures for Data-Driven Learning of Brain’s Network Connectivity

Brain disorders are often linked to disruptions in the dynamics of the brain\u27s intrinsic functional networks. It is crucial to identify these networks and determine disruptions in their interactions to classify, understand, and possibly cure brain disorders. Brain\u27s network interactions are commonly assessed via functional (network)\ connectivity, captured as an undirected matrix of Pearson correlation coefficients. Functional connectivity can represent static and dynamic relations. However, often these are modeled using a fixed choice for the data window. Alternatively, deep learning models may flexibly learn various representations from the same data based on the model architecture and the training task. The representations produced by deep learning models are often difficult to interpret and require additional posthoc methods, e.g., saliency maps. Also, deep learning models typically require many input samples to learn features and perform the downstream task well. This dissertation introduces deep learning architectures that work on functional MRI data to estimate disorder-specific brain network connectivity and provide high classification accuracy in discriminating controls and patients. To handle the relatively low number of labeled subjects in the field of neuroimaging, this research proposes deep learning architectures that leverage self-supervised pre-training to increase downstream classification. To increase the interpretability and avoid using a posthoc method, deep learning architectures are proposed that expose a directed graph layer representing the model\u27s learning about relevant brain connectivity. The proposed models estimate task-specific directed connectivity matrices for each subject using the same data but training different models on their own discriminative tasks. The proposed architectures are tested with multiple neuroimaging datasets to discriminate controls and patients with schizophrenia, autism, and dementia, as well as age and gender prediction. The proposed approach reveals that differences in connectivity among sensorimotor networks relative to default-mode networks are an essential indicator of dementia and gender. Dysconnectivity between networks, especially sensorimotor and visual, is linked with schizophrenic patients. However, schizophrenic patients show increased intra-network default-mode connectivity compared to healthy controls. Sensorimotor connectivity is vital for both dementia and schizophrenia prediction, but the differences are in inter and intra-network connectivity

Phenotyping functional brain dynamics:A deep learning prespective on psychiatry

This thesis explores the potential of deep learning (DL) techniques combined with multi-site functional magnetic resonance imaging (fMRI) to enable automated diagnosis and biomarker discovery for psychiatric disorders. This marks a shift from the convention in the field of applying standard machine learning techniques on hand-crafted features from a single cohort.To enable this, we have focused on three main strategies: utilizing minimally pre-processed data to maintain spatio-temporal dynamics, developing sample-efficient DL models, and applying emerging DL training techniques like self-supervised and transfer learning to leverage large population-based datasets.Our empirical results suggest that DL models can sometimes outperform existing machine learning methods in diagnosing Autism Spectrum Disorder (ASD) and Major Depressive Disorder (MDD) from resting-state fMRI data, despite the smaller datasets and the high data dimensionality. Nonetheless, the generalization performance of these models is currently insufficient for clinical use, raising questions about the feasibility of applying supervised DL for diagnosis or biomarker discovery due to the highly heterogeneous nature of the disorders. Our findings suggest that normative modeling on functional brain dynamics provides a promising alternative to the current paradigm

Deep Interpretability Methods for Neuroimaging

Brain dynamics are highly complex and yet hold the key to understanding brain function and dysfunction. The dynamics captured by resting-state functional magnetic resonance imaging data are noisy, high-dimensional, and not readily interpretable. The typical approach of reducing this data to low-dimensional features and focusing on the most predictive features comes with strong assumptions and can miss essential aspects of the underlying dynamics. In contrast, introspection of discriminatively trained deep learning models may uncover disorder-relevant elements of the signal at the level of individual time points and spatial locations. Nevertheless, the difficulty of reliable training on high-dimensional but small-sample datasets and the unclear relevance of the resulting predictive markers prevent the widespread use of deep learning in functional neuroimaging. In this dissertation, we address these challenges by proposing a deep learning framework to learn from high-dimensional dynamical data while maintaining stable, ecologically valid interpretations. The developed model is pre-trainable and alleviates the need to collect an enormous amount of neuroimaging samples to achieve optimal training. We also provide a quantitative validation module, Retain and Retrain (RAR), that can objectively verify the higher predictability of the dynamics learned by the model. Results successfully demonstrate that the proposed framework enables learning the fMRI dynamics directly from small data and capturing compact, stable interpretations of features predictive of function and dysfunction. We also comprehensively reviewed deep interpretability literature in the neuroimaging domain. Our analysis reveals the ongoing trend of interpretability practices in neuroimaging studies and identifies the gaps that should be addressed for effective human-machine collaboration in this domain. This dissertation also proposed a post hoc interpretability method, Geometrically Guided Integrated Gradients (GGIG), that leverages geometric properties of the functional space as learned by a deep learning model. With extensive experiments and quantitative validation on MNIST and ImageNet datasets, we demonstrate that GGIG outperforms integrated gradients (IG), which is considered to be a popular interpretability method in the literature. As GGIG is able to identify the contours of the discriminative regions in the input space, GGIG may be useful in various medical imaging tasks where fine-grained localization as an explanation is beneficial

AAAI Workshop on Artificial Intelligence with Biased or Scarce Data (AIBSD)

This book is a collection of the accepted papers presented at the Workshop on Artificial Intelligence with Biased or Scarce Data (AIBSD) in conjunction with the 36th AAAI Conference on Artificial Intelligence 2022. During AIBSD 2022, the attendees addressed the existing issues of data bias and scarcity in Artificial Intelligence and discussed potential solutions in real-world scenarios. A set of papers presented at AIBSD 2022 is selected for further publication and included in this book

Approaches For Capturing Time-Varying Functional Network Connectivity With Application to Normative Development and Mental Illness

Since the beginning of medical science, the human brain has remained an unsolved puzzle; an illusive organ that controls everything- from breathing to heartbeats, from emotion to anger, and more. With the power of advanced neuroimaging techniques, scientists have now started to solve this nearly impossible puzzle, piece by piece. Over the past decade, various in vivo techniques, including functional magnetic resonance imaging (fMRI), have been increasingly used to understand brain functions. fMRI is extensively being used to facilitate the identification of various neuropsychological disorders such as schizophrenia (SZ), bipolar disorder (BP) and autism spectrum disorder (ASD). These disorders are currently diagnosed based on patients’ self-reported experiences, and observed symptoms and behaviors over the course of the illnesses. Therefore, efficient identification of biological-based markers (biomarkers) can lead to early diagnosis of these mental disorders, and provide a trajectory for disease progression. By applying advanced machine learning techniques on fMRI data, significant differences in brain function among patients with mental disorders and healthy controls can be identified. Moreover, by jointly estimating information from multiple modalities, such as, functional brain data and genetic factors, we can now investigate the relationship between brain function and genes. Functional connectivity (FC) has become a very common measure to characterize brain functions, where FC is defined as the temporal covariance of neural signals between multiple spatially distinct brain regions. Recently, researchers are studying the FC among functionally specialized brain networks which can be defined as a higher level of FC, and is termed as functional network connectivity (FNC, defined as the correlation value that summarizes the overall connection between brain ‘networks’ over time). Most functional connectivity studies have made the limiting assumption that connectivity is stationary over multiple minutes, and ignore to identify the time-varying and reoccurring patterns of FNC among brain regions (known as time-varying FNC). In this dissertation, we demonstrate the use of time-varying FNC features as potential biomarkers to differentiate between patients with mental disorders and healthy subjects. The developmental characteristics of time-varying FNC in children with typically developing brain and ASD have been extensively studies in a cross-sectional framework, and age-, sex- and disease-related FNC profiles have been proposed. Also, time-varying FNC is characterized in healthy adults and patients with severe mental disorders (SZ and BP). Moreover, an efficient classification algorithm is designed to identify patients and controls at individual level. Finally, a new framework is proposed to jointly utilize information from brain’s functional network connectivity and genetic features to find the associations between them. The frameworks that we presented here can help us understand the important role played by time-varying FNC to identify potential biomarkers for the diagnosis of severe mental disorders

Decoding Neural Signals with Computational Models: A Systematic Review of Invasive BMI

There are significant milestones in modern human's civilization in which mankind stepped into a different level of life with a new spectrum of possibilities and comfort. From fire-lighting technology and wheeled wagons to writing, electricity and the Internet, each one changed our lives dramatically. In this paper, we take a deep look into the invasive Brain Machine Interface (BMI), an ambitious and cutting-edge technology which has the potential to be another important milestone in human civilization. Not only beneficial for patients with severe medical conditions, the invasive BMI technology can significantly impact different technologies and almost every aspect of human's life. We review the biological and engineering concepts that underpin the implementation of BMI applications. There are various essential techniques that are necessary for making invasive BMI applications a reality. We review these through providing an analysis of (i) possible applications of invasive BMI technology, (ii) the methods and devices for detecting and decoding brain signals, as well as (iii) possible options for stimulating signals into human's brain. Finally, we discuss the challenges and opportunities of invasive BMI for further development in the area.Comment: 51 pages, 14 figures, review articl