The effect of using multiple connectivity metrics in brain Functional Connectivity studies

Tese de mestrado integrado, Engenharia Biomédica e Biofísica (Sinais e Imagens Médicas) Universidade de Lisboa, Faculdade de Ciências, 2022Resting-state functional magnetic resonance imaging (rs-fMRI) has the potential to assist as a diagnostic or prognostic tool for a diverse set of neurological and neuropsychiatric disorders, which are often difficult to differentiate. fMRI focuses on the study of the brain functional Connectome, which is characterized by the functional connections and neuronal activity among different brain regions, also interpreted as communications between pairs of regions. This Functional Connectivity (FC) is quantified through the statistical dependences between brain regions’ blood-oxygen-level-dependent (BOLD) signals time-series, being traditionally evaluated by correlation coefficient metrics and represented as FC matrices. However, several studies underlined limitations regarding the use of correlation metrics to fully capture information from these signals, leading investigators towards different statistical metrics that would fill those shortcomings. Recently, investigators have turned their attention to Deep Learning (DL) models, outperforming traditional Machine Learning (ML) techniques due to their ability to automatically extract relevant information from high-dimensional data, like FC data, using these models with rs-fMRI data to improve diagnostic predictions, as well as to understand pathological patterns in functional Connectome, that can lead to the discovery of new biomarkers. In spite of very encouraging performances, the black-box nature of DL algorithms makes difficult to know which input information led the model to a certain prediction, restricting its use in clinical settings. The objective of this dissertation is to exploit the power of DL models, understanding how FC matrices created from different statistical metrics can provide information about the brain FC, beyond the conventionally used correlation family. Two publicly available datasets where studied, the ABIDE I dataset, composed by healthy and autism spectrum disease (ASD) individuals, and the ADHD-200 dataset, with typically developed controls and individuals with attention-deficit/hyperactive disorder (ADHD). The computation of the FC matrices of both datasets, using different statistical metrics, was performed in MATLAB using MULAN’s toolbox functions, encompassing the correlation coefficient, non-linear correlation coefficient, mutual information, coherence and transfer entropy. The classification of FC data was performed using two DL models, the improved ConnectomeCNN model and the innovative ConnectomeCNN-Autoencoder model. Moreover, another goal is to study the effect of a multi-metric approach in classification performances, combining multiple FC matrices computed from the different statistical metrics used, as well as to study the use of Explainable Artificial Intelligence (XAI) techniques, namely Layer-wise Relevance Propagation method (LRP), to surpass the black-box problem of DL models used, in order to reveal the most important brain regions in ADHD. The results show that the use of other statistical metrics to compute FC matrices can be a useful complement to the traditional correlation metric methods for the classification between healthy subjects and subjects diagnosed with ADHD and ASD. Namely, non-linear metrics like h2 and mutual information, achieved similar and, in some cases, even slightly better performances than correlation methods. The use of FC multi-metric, despite not showing improvements in classification performance compared to the best individual method, presented promising results, namely the ability of this approach to select the best features from all the FC matrices combined, achieving a similar performance in relation to the best individual metric in each of the evaluation measures of the model, leading to a more complete classification. The LRP analysis applied to ADHD-200 dataset proved to be promising, identifying brain regions related to the pathophysiology of ADHD, which are in broad accordance with FC and structural study’s findings.A ressonância magnética funcional em estado de repouso (rs-fMRI) tem o potencial de ser uma ferramenta auxiliar de diagnóstico ou prognóstico para um conjunto diversificado de distúrbios neurológicos e neuropsiquiátricos, que muitas vezes são difíceis de diferenciar. A análise de dados de rs-fMRI recorre muitas vezes ao conceito de conectoma funcional do cérebro, que se caracteriza pelas conexões funcionais entre as diferentes regiões do cérebro, sendo estas conexões interpretadas como comunicações entre diferentes pares de regiões cerebrais. Esta conectividade funcional é quantificada através de dependências estatísticas entre os sinais fMRI das regiões cerebrais, sendo estas tradicionalmente calculadas através da métrica coeficiente de correlação, e representadas através de matrizes de conectividade funcional. No entanto, vários estudos demonstraram limitações em relação ao uso de métricas de correlação, em que estas não conseguem capturar por completo todas as informações presentes nesses sinais, levando os investigadores à procura de diferentes métricas estatísticas que pudessem preencher essas lacunas na obtenção de informações mais completas desses sinais. O estudo destes distúrbios neurológicos e neuropsiquiátricos começou por se basear em técnicas como mapeamento paramétrico estatístico, no contexto de estudos de fMRI baseados em tarefas. Porém, essas técnicas apresentam certas limitações, nomeadamente a suposição de que cada região cerebral atua de forma independente, o que não corresponde ao conhecimento atual sobre o funcionamento do cérebro. O surgimento da rs-fMRI permitiu obter uma perspetiva mais global e deu origem a uma vasta literatura sobre o efeito de patologias nos padrões de conetividade em repouso, incluindo tentativas de diagnóstico automatizado com base em biomarcadores extraídos dos conectomas. Nos últimos anos, os investigadores voltaram a sua atenção para técnicas de diferentes ramos de Inteligência Artificial, mais propriamente para os algoritmos de Deep Learning (DL), uma vez que são capazes de superar os algoritmos tradicionais de Machine Learning (ML), que foram aplicados a estes estudos numa fase inicial, devido à sua capacidade de extrair automaticamente informações relevantes de dados de alta dimensão, como é o caso dos dados de conectividade funcional. Esses modelos utilizam os dados obtidos da rs-fMRI para melhorar as previsões de diagnóstico em relação às técnicas usadas atualmente em termos de precisão e rapidez, bem como para compreender melhor os padrões patológicos nas conexões funcionais destes distúrbios, podendo levar à descoberta de novos biomarcadores. Apesar do notável desempenho destes modelos, a arquitetura natural em caixa-preta dos algoritmos de DL, torna difícil saber quais as informações dos dados de entrada que levaram o modelo a executar uma determinada previsão, podendo este utilizar informações erradas dos dados para alcançar uma dada inferência, restringindo o seu uso em ambientes clínicos. O objetivo desta dissertação, desenvolvida no Instituto de Biofísica e Engenharia Biomédica, é explorar o poder dos modelos DL, de forma a avaliar até que ponto matrizes de conectividade funcional criadas a partir de diferentes métricas estatísticas podem fornecer mais informações sobre a conectividade funcional do cérebro, para além das métricas de correlação convencionalmente usadas neste tipo de estudos. Foram estudados dois conjuntos de dados bastante utilizados em estudos de Neurociência e que estão disponíveis publicamente: o conjunto de dados ABIDE-I, composto por indivíduos saudáveis e indivíduos com doenças do espectro do autismo (ASD), e o conjunto de dados ADHD-200, com controlos tipicamente desenvolvidos e indivíduos com transtorno do défice de atenção e hiperatividade (ADHD). Numa primeira fase foi realizada a computação das matrizes de conetividade funcional de ambos os conjuntos de dados, usando as diferentes métricas estatísticas. Para isso, foi desenvolvido código de MATLAB, onde se utilizam as séries temporais dos sinais BOLD obtidas dos dois conjuntos de dados para criar essas mesmas matrizes de conectividade funcional, incorporando funções de diferentes métricas estatísticas da caixa de ferramentas MULAN, compreendendo o coeficiente de correlação, o coeficiente de correlação não linear, a informação mútua, a coerência e a entropia de transferência. De seguida, a classificação dos dados de conectividade funcional, de forma a avaliar o efeito do uso de diferentes métricas estatísticas para a criação de matrizes de conectividade funcional na discriminação de sujeitos saudáveis e patológicos, foi realizada usando dois modelos de DL. O modelo ConnectomeCNN melhorado e o modelo inovador ConnectomeCNN-Autoencoder foram desenvolvidos com recurso à biblioteca de Redes Neuronais Keras, juntamente com o seu backend Tensorflow, ambos em Python. Estes modelos, desenvolvidos previamente no Instituto de Biofísica e Engenharia Biomédica, tiveram de ser otimizados de forma a obter a melhor performance, onde vários parâmetros dos modelos e do respetivo treino dos mesmos foram testados para os dados a estudar. Pretendeu-se também estudar o efeito de uma abordagem multi-métrica nas tarefas de classificação dos sujeitos de ambos os conjuntos de dados, sendo que, para estudar essa abordagem as diferentes matrizes calculadas a partir das diferentes métricas estatísticas utilizadas, foram combinadas, sendo usados os mesmos modelos que foram aplicados às matrizes de conectividade funcional de cada métrica estatística individualmente. É importante realçar que na abordagem multi-métrica também foi realizada a otimização dos parâmetros dos modelos utilizados e do respetivo treino, de modo a conseguir a melhor performance dos mesmos para estes dados. Para além destes dois objetivos, estudou-se o uso de técnicas de Inteligência Artificial Explicável (XAI), mais especificamente o método Layer-wise Relevance Propagation (LRP), com vista a superar o problema da caixa-preta dos modelos de DL, com a finalidade de explicar como é que os modelos estão a utilizar os dados de entrada para realizar uma dada previsão. O método LRP foi aplicado aos dois modelos utilizados anteriormente, usando como dados de entrada o conjunto de dados ADHD-200, permitindo assim revelar quais as regiões cerebrais mais importantes no que toca a um diagnóstico relacionado com o ADHD. Os resultados obtidos mostram que o uso de outras métricas estatísticas para criar as matrizes de Conectividade Funcional podem ser um complemento bastante útil às métricas estatísticas tradicionalmente utilizadas para a classificação entre indivíduos saudáveis e indivíduos como ASD e ADHD. Nomeadamente métricas estatísticas não lineares como o h2 e a informação mútua, obtiveram desempenhos semelhantes e, em alguns casos, desempenhos ligeiramente melhores em relação aos desempenhos obtidos por métodos de correlação, convencionalmente usados nestes estudos de conectividade funcional. A utilização da multi-métrica de conectividade funcional, apesar de não apresentar melhorias no desempenho geral da classificação em relação ao melhor método das matrizes de conectividade funcional individuais do conjunto de métricas estatísticas abordadas, apresenta resultados que justificam a exploração mais aprofundada deste tipo de abordagem, de forma a compreender melhor a complementaridade das métricas e a melhor maneira de as utilizar. O uso do método LRP aplicado ao conjunto de dados do ADHD-200 mostrou a sua aplicabilidade a este tipo de estudos e a modelos de DL, identificando as regiões cerebrais mais relacionadas à fisiopatologia do diagnóstico do ADHD que são compatíveis com o que é reportado por diversos estudos de conectividade funcional e estudos de alterações estruturais associados a esta doença. O facto destas técnicas de XAI demonstrarem como é que os modelos de DL estão a usar os dados de entrada para efetuar as previsões, pode significar uma mais rápida e aceite adoção destes algoritmos em ambientes clínicos. Estas técnicas podem auxiliar o diagnóstico e prognóstico destes distúrbios neurológicos e neuropsiquiátricos, que são na maioria das vezes difíceis de diferenciar, permitindo aos médicos adquirirem um conhecimento em relação à previsão realizada e poder explicar a mesma aos seus pacientes

Automatic Autism Spectrum Disorder Detection Using Artificial Intelligence Methods with MRI Neuroimaging: A Review

Autism spectrum disorder (ASD) is a brain condition characterized by diverse signs and symptoms that appear in early childhood. ASD is also associated with communication deficits and repetitive behavior in affected individuals. Various ASD detection methods have been developed, including neuroimaging modalities and psychological tests. Among these methods, magnetic resonance imaging (MRI) imaging modalities are of paramount importance to physicians. Clinicians rely on MRI modalities to diagnose ASD accurately. The MRI modalities are non-invasive methods that include functional (fMRI) and structural (sMRI) neuroimaging methods. However, the process of diagnosing ASD with fMRI and sMRI for specialists is often laborious and time-consuming; therefore, several computer-aided design systems (CADS) based on artificial intelligence (AI) have been developed to assist the specialist physicians. Conventional machine learning (ML) and deep learning (DL) are the most popular schemes of AI used for diagnosing ASD. This study aims to review the automated detection of ASD using AI. We review several CADS that have been developed using ML techniques for the automated diagnosis of ASD using MRI modalities. There has been very limited work on the use of DL techniques to develop automated diagnostic models for ASD. A summary of the studies developed using DL is provided in the appendix. Then, the challenges encountered during the automated diagnosis of ASD using MRI and AI techniques are described in detail. Additionally, a graphical comparison of studies using ML and DL to diagnose ASD automatically is discussed. We conclude by suggesting future approaches to detecting ASDs using AI techniques and MRI neuroimaging

An Overview on Artificial Intelligence Techniques for Diagnosis of Schizophrenia Based on Magnetic Resonance Imaging Modalities: Methods, Challenges, and Future Works

Schizophrenia (SZ) is a mental disorder that typically emerges in late adolescence or early adulthood. It reduces the life expectancy of patients by 15 years. Abnormal behavior, perception of emotions, social relationships, and reality perception are among its most significant symptoms. Past studies have revealed the temporal and anterior lobes of hippocampus regions of brain get affected by SZ. Also, increased volume of cerebrospinal fluid (CSF) and decreased volume of white and gray matter can be observed due to this disease. The magnetic resonance imaging (MRI) is the popular neuroimaging technique used to explore structural/functional brain abnormalities in SZ disorder owing to its high spatial resolution. Various artificial intelligence (AI) techniques have been employed with advanced image/signal processing methods to obtain accurate diagnosis of SZ. This paper presents a comprehensive overview of studies conducted on automated diagnosis of SZ using MRI modalities. Main findings, various challenges, and future works in developing the automated SZ detection are described in this paper

Automatic autism spectrum disorder detection using artificial intelligence methods with MRI neuroimaging: A review

Autism spectrum disorder (ASD) is a brain condition characterized by diverse signs and symptoms that appear in early childhood. ASD is also associated with communication deficits and repetitive behavior in affected individuals. Various ASD detection methods have been developed, including neuroimaging modalities and psychological tests. Among these methods, magnetic resonance imaging (MRI) imaging modalities are of paramount importance to physicians. Clinicians rely on MRI modalities to diagnose ASD accurately. The MRI modalities are non-invasive methods that include functional (fMRI) and structural (sMRI) neuroimaging methods. However, diagnosing ASD with fMRI and sMRI for specialists is often laborious and time-consuming; therefore, several computer-aided design systems (CADS) based on artificial intelligence (AI) have been developed to assist specialist physicians. Conventional machine learning (ML) and deep learning (DL) are the most popular schemes of AI used for diagnosing ASD. This study aims to review the automated detection of ASD using AI. We review several CADS that have been developed using ML techniques for the automated diagnosis of ASD using MRI modalities. There has been very limited work on the use of DL techniques to develop automated diagnostic models for ASD. A summary of the studies developed using DL is provided in the Supplementary Appendix. Then, the challenges encountered during the automated diagnosis of ASD using MRI and AI techniques are described in detail. Additionally, a graphical comparison of studies using ML and DL to diagnose ASD automatically is discussed. We suggest future approaches to detecting ASDs using AI techniques and MRI neuroimaging.Qatar National Librar

Deep Interpretability Methods for Neuroimaging

Brain dynamics are highly complex and yet hold the key to understanding brain function and dysfunction. The dynamics captured by resting-state functional magnetic resonance imaging data are noisy, high-dimensional, and not readily interpretable. The typical approach of reducing this data to low-dimensional features and focusing on the most predictive features comes with strong assumptions and can miss essential aspects of the underlying dynamics. In contrast, introspection of discriminatively trained deep learning models may uncover disorder-relevant elements of the signal at the level of individual time points and spatial locations. Nevertheless, the difficulty of reliable training on high-dimensional but small-sample datasets and the unclear relevance of the resulting predictive markers prevent the widespread use of deep learning in functional neuroimaging. In this dissertation, we address these challenges by proposing a deep learning framework to learn from high-dimensional dynamical data while maintaining stable, ecologically valid interpretations. The developed model is pre-trainable and alleviates the need to collect an enormous amount of neuroimaging samples to achieve optimal training. We also provide a quantitative validation module, Retain and Retrain (RAR), that can objectively verify the higher predictability of the dynamics learned by the model. Results successfully demonstrate that the proposed framework enables learning the fMRI dynamics directly from small data and capturing compact, stable interpretations of features predictive of function and dysfunction. We also comprehensively reviewed deep interpretability literature in the neuroimaging domain. Our analysis reveals the ongoing trend of interpretability practices in neuroimaging studies and identifies the gaps that should be addressed for effective human-machine collaboration in this domain. This dissertation also proposed a post hoc interpretability method, Geometrically Guided Integrated Gradients (GGIG), that leverages geometric properties of the functional space as learned by a deep learning model. With extensive experiments and quantitative validation on MNIST and ImageNet datasets, we demonstrate that GGIG outperforms integrated gradients (IG), which is considered to be a popular interpretability method in the literature. As GGIG is able to identify the contours of the discriminative regions in the input space, GGIG may be useful in various medical imaging tasks where fine-grained localization as an explanation is beneficial

An overview of artificial intelligence techniques for diagnosis of Schizophrenia based on magnetic resonance imaging modalities: Methods, challenges, and future works

Schizophrenia (SZ) is a mental disorder that typically emerges in late adolescence or early adulthood. It reduces the life expectancy of patients by 15 years. Abnormal behavior, perception of emotions, social relationships, and reality perception are among its most significant symptoms. Past studies have revealed that SZ affects the temporal and anterior lobes of hippocampus regions of the brain. Also, increased volume of cerebrospinal fluid (CSF) and decreased volume of white and gray matter can be observed due to this disease. Magnetic resonance imaging (MRI) is the popular neuroimaging technique used to explore structural/functional brain abnormalities in SZ disorder, owing to its high spatial resolution. Various artificial intelligence (AI) techniques have been employed with advanced image/signal processing methods to accurately diagnose SZ. This paper presents a comprehensive overview of studies conducted on the automated diagnosis of SZ using MRI modalities. First, an AI-based computer aided-diagnosis system (CADS) for SZ diagnosis and its relevant sections are presented. Then, this section introduces the most important conventional machine learning (ML) and deep learning (DL) techniques in the diagnosis of diagnosing SZ. A comprehensive comparison is also made between ML and DL studies in the discussion section. In the following, the most important challenges in diagnosing SZ are addressed. Future works in diagnosing SZ using AI techniques and MRI modalities are recommended in another section. Results, conclusion, and research findings are also presented at the end.Ministerio de Ciencia e Innovación (España)/ FEDER under the RTI2018-098913-B100 projectConsejería de Economía, Innovación, Ciencia y Empleo (Junta de Andalucía) and FEDER under CV20-45250 and A-TIC-080-UGR18 project

Decoding Task-Based fMRI Data Using Graph Neural Networks, Considering Individual Differences

Functional magnetic resonance imaging (fMRI) is a non-invasive technology that provides high spatial resolution in determining the human brain\u27s responses and measures regional brain activity through metabolic changes in blood oxygen consumption associated with neural activity. Task fMRI provides an opportunity to analyze the working mechanisms of the human brain during specific task performance. Over the past several years, a variety of computational methods have been proposed to decode task fMRI data that can identify brain regions associated with different task stimulations. Despite the advances made by these methods, several limitations exist due to graph representations and graph embeddings transferred from task fMRI signals. In the present study, we proposed an end-to-end graph convolutional network by combining the convolutional neural network with graph representation, with three convolutional layers to classify task fMRI data from the Human Connectome Project (302 participants, 22–35 years of age). One goal of this dissertation was to improve classification performance. We applied four of the most widely used node embedding algorithms—NetMF, RandNE, Node2Vec, and Walklets—to automatically extract the structural properties of the nodes in the brain functional graph, then evaluated the performance of the classification model. The empirical results indicated that the proposed GCN framework accurately identified the brain\u27s state in task fMRI data and achieved comparable macro F1 scores of 0.978 and 0.976 with the NetMF and RandNE embedding methods, respectively. Another goal of the dissertation was to assess the effects of individual differences (i.e., gender and fluid intelligence) on classification performance. We tested the proposed GCN framework on sub-datasets divided according to gender and fluid intelligence. Experimental results indicated significant differences in the classification predictions of gender, but not high/low fluid intelligence fMRI data. Our experiments yielded promising results and demonstrated the superior ability of our GCN in modeling task fMRI data

Modern Views of Machine Learning for Precision Psychiatry

In light of the NIMH's Research Domain Criteria (RDoC), the advent of functional neuroimaging, novel technologies and methods provide new opportunities to develop precise and personalized prognosis and diagnosis of mental disorders. Machine learning (ML) and artificial intelligence (AI) technologies are playing an increasingly critical role in the new era of precision psychiatry. Combining ML/AI with neuromodulation technologies can potentially provide explainable solutions in clinical practice and effective therapeutic treatment. Advanced wearable and mobile technologies also call for the new role of ML/AI for digital phenotyping in mobile mental health. In this review, we provide a comprehensive review of the ML methodologies and applications by combining neuroimaging, neuromodulation, and advanced mobile technologies in psychiatry practice. Additionally, we review the role of ML in molecular phenotyping and cross-species biomarker identification in precision psychiatry. We further discuss explainable AI (XAI) and causality testing in a closed-human-in-the-loop manner, and highlight the ML potential in multimedia information extraction and multimodal data fusion. Finally, we discuss conceptual and practical challenges in precision psychiatry and highlight ML opportunities in future research

Schizo-Net: A novel Schizophrenia Diagnosis Framework Using Late Fusion Multimodal Deep Learning on Electroencephalogram-Based Brain Connectivity Indices

Schizophrenia (SCZ) is a serious mental condition that causes hallucinations, delusions, and disordered thinking. Traditionally, SCZ diagnosis involves the subject’s interview by a skilled psychiatrist. The process needs time and is bound to human errors and bias. Recently, brain connectivity indices have been used in a few pattern recognition methods to discriminate neuro-psychiatric patients from healthy subjects. The study presents Schizo-Net , a novel, highly accurate, and reliable SCZ diagnosis model based on a late multimodal fusion of estimated brain connectivity indices from EEG activity. First, the raw EEG activity is pre-processed exhaustively to remove unwanted artifacts. Next, six brain connectivity indices are estimated from the windowed EEG activity, and six different deep learning architectures (with varying neurons and hidden layers) are trained. The present study is the first which considers a large number of brain connectivity indices, especially for SCZ. A detailed study was also performed that identifies SCZ-related changes occurring in brain connectivity, and the vital significance of BCI is drawn in this regard to identify the biomarkers of the disease. Schizo-Net surpasses current models and achieves 99.84% accuracy. An optimum deep learning architecture selection is also performed for improved classification. The study also establishes that Late fusion technique outperforms single architecture-based prediction in diagnosing SCZ