228 research outputs found
Interpreting Age Effects of Human Fetal Brain from Spontaneous fMRI using Deep 3D Convolutional Neural Networks
Understanding human fetal neurodevelopment is of great clinical importance as
abnormal development is linked to adverse neuropsychiatric outcomes after
birth. Recent advances in functional Magnetic Resonance Imaging (fMRI) have
provided new insight into development of the human brain before birth, but
these studies have predominately focused on brain functional connectivity (i.e.
Fisher z-score), which requires manual processing steps for feature extraction
from fMRI images. Deep learning approaches (i.e., Convolutional Neural
Networks) have achieved remarkable success on learning directly from image
data, yet have not been applied on fetal fMRI for understanding fetal
neurodevelopment. Here, we bridge this gap by applying a novel application of
deep 3D CNN to fetal blood oxygen-level dependence (BOLD) resting-state fMRI
data. Specifically, we test a supervised CNN framework as a data-driven
approach to isolate variation in fMRI signals that relate to younger v.s. older
fetal age groups. Based on the learned CNN, we further perform sensitivity
analysis to identify brain regions in which changes in BOLD signal are strongly
associated with fetal brain age. The findings demonstrate that deep CNNs are a
promising approach for identifying spontaneous functional patterns in fetal
brain activity that discriminate age groups. Further, we discovered that
regions that most strongly differentiate groups are largely bilateral, share
similar distribution in older and younger age groups, and are areas of
heightened metabolic activity in early human development.Comment: 9 page
Deep Interpretability Methods for Neuroimaging
Brain dynamics are highly complex and yet hold the key to understanding brain function and dysfunction. The dynamics captured by resting-state functional magnetic resonance imaging data are noisy, high-dimensional, and not readily interpretable. The typical approach of reducing this data to low-dimensional features and focusing on the most predictive features comes with strong assumptions and can miss essential aspects of the underlying dynamics. In contrast, introspection of discriminatively trained deep learning models may uncover disorder-relevant elements of the signal at the level of individual time points and spatial locations. Nevertheless, the difficulty of reliable training on high-dimensional but small-sample datasets and the unclear relevance of the resulting predictive markers prevent the widespread use of deep learning in functional neuroimaging. In this dissertation, we address these challenges by proposing a deep learning framework to learn from high-dimensional dynamical data while maintaining stable, ecologically valid interpretations. The developed model is pre-trainable and alleviates the need to collect an enormous amount of neuroimaging samples to achieve optimal training.
We also provide a quantitative validation module, Retain and Retrain (RAR), that can objectively verify the higher predictability of the dynamics learned by the model. Results successfully demonstrate that the proposed framework enables learning the fMRI dynamics directly from small data and capturing compact, stable interpretations of features predictive of function and dysfunction. We also comprehensively reviewed deep interpretability literature in the neuroimaging domain. Our analysis reveals the ongoing trend of interpretability practices in neuroimaging studies and identifies the gaps that should be addressed for effective human-machine collaboration in this domain.
This dissertation also proposed a post hoc interpretability method, Geometrically Guided Integrated Gradients (GGIG), that leverages geometric properties of the functional space as learned by a deep learning model. With extensive experiments and quantitative validation on MNIST and ImageNet datasets, we demonstrate that GGIG outperforms integrated gradients (IG), which is considered to be a popular interpretability method in the literature. As GGIG is able to identify the contours of the discriminative regions in the input space, GGIG may be useful in various medical imaging tasks where fine-grained localization as an explanation is beneficial
Automatic Autism Spectrum Disorder Detection Using Artificial Intelligence Methods with MRI Neuroimaging: A Review
Autism spectrum disorder (ASD) is a brain condition characterized by diverse
signs and symptoms that appear in early childhood. ASD is also associated with
communication deficits and repetitive behavior in affected individuals. Various
ASD detection methods have been developed, including neuroimaging modalities
and psychological tests. Among these methods, magnetic resonance imaging (MRI)
imaging modalities are of paramount importance to physicians. Clinicians rely
on MRI modalities to diagnose ASD accurately. The MRI modalities are
non-invasive methods that include functional (fMRI) and structural (sMRI)
neuroimaging methods. However, the process of diagnosing ASD with fMRI and sMRI
for specialists is often laborious and time-consuming; therefore, several
computer-aided design systems (CADS) based on artificial intelligence (AI) have
been developed to assist the specialist physicians. Conventional machine
learning (ML) and deep learning (DL) are the most popular schemes of AI used
for diagnosing ASD. This study aims to review the automated detection of ASD
using AI. We review several CADS that have been developed using ML techniques
for the automated diagnosis of ASD using MRI modalities. There has been very
limited work on the use of DL techniques to develop automated diagnostic models
for ASD. A summary of the studies developed using DL is provided in the
appendix. Then, the challenges encountered during the automated diagnosis of
ASD using MRI and AI techniques are described in detail. Additionally, a
graphical comparison of studies using ML and DL to diagnose ASD automatically
is discussed. We conclude by suggesting future approaches to detecting ASDs
using AI techniques and MRI neuroimaging
Automatic autism spectrum disorder detection using artificial intelligence methods with MRI neuroimaging: A review
Autism spectrum disorder (ASD) is a brain condition characterized by
diverse signs and symptoms that appear in early childhood. ASD is also
associated with communication deficits and repetitive behavior in affected
individuals. Various ASD detection methods have been developed, including
neuroimaging modalities and psychological tests. Among these methods,
magnetic resonance imaging (MRI) imaging modalities are of paramount
importance to physicians. Clinicians rely on MRI modalities to diagnose
ASD accurately. The MRI modalities are non-invasive methods that include
functional (fMRI) and structural (sMRI) neuroimaging methods. However,
diagnosing ASD with fMRI and sMRI for specialists is often laborious and
time-consuming; therefore, several computer-aided design systems (CADS)
based on artificial intelligence (AI) have been developed to assist specialist
physicians. Conventional machine learning (ML) and deep learning (DL) are
the most popular schemes of AI used for diagnosing ASD. This study aims to
review the automated detection of ASD using AI. We review several CADS that
have been developed using ML techniques for the automated diagnosis of
ASD using MRI modalities. There has been very limited work on the use of DL techniques to develop automated diagnostic models for ASD. A summary of
the studies developed using DL is provided in the Supplementary Appendix.
Then, the challenges encountered during the automated diagnosis of ASD
using MRI and AI techniques are described in detail. Additionally, a graphical
comparison of studies using ML and DL to diagnose ASD automatically
is discussed. We suggest future approaches to detecting ASDs using AI
techniques and MRI neuroimaging.Qatar National
Librar
The role of MRI in diagnosing autism: a machine learning perspective.
There is approximately 1 in every 44 children in the United States suffers from autism spectrum disorder (ASD), a disorder characterized by social and behavioral impairments. Communication difficulties, interpersonal difficulties, and behavioral difficulties are the top common symptoms. Even though symptoms can begin as early as infancy, it may take multiple visits to a pediatric specialist before an accurate diagnosis can be made. In addition, the diagnosis can be subjective, and different specialists may give different scores. There is a growing body of research suggesting differences in brain development and/or environmental and/or genetic factors contribute to autism development, but scientists have yet to identify exactly the pathology of this disorder. ASD can currently be diagnosed by a set of diagnostic evaluations, regarded as the gold standard, such as the Autism Diagnostic Observation Schedule (ADOS) or the Autism Diagnostic Interview-Revised (ADI-R). A team of qualified clinicians is needed for performing the behavioral and communication tests as well as clinical history information, hence a considerable amount of time, effort, and subjective judgment is involved in using these gold-standard diagnostic instruments. In addition to standard observational assessment, recent advancements in neuroimaging and machine learning suggest a rapid and objective alternative, using brain imaging. An investigation of the employment of different imaging modalities, namely Diffusion Tensor Imaging (DTI), and resting state functional MRI (rs-fMRI) for autism diagnosis is presented in this comprehensive work. A detailed study of the implementation of feature engineering tools to find discriminant insights from different brain imaging modalities, including the use of novel feature representations, and the use of a machine learning framework to assist in the accurate classification of autistic individuals is introduced in this dissertation. Based on three large publicly available datasets, this extensive research highlights different decisions along the pipeline and their impact on diagnostic accuracy. It also identifies potentially impacted brain regions that contribute to an autism diagnosis. Achieving high global state-of-the-art cross-validated accuracy confirms the benefits of feature representation and feature engineering in extracting useful information, as well as the potential benefits of utilizing neuroimaging in the diagnosis of autism. This should enable an early, automated, and more objective personalized diagnosis
The effect of using multiple connectivity metrics in brain Functional Connectivity studies
Tese de mestrado integrado, Engenharia Biomédica e Biofísica (Sinais e Imagens Médicas) Universidade de Lisboa, Faculdade de Ciências, 2022Resting-state functional magnetic resonance imaging (rs-fMRI) has the potential to assist as a
diagnostic or prognostic tool for a diverse set of neurological and neuropsychiatric disorders, which are
often difficult to differentiate. fMRI focuses on the study of the brain functional Connectome, which is
characterized by the functional connections and neuronal activity among different brain regions, also
interpreted as communications between pairs of regions. This Functional Connectivity (FC) is quantified
through the statistical dependences between brain regions’ blood-oxygen-level-dependent (BOLD)
signals time-series, being traditionally evaluated by correlation coefficient metrics and represented as
FC matrices. However, several studies underlined limitations regarding the use of correlation metrics to
fully capture information from these signals, leading investigators towards different statistical metrics
that would fill those shortcomings. Recently, investigators have turned their attention to Deep Learning
(DL) models, outperforming traditional Machine Learning (ML) techniques due to their ability to
automatically extract relevant information from high-dimensional data, like FC data, using these models
with rs-fMRI data to improve diagnostic predictions, as well as to understand pathological patterns in
functional Connectome, that can lead to the discovery of new biomarkers. In spite of very encouraging
performances, the black-box nature of DL algorithms makes difficult to know which input information
led the model to a certain prediction, restricting its use in clinical settings.
The objective of this dissertation is to exploit the power of DL models, understanding how FC
matrices created from different statistical metrics can provide information about the brain FC, beyond
the conventionally used correlation family. Two publicly available datasets where studied, the ABIDE I dataset, composed by healthy and autism spectrum disease (ASD) individuals, and the ADHD-200
dataset, with typically developed controls and individuals with attention-deficit/hyperactive disorder
(ADHD). The computation of the FC matrices of both datasets, using different statistical metrics, was
performed in MATLAB using MULAN’s toolbox functions, encompassing the correlation coefficient,
non-linear correlation coefficient, mutual information, coherence and transfer entropy. The
classification of FC data was performed using two DL models, the improved ConnectomeCNN model
and the innovative ConnectomeCNN-Autoencoder model. Moreover, another goal is to study the effect
of a multi-metric approach in classification performances, combining multiple FC matrices computed
from the different statistical metrics used, as well as to study the use of Explainable Artificial
Intelligence (XAI) techniques, namely Layer-wise Relevance Propagation method (LRP), to surpass the
black-box problem of DL models used, in order to reveal the most important brain regions in ADHD.
The results show that the use of other statistical metrics to compute FC matrices can be a useful
complement to the traditional correlation metric methods for the classification between healthy subjects
and subjects diagnosed with ADHD and ASD. Namely, non-linear metrics like h2 and mutual
information, achieved similar and, in some cases, even slightly better performances than correlation
methods. The use of FC multi-metric, despite not showing improvements in classification performance
compared to the best individual method, presented promising results, namely the ability of this approach
to select the best features from all the FC matrices combined, achieving a similar performance in relation
to the best individual metric in each of the evaluation measures of the model, leading to a more complete
classification. The LRP analysis applied to ADHD-200 dataset proved to be promising, identifying brain
regions related to the pathophysiology of ADHD, which are in broad accordance with FC and structural
study’s findings.A ressonância magnética funcional em estado de repouso (rs-fMRI) tem o potencial de ser uma
ferramenta auxiliar de diagnóstico ou prognóstico para um conjunto diversificado de distúrbios
neurológicos e neuropsiquiátricos, que muitas vezes são difíceis de diferenciar. A análise de dados de
rs-fMRI recorre muitas vezes ao conceito de conectoma funcional do cérebro, que se caracteriza pelas
conexões funcionais entre as diferentes regiões do cérebro, sendo estas conexões interpretadas como
comunicações entre diferentes pares de regiões cerebrais. Esta conectividade funcional é quantificada
através de dependências estatísticas entre os sinais fMRI das regiões cerebrais, sendo estas
tradicionalmente calculadas através da métrica coeficiente de correlação, e representadas através de
matrizes de conectividade funcional. No entanto, vários estudos demonstraram limitações em relação ao
uso de métricas de correlação, em que estas não conseguem capturar por completo todas as informações
presentes nesses sinais, levando os investigadores à procura de diferentes métricas estatísticas que
pudessem preencher essas lacunas na obtenção de informações mais completas desses sinais.
O estudo destes distúrbios neurológicos e neuropsiquiátricos começou por se basear em técnicas
como mapeamento paramétrico estatístico, no contexto de estudos de fMRI baseados em tarefas. Porém,
essas técnicas apresentam certas limitações, nomeadamente a suposição de que cada região cerebral atua
de forma independente, o que não corresponde ao conhecimento atual sobre o funcionamento do cérebro.
O surgimento da rs-fMRI permitiu obter uma perspetiva mais global e deu origem a uma vasta literatura
sobre o efeito de patologias nos padrões de conetividade em repouso, incluindo tentativas de diagnóstico
automatizado com base em biomarcadores extraídos dos conectomas. Nos últimos anos, os
investigadores voltaram a sua atenção para técnicas de diferentes ramos de Inteligência Artificial, mais
propriamente para os algoritmos de Deep Learning (DL), uma vez que são capazes de superar os
algoritmos tradicionais de Machine Learning (ML), que foram aplicados a estes estudos numa fase
inicial, devido à sua capacidade de extrair automaticamente informações relevantes de dados de alta
dimensão, como é o caso dos dados de conectividade funcional. Esses modelos utilizam os dados obtidos
da rs-fMRI para melhorar as previsões de diagnóstico em relação às técnicas usadas atualmente em
termos de precisão e rapidez, bem como para compreender melhor os padrões patológicos nas conexões
funcionais destes distúrbios, podendo levar à descoberta de novos biomarcadores. Apesar do notável
desempenho destes modelos, a arquitetura natural em caixa-preta dos algoritmos de DL, torna difícil
saber quais as informações dos dados de entrada que levaram o modelo a executar uma determinada
previsão, podendo este utilizar informações erradas dos dados para alcançar uma dada inferência,
restringindo o seu uso em ambientes clínicos.
O objetivo desta dissertação, desenvolvida no Instituto de Biofísica e Engenharia Biomédica, é
explorar o poder dos modelos DL, de forma a avaliar até que ponto matrizes de conectividade funcional
criadas a partir de diferentes métricas estatísticas podem fornecer mais informações sobre a
conectividade funcional do cérebro, para além das métricas de correlação convencionalmente usadas
neste tipo de estudos. Foram estudados dois conjuntos de dados bastante utilizados em estudos de
Neurociência e que estão disponíveis publicamente: o conjunto de dados ABIDE-I, composto por
indivíduos saudáveis e indivíduos com doenças do espectro do autismo (ASD), e o conjunto de dados
ADHD-200, com controlos tipicamente desenvolvidos e indivíduos com transtorno do défice de atenção
e hiperatividade (ADHD).
Numa primeira fase foi realizada a computação das matrizes de conetividade funcional de ambos os
conjuntos de dados, usando as diferentes métricas estatísticas. Para isso, foi desenvolvido código de
MATLAB, onde se utilizam as séries temporais dos sinais BOLD obtidas dos dois conjuntos de dados
para criar essas mesmas matrizes de conectividade funcional, incorporando funções de diferentes
métricas estatísticas da caixa de ferramentas MULAN, compreendendo o coeficiente de correlação, o
coeficiente de correlação não linear, a informação mútua, a coerência e a entropia de transferência. De
seguida, a classificação dos dados de conectividade funcional, de forma a avaliar o efeito do uso de
diferentes métricas estatísticas para a criação de matrizes de conectividade funcional na discriminação
de sujeitos saudáveis e patológicos, foi realizada usando dois modelos de DL. O modelo
ConnectomeCNN melhorado e o modelo inovador ConnectomeCNN-Autoencoder foram desenvolvidos
com recurso à biblioteca de Redes Neuronais Keras, juntamente com o seu backend Tensorflow, ambos
em Python. Estes modelos, desenvolvidos previamente no Instituto de Biofísica e Engenharia
Biomédica, tiveram de ser otimizados de forma a obter a melhor performance, onde vários parâmetros
dos modelos e do respetivo treino dos mesmos foram testados para os dados a estudar. Pretendeu-se
também estudar o efeito de uma abordagem multi-métrica nas tarefas de classificação dos sujeitos de
ambos os conjuntos de dados, sendo que, para estudar essa abordagem as diferentes matrizes calculadas
a partir das diferentes métricas estatísticas utilizadas, foram combinadas, sendo usados os mesmos
modelos que foram aplicados às matrizes de conectividade funcional de cada métrica estatística
individualmente. É importante realçar que na abordagem multi-métrica também foi realizada a
otimização dos parâmetros dos modelos utilizados e do respetivo treino, de modo a conseguir a melhor
performance dos mesmos para estes dados. Para além destes dois objetivos, estudou-se o uso de técnicas
de Inteligência Artificial Explicável (XAI), mais especificamente o método Layer-wise Relevance
Propagation (LRP), com vista a superar o problema da caixa-preta dos modelos de DL, com a finalidade
de explicar como é que os modelos estão a utilizar os dados de entrada para realizar uma dada previsão.
O método LRP foi aplicado aos dois modelos utilizados anteriormente, usando como dados de entrada
o conjunto de dados ADHD-200, permitindo assim revelar quais as regiões cerebrais mais importantes
no que toca a um diagnóstico relacionado com o ADHD.
Os resultados obtidos mostram que o uso de outras métricas estatísticas para criar as matrizes de
Conectividade Funcional podem ser um complemento bastante útil às métricas estatísticas
tradicionalmente utilizadas para a classificação entre indivíduos saudáveis e indivíduos como ASD e
ADHD. Nomeadamente métricas estatísticas não lineares como o h2 e a informação mútua, obtiveram
desempenhos semelhantes e, em alguns casos, desempenhos ligeiramente melhores em relação aos
desempenhos obtidos por métodos de correlação, convencionalmente usados nestes estudos de
conectividade funcional. A utilização da multi-métrica de conectividade funcional, apesar de não
apresentar melhorias no desempenho geral da classificação em relação ao melhor método das matrizes
de conectividade funcional individuais do conjunto de métricas estatísticas abordadas, apresenta
resultados que justificam a exploração mais aprofundada deste tipo de abordagem, de forma a
compreender melhor a complementaridade das métricas e a melhor maneira de as utilizar. O uso do
método LRP aplicado ao conjunto de dados do ADHD-200 mostrou a sua aplicabilidade a este tipo de
estudos e a modelos de DL, identificando as regiões cerebrais mais relacionadas à fisiopatologia do
diagnóstico do ADHD que são compatíveis com o que é reportado por diversos estudos de conectividade
funcional e estudos de alterações estruturais associados a esta doença. O facto destas técnicas de XAI
demonstrarem como é que os modelos de DL estão a usar os dados de entrada para efetuar as previsões,
pode significar uma mais rápida e aceite adoção destes algoritmos em ambientes clínicos. Estas técnicas
podem auxiliar o diagnóstico e prognóstico destes distúrbios neurológicos e neuropsiquiátricos, que são
na maioria das vezes difíceis de diferenciar, permitindo aos médicos adquirirem um conhecimento em
relação à previsão realizada e poder explicar a mesma aos seus pacientes
Promises and pitfalls of deep neural networks in neuroimaging-based psychiatric research
By promising more accurate diagnostics and individual treatment
recommendations, deep neural networks and in particular convolutional neural
networks have advanced to a powerful tool in medical imaging. Here, we first
give an introduction into methodological key concepts and resulting
methodological promises including representation and transfer learning, as well
as modelling domain-specific priors. After reviewing recent applications within
neuroimaging-based psychiatric research, such as the diagnosis of psychiatric
diseases, delineation of disease subtypes, normative modeling, and the
development of neuroimaging biomarkers, we discuss current challenges. This
includes for example the difficulty of training models on small, heterogeneous
and biased data sets, the lack of validity of clinical labels, algorithmic
bias, and the influence of confounding variables
Recommended from our members
Application of Deep Learning to Brain Connectivity Classification in Large MRI Datasets
The use of machine learning for whole-brain classification of magnetic resonance imaging (MRI) data is of clear interest, both for understanding phenotypic differences in brain structure and function and for diagnostic applications. Developments of deep learning models in the past decade have revolutionized photographic image and speech recognition, bringing promise to do the same to other fields of science. However, there are many practical and theoretical challenges in the translation of such methods to the unique context of MRIs of the brain. This thesis presents a theoretical underpinning for whole-brain classification of extremely large datasets of multi-site MRIs, including machine learning model architecture, dataset curation methods, machine learning visualization methods, encoding of MRI data, and feature extraction. To replicate large sample sizes typically applied to deep learning models, a dataset of over 50,000 functional and structural MRIs was amassed from nine different databases, and the undertaken analyses were conducted on three covariates commonly found across these collections: sex, resting state/task, and autism spectrum disorder. I find that deep learning is not only a method that has promise for clinical application in the future, but also a powerful statistical tool for analyzing complex, nonlinear relationships in brain data where conventional statistics may fail. However, results are also dependent on factors such as dataset imbalances, confounding factors such as motion and head size, selected methods of encoding MRI data, variability of machine learning models and selected methods of visualizing the machine learning results. In this thesis, I present the following methodological innovations: (1) a method of balancing datasets as a means of regressing out measurable confounding factors; (2) a means of removing spatial biases from deep learning visualization methods; (3) methods of encoding functional and structural datasets as connectivity matrices; (4) the use of ensemble models and convolutional neural network architectures to improve classification accuracy and consistency; (5) adaptation of deep learning visualization methods to study brain connections utilized in the classification process. Additionally, I discuss interpretations, limitations, and future directions of this research.Gates Cambridge Scholarshi
- …