A Deep Probabilistic Spatiotemporal Framework for Dynamic Graph Representation Learning with Application to Brain Disorder Identification

Recent applications of pattern recognition techniques on brain connectome classification using functional connectivity (FC) neglect the non-Euclidean topology and causal dynamics of brain connectivity across time. In this paper, a deep probabilistic spatiotemporal framework developed based on variational Bayes (DSVB) is proposed to learn time-varying topological structures in dynamic brain FC networks for autism spectrum disorder (ASD) identification. The proposed framework incorporates a spatial-aware recurrent neural network to capture rich spatiotemporal patterns across dynamic FC networks, followed by a fully-connected neural network to exploit these learned patterns for subject-level classification. To overcome model overfitting on limited training datasets, an adversarial training strategy is introduced to learn graph embedding models that generalize well to unseen brain networks. Evaluation on the ABIDE resting-state functional magnetic resonance imaging dataset shows that our proposed framework significantly outperformed state-of-the-art methods in identifying ASD. Dynamic FC analyses with DSVB learned embeddings reveal apparent group difference between ASD and healthy controls in network profiles and switching dynamics of brain states

CI-GNN: A Granger Causality-Inspired Graph Neural Network for Interpretable Brain Network-Based Psychiatric Diagnosis

There is a recent trend to leverage the power of graph neural networks (GNNs) for brain-network based psychiatric diagnosis, which,in turn, also motivates an urgent need for psychiatrists to fully understand the decision behavior of the used GNNs. However, most of the existing GNN explainers are either post-hoc in which another interpretive model needs to be created to explain a well-trained GNN, or do not consider the causal relationship between the extracted explanation and the decision, such that the explanation itself contains spurious correlations and suffers from weak faithfulness. In this work, we propose a granger causality-inspired graph neural network (CI-GNN), a built-in interpretable model that is able to identify the most influential subgraph (i.e., functional connectivity within brain regions) that is causally related to the decision (e.g., major depressive disorder patients or healthy controls), without the training of an auxillary interpretive network. CI-GNN learns disentangled subgraph-level representations {\alpha} and \b{eta} that encode, respectively, the causal and noncausal aspects of original graph under a graph variational autoencoder framework, regularized by a conditional mutual information (CMI) constraint. We theoretically justify the validity of the CMI regulation in capturing the causal relationship. We also empirically evaluate the performance of CI-GNN against three baseline GNNs and four state-of-the-art GNN explainers on synthetic data and three large-scale brain disease datasets. We observe that CI-GNN achieves the best performance in a wide range of metrics and provides more reliable and concise explanations which have clinical evidence.Comment: 45 pages, 13 figure

Optimisation de réseaux de neurones à décharges avec contraintes matérielles pour processeur neuromorphique

Les modèles informatiques basés sur l'apprentissage machine ont démarré la seconde révolution de l'intelligence artificielle. Capables d'atteindre des performances que l'on crut inimaginables au préalable, ces modèles semblent devenir partie courante dans plusieurs domaines. La face cachée de ceux-ci est que l'énergie consommée pour l'apprentissage, et l'utilisation de ces techniques, est colossale. La dernière décennie a été marquée par l'arrivée de plusieurs processeurs neuromorphiques pouvant simuler des réseaux de neurones avec une faible consommation d'énergie. Ces processeurs offrent une alternative aux conventionnelles cartes graphiques qui demeurent à ce jour essentielles au domaine. Ces processeurs sont capables de réduire la consommation d'énergie en utilisant un modèle de neurone événementiel, plus communément appelé neurone à décharge. Ce type de neurone est fondamentalement différent du modèle classique, et possède un aspect temporel important. Les méthodes, algorithmes et outils développés pour le modèle de neurone classique ne sont pas adaptés aux neurones à décharges. Cette thèse de doctorat décrit plusieurs approches fondamentales, dédiées à la création de processeurs neuromorphiques analogiques, qui permettent de pallier l'écart existant entre les systèmes à base de neurones conventionnels et à décharges. Dans un premier temps, nous présentons une nouvelle règle de plasticité synaptique permettant l'apprentissage non supervisé des réseaux de neurones récurrents utilisant ce nouveau type de neurone. Puis, nous proposons deux nouvelles méthodes pour la conception des topologies de ce même type de réseau. Finalement, nous améliorons les techniques d'apprentissage supervisé en augmentant la capacité de mémoire de réseaux récurrents. Les éléments de cette thèse marient l'inspiration biologique du cerveau, l'ingénierie neuromorphique et l'informatique fondamentale pour permettre d'optimiser les réseaux de neurones pouvant fonctionner sur des processeurs neuromorphiques analogiques

Methods for improved mapping of brain lesion connectivity

Recent advances over the past two decades in neuroimaging methods have enabled us to map the connectivity of the brain. In parallel, pathophysiological models of brain disease have shifted from an emphasis on understanding pathology in specific brain regions to characterizing disruptions to interconnected neural networks. Nevertheless, these recent methods for mapping brain connectivity are still under development. Every step of the mapping process becomes a potential source for additional error due to noise or artifacts that could impact final analyses. Segmentation, parcellation, registration, and tractography are some of the steps where this occurs. Moreover, mapping the connectivity in a brain lesion is even more susceptible to errors in these steps. In this body of work, I describe multiple new methods for improving the accuracy of mapping lesion connectivity by reducing errors at the tractography stage which is the most error prone stage. First, we develop an approach for directly normalizing streamlines into a template space that avoids performing tractography in the normalized template space, reducing the error of connectomes constructed in the template space with respect to the ground truth native space connectome. Second, we develop a rapid approach for performing shortest path tractography and constructing shortest path probability weighted connectomes which increases the connection specificity relative to local streamline tracking approaches. We then demonstrate how our shortest path tractography approach can be used construct a disconnectome, a connectivity map of the proportion of connections lost due to intersecting a lesion. We then develop a fast, greedy graph-theoretic algorithm that extracts the maximally disconnected subgraph containing brain regions with the greatest shared loss of connectivity. Finally, we demonstrate how combining methods from diffusion based image inpainting and optimal estimation can be used to restore or inpaint corrupted fiber diffusion models in lesioned white matter tissue, enabling tractography and the study of lesion connectivity and modeling of microstructural measures in the patient’s native space

Blending generative models with deep learning for multidimensional phenotypic prediction from brain connectivity data

Network science as a discipline has provided us with foundational machinery to study complex relational entities such as social networks, genomics, econometrics etc. The human brain is a complex network that has recently garnered immense interest within the data science community. Connectomics or the study of the underlying connectivity patterns in the brain has become an important field of study for the characterization of various neurological disorders such as Autism, Schizophrenia etc. Such connectomic studies have provided several fundamental insights into its intrinsic organisation and implications on our behavior and health. This thesis proposes a collection of mathematical models that are capable of fusing information from functional and structural connectivity with phenotypic information. Here, functional connectivity is measured by resting state functional MRI (rs-fMRI), while anatomical connectivity is captured using Diffusion Tensor Imaging (DTI). The phenotypic information of interest could refer to continuous measures of behavior or cognition, or may capture levels of impairment in the case of neuropsychiatric disorders. We first develop a joint network optimization framework to predict clinical severity from rs-fMRI connectivity matrices. This model couples two key terms into a unified optimization framework: a generative matrix factorization and a discriminative linear regression model. We demonstrate that the proposed joint inference strategy is successful in generalizing to prediction of impairments in Autism Spectrum Disorder (ASD) when compared with several machine learning, graph theoretic and statistical baselines. At the same time, the model is capable of extracting functional brain biomarkers that are informative of individual measures of clinical severity. We then present two modeling extensions to non-parametric and neural network regression models that are coupled with the same generative framework. Building on these general principles, we extend our framework to incorporate multimodal information from Diffusion Tensor Imaging (DTI) and dynamic functional connectivity. At a high level, our generative matrix factorization now estimates a time-varying functional decomposition. At the same time, it is guided by anatomical connectivity priors in a graph-based regularization setup. This connectivity model is coupled with a deep network that predicts multidimensional clinical characterizations and models the temporal dynamics of the functional scan. This framework allows us to simultaneously explain multiple impairments, isolate stable multi-modal connectivity signatures, and study the evolution of various brain states at rest. Lastly, we shift our focus to end-to-end geometric frameworks. These are designed to characterize the complementarity between functional and structural connectivity data spaces, while using clinical information as a secondary guide. As an alternative to the previous generative framework for functional connectivity, our representation learning scheme of choice is a matrix autoencoder that is crafted to reflect the underlying data geometry. This is coupled with a manifold alignment model that maps from function to structure and a deep network that maps to phenotypic information. We demonstrate that the model reliably recovers structural connectivity patterns across individuals, while robustly extracting predictive yet interpretable brain biomarkers. Finally, we also present a preliminary analytical and experimental exposition on the theoretical aspects of the matrix autoencoder representation

Unsupervised deep learning of human brain diffusion magnetic resonance imaging tractography data

L'imagerie par résonance magnétique de diffusion est une technique non invasive permettant de connaître la microstructure organisationnelle des tissus biologiques. Les méthodes computationnelles qui exploitent la préférence orientationnelle de la diffusion dans des structures restreintes pour révéler les voies axonales de la matière blanche du cerveau sont appelées tractographie. Ces dernières années, diverses méthodes de tractographie ont été utilisées avec succès pour découvrir l'architecture de la matière blanche du cerveau. Pourtant, ces techniques de reconstruction souffrent d'un certain nombre de défauts dérivés d'ambiguïtés fondamentales liées à l'information orientationnelle. Cela a des conséquences dramatiques, puisque les cartes de connectivité de la matière blanche basées sur la tractographie sont dominées par des faux positifs. Ainsi, la grande proportion de voies invalides récupérées demeure un des principaux défis à résoudre par la tractographie pour obtenir une description anatomique fiable de la matière blanche. Des approches méthodologiques innovantes sont nécessaires pour aider à résoudre ces questions. Les progrès récents en termes de puissance de calcul et de disponibilité des données ont rendu possible l'application réussie des approches modernes d'apprentissage automatique à une variété de problèmes, y compris les tâches de vision par ordinateur et d'analyse d'images. Ces méthodes modélisent et trouvent les motifs sous-jacents dans les données, et permettent de faire des prédictions sur de nouvelles données. De même, elles peuvent permettre d'obtenir des représentations compactes des caractéristiques intrinsèques des données d'intérêt. Les approches modernes basées sur les données, regroupées sous la famille des méthodes d'apprentissage profond, sont adoptées pour résoudre des tâches d'analyse de données d'imagerie médicale, y compris la tractographie. Dans ce contexte, les méthodes deviennent moins dépendantes des contraintes imposées par les approches classiques utilisées en tractographie. Par conséquent, les méthodes inspirées de l'apprentissage profond conviennent au changement de paradigme requis, et peuvent ouvrir de nouvelles possibilités de modélisation, en améliorant ainsi l'état de l'art en tractographie. Dans cette thèse, un nouveau paradigme basé sur les techniques d'apprentissage de représentation est proposé pour générer et analyser des données de tractographie. En exploitant les architectures d'autoencodeurs, ce travail tente d'explorer leur capacité à trouver un code optimal pour représenter les caractéristiques des fibres de la matière blanche. Les contributions proposées exploitent ces représentations pour une variété de tâches liées à la tractographie, y compris (i) le filtrage et (ii) le regroupement efficace sur les résultats générés par d'autres méthodes, ainsi que (iii) la reconstruction proprement dite des fibres de la matière blanche en utilisant une méthode générative. Ainsi, les méthodes issues de cette thèse ont été nommées (i) FINTA (Filtering in Tractography using Autoencoders), (ii) CINTA (Clustering in Tractography using Autoencoders), et (iii) GESTA (Generative Sampling in Bundle Tractography using Autoencoders), respectivement. Les performances des méthodes proposées sont évaluées par rapport aux méthodes de l'état de l'art sur des données de diffusion synthétiques et des données de cerveaux humains chez l'adulte sain in vivo. Les résultats montrent que (i) la méthode de filtrage proposée offre une sensibilité et spécificité supérieures par rapport à d'autres méthodes de l'état de l'art; (ii) le regroupement des tractes dans des faisceaux est fait de manière consistante; et (iii) l'approche générative échantillonnant des tractes comble mieux l'espace de la matière blanche dans des régions difficiles à reconstruire. Enfin, cette thèse révèle les possibilités des autoencodeurs pour l'analyse des données des fibres de la matière blanche, et ouvre la voie à fournir des données de tractographie plus fiables.Abstract : Diffusion magnetic resonance imaging is a non-invasive technique providing insights into the organizational microstructure of biological tissues. The computational methods that exploit the orientational preference of the diffusion in restricted structures to reveal the brain's white matter axonal pathways are called tractography. In recent years, a variety of tractography methods have been successfully used to uncover the brain's white matter architecture. Yet, these reconstruction techniques suffer from a number of shortcomings derived from fundamental ambiguities inherent to the orientation information. This has dramatic consequences, since current tractography-based white matter connectivity maps are dominated by false positive connections. Thus, the large proportion of invalid pathways recovered remains one of the main challenges to be solved by tractography to obtain a reliable anatomical description of the white matter. Methodological innovative approaches are required to help solving these questions. Recent advances in computational power and data availability have made it possible to successfully apply modern machine learning approaches to a variety of problems, including computer vision and image analysis tasks. These methods model and learn the underlying patterns in the data, and allow making accurate predictions on new data. Similarly, they may enable to obtain compact representations of the intrinsic features of the data of interest. Modern data-driven approaches, grouped under the family of deep learning methods, are being adopted to solve medical imaging data analysis tasks, including tractography. In this context, the proposed methods are less dependent on the constraints imposed by current tractography approaches. Hence, deep learning-inspired methods are suit for the required paradigm shift, may open new modeling possibilities, and thus improve the state of the art in tractography. In this thesis, a new paradigm based on representation learning techniques is proposed to generate and to analyze tractography data. By harnessing autoencoder architectures, this work explores their ability to find an optimal code to represent the features of the white matter fiber pathways. The contributions exploit such representations for a variety of tractography-related tasks, including efficient (i) filtering and (ii) clustering on results generated by other methods, and (iii) the white matter pathway reconstruction itself using a generative method. The methods issued from this thesis have been named (i) FINTA (Filtering in Tractography using Autoencoders), (ii) CINTA (Clustering in Tractography using Autoencoders), and (iii) GESTA (Generative Sampling in Bundle Tractography using Autoencoders), respectively. The proposed methods' performance is assessed against current state-of-the-art methods on synthetic data and healthy adult human brain in vivo data. Results show that the (i) introduced filtering method has superior sensitivity and specificity over other state-of-the-art methods; (ii) the clustering method groups streamlines into anatomically coherent bundles with a high degree of consistency; and (iii) the generative streamline sampling technique successfully improves the white matter coverage in hard-to-track bundles. In summary, this thesis unlocks the potential of deep autoencoder-based models for white matter data analysis, and paves the way towards delivering more reliable tractography data

Reconfigurable computing for large-scale graph traversal algorithms

This thesis proposes a reconfigurable computing approach for supporting parallel processing in large-scale graph traversal algorithms. Our approach is based on a reconfigurable hardware architecture which exploits the capabilities of both FPGAs (Field-Programmable Gate Arrays) and a multi-bank parallel memory subsystem. The proposed methodology to accelerate graph traversal algorithms has been applied to three case studies, revealing that application-specific hardware customisations can benefit performance. A summary of our four contributions is as follows. First, a reconfigurable computing approach to accelerate large-scale graph traversal algorithms. We propose a reconfigurable hardware architecture which decouples computation and communication while keeping multiple memory requests in flight at any given time, taking advantage of the high bandwidth of multi-bank memory subsystems. Second, a demonstration of the effectiveness of our approach through two case studies: the breadth-first search algorithm, and a graphlet counting algorithm from bioinformatics. Both case studies involve graph traversal, but each of them adopts a different graph data representation. Third, a method for using on-chip memory resources in FPGAs to reduce off-chip memory accesses for accelerating graph traversal algorithms, through a case-study of the All-Pairs Shortest-Paths algorithm. This case study has been applied to process human brain network data. Fourth, an evaluation of an approach based on instruction-set extension for FPGA design against many-core GPUs (Graphics Processing Units), based on a set of benchmarks with different memory access characteristics. It is shown that while GPUs excel at streaming applications, the proposed approach can outperform GPUs in applications with poor locality characteristics, such as graph traversal problems.Open Acces

Complex Systems Engineering: Designing Advanced Functions In Dynamical And Mechanical Systems

From computation in neural networks to allostery in proteins, numerous natural and artificial systems are comprised of many interacting parts that give rise to advanced functions. To study such complex systems, a diverse array of interdisciplinary tools have been developed that relate the interactions of existing systems to their functions. However, engineering the interactions to perform designed functions in novel systems remains a significant challenge due to the nonlinearities in the interactions and the vast dimensionality of the design space. Here we develop design principles for complex dynamical and mechanical systems at the lowest level of their microstate interactions. In dynamical neural systems, we use methods from control theory and dynamical systems theory to mathematically map precise patterns of neural connectivity to the control of neural states in human and non-human brains (Chapter 2) and to the learning of computations on internal representations in artificial recurrent neural networks (Chapter 4). In mechanical systems, we use methods from algebraic geometry and dynamical systems to mathematically map precise patterns of mechanical constraints to design shape changes as a minimal model of protein allostery and cooperativity (Chapter 6) and to engineer mechanical metamaterials that possess arbitrarily complex shape changes (Chapter 8). These intuitive maps allow us to navigate previously unexplored design spaces in nonlinear and high-dimensional regimes, enabling us to reverse engineer form from function in novel complex systems that have yet to exist