Machine Learning Methods for Brain Image Analysis

Understanding how the brain functions and quantifying compound interactions between complex synaptic networks inside the brain remain some of the most challenging problems in neuroscience. Lack or abundance of data, shortage of manpower along with heterogeneity of data following from various species all served as an added complexity to the already perplexing problem. The ability to process vast amount of brain data need to be performed automatically, yet with an accuracy close to manual human-level performance. These automated methods essentially need to generalize well to be able to accommodate data from different species. Also, novel approaches and techniques are becoming a necessity to reveal the correlations between different data modalities in the brain at the global level. In this dissertation, I mainly focus on two problems: automatic segmentation of brain electron microscopy (EM) images and stacks, and integrative analysis of the gene expression and synaptic connectivity in the brain. I propose to use deep learning algorithms for the 2D segmentation of EM images. I designed an automated pipeline with novel insights that was able to achieve state-of-the-art performance on the segmentation of the \textit{Drosophila} brain. I also propose a novel technique for 3D segmentation of EM image stacks that can be trained end-to-end with no prior knowledge of the data. This technique was evaluated in an ongoing online challenge for 3D segmentation of neurites where it achieved accuracy close to a second human observer. Later, I employed ensemble learning methods to perform the first systematic integrative analysis of the genome and connectome in the mouse brain at both the regional- and voxel-level. I show that the connectivity signals can be predicted from the gene expression signatures with an extremely high accuracy. Furthermore, I show that only a certain fraction of genes are responsible for this predictive aspect. Rich functional and cellular analysis of these genes are detailed to validate these findings

Data driven approaches for investigating molecular heterogeneity of the brain

It has been proposed that one of the clearest organizing principles for most sensory systems is the existence of parallel subcircuits and processing streams that form orderly and systematic mappings from stimulus space to neurons. Although the spatial heterogeneity of the early olfactory circuitry has long been recognized, we know comparatively little about the circuits that propagate sensory signals downstream. Investigating the potential modularity of the bulb’s intrinsic circuits proves to be a difficult task as termination patterns of converging projections, as with the bulb’s inputs, are not feasibly realized. Thus, if such circuit motifs exist, their detection essentially relies on identifying differential gene expression, or “molecular signatures,” that may demarcate functional subregions. With the arrival of comprehensive (whole genome, cellular resolution) datasets in biology and neuroscience, it is now possible for us to carry out large-scale investigations and make particular use of the densely catalogued, whole genome expression maps of the Allen Brain Atlas to carry out systematic investigations of the molecular topography of the olfactory bulb’s intrinsic circuits. To address the challenges associated with high-throughput and high-dimensional datasets, a deep learning approach will form the backbone of our informatic pipeline. In the proposed work, we test the hypothesis that the bulb’s intrinsic circuits are parceled into distinct, parallel modules that can be defined by genome-wide patterns of expression. In pursuit of this aim, our deep learning framework will facilitate the group-registration of the mitral cell layers of ~ 50,000 in-situ olfactory bulb circuits to test this hypothesis

Neuromorphic Engineering Editors' Pick 2021

This collection showcases well-received spontaneous articles from the past couple of years, which have been specially handpicked by our Chief Editors, Profs. André van Schaik and Bernabé Linares-Barranco. The work presented here highlights the broad diversity of research performed across the section and aims to put a spotlight on the main areas of interest. All research presented here displays strong advances in theory, experiment, and methodology with applications to compelling problems. This collection aims to further support Frontiers’ strong community by recognizing highly deserving authors

Development of the multi-scale dynamic modeling techniques for multi-modal brain recordings

Fundamental principles underlying computation in multi-scale brain networks illustrate how multiple brain areas and their coordinated activity give rise to complex cognitive functions. Whereas brain activity has been studied in the micro- to meso-scale in studying the connections between the dynamical patterns and the behaviors, the investigation of the neural population dynamics is mainly limited by the single-scale analysis. My goal is to develop a multi-scale dynamical model for the collective activity of neuronal populations. First, I introduced a bio-inspired deep learning approach, termed NeuroBondGraph Network (NBGNet), to capture cross-scale dynamics that can infer and map the neural data from multiple scales (Chapter 2). The NBGNet not only exhibits more than an 11-fold improvement in reconstruction accuracy, but also predicts synchronous neural activity and preserves correlated low-dimensional latent dynamics. I also show that the NBGNet robustly predicts held-out data across a long time scale (two weeks) without retraining. The effective connectivity defined from the presented model agrees with the established neuroanatomical hierarchy of motor control in the literature. I then introduced the m̲ulti-s̲cale n̲eural d̲ynamics n̲eural o̲rdinary d̲ifferential e̲quation (msDyNODE) to uncover multiscale brain communications governing cognitive behaviors (Chapter 3). I demonstrated that msDyNODE successfully captured multiscale activity using both simulations and electrophysiological experiments. The msDyNODE-derived casual interactions between recording channels and scales not only aligned well with the abstraction of the hierarchical anatomy of the mammalian nervous system but also exhibited behavioral dependences. This work offers a new approach for in depth mechanistic studies on the brain where simultaneous acquisition of multi-scale neural activity is available. While the traditional neuronal tracing method serves as gold standard to assess the neural connectivity, toxicity and the limited efficiency prevent it from serving as a reliable tool to validate the multi-scale connectivity. It has been demonstrated that dendritic spines increase or become bigger after long-term potentiation while decrease or become smaller after long-term depression. Thus, we hypothesized the existence of excitatory and inhibitory connectivity can be revealed by the dynamics of dendritic spines. Since the dendritic spines can be smaller than the Abbe diffraction limit (~200 nm), we need an in-vivo super-resolution microscopy to achieve our goal. As a super-resolution imaging method, stimulated emission depletion (STED) microscopy has unraveled fine intracellular structures and provided insights into nanoscale organizations in cells. Although image resolution can be further enhanced by continuously increasing the STED-beam power, the resulting photodamage and phototoxicity are major issues for real-world applications of STED microscopy. In Chapter 4, I demonstrated that, with 50% less STED-beam power, the STED image resolution can be improved up to 1.45-fold using the separation of photons by a lifetime tuning (SPLIT) scheme combined with a deep learning-based phasor analysis algorithm termed flimGANE (f̲luorescence l̲ifetime i̲m̲aging based on a g̲enerative a̲dversarial n̲e̲twork). This work offers a new approach for STED imaging in situations where only a limited photon budget is available. In summary, I emphasized that this is the first time we are able to infer the nonlinear multiscale interactions in the brain networks by our multi-scale dynamic modeling approach. This opens a window of opportunity that multi-scale connectivity can provide a mechanistic understanding of brain computations underlying behaviors or mental states.Biomedical Engineerin

A Review of Findings from Neuroscience and Cognitive Psychology as Possible Inspiration for the Path to Artificial General Intelligence

This review aims to contribute to the quest for artificial general intelligence by examining neuroscience and cognitive psychology methods for potential inspiration. Despite the impressive advancements achieved by deep learning models in various domains, they still have shortcomings in abstract reasoning and causal understanding. Such capabilities should be ultimately integrated into artificial intelligence systems in order to surpass data-driven limitations and support decision making in a way more similar to human intelligence. This work is a vertical review that attempts a wide-ranging exploration of brain function, spanning from lower-level biological neurons, spiking neural networks, and neuronal ensembles to higher-level concepts such as brain anatomy, vector symbolic architectures, cognitive and categorization models, and cognitive architectures. The hope is that these concepts may offer insights for solutions in artificial general intelligence.Comment: 143 pages, 49 figures, 244 reference

AHA! an 'Artificial Hippocampal Algorithm' for Episodic Machine Learning

The majority of ML research concerns slow, statistical learning of i.i.d. samples from large, labelled datasets. Animals do not learn this way. An enviable characteristic of animal learning is `episodic' learning - the ability to memorise a specific experience as a composition of existing concepts, after just one experience, without provided labels. The new knowledge can then be used to distinguish between similar experiences, to generalise between classes, and to selectively consolidate to long-term memory. The Hippocampus is known to be vital to these abilities. AHA is a biologically-plausible computational model of the Hippocampus. Unlike most machine learning models, AHA is trained without external labels and uses only local credit assignment. We demonstrate AHA in a superset of the Omniglot one-shot classification benchmark. The extended benchmark covers a wider range of known hippocampal functions by testing pattern separation, completion, and recall of original input. These functions are all performed within a single configuration of the computational model. Despite these constraints, image classification results are comparable to conventional deep convolutional ANNs

Neuromodulatory effects on early visual signal processing

Understanding how the brain processes information and generates simple to complex behavior constitutes one of the core objectives in systems neuroscience. However, when studying different neural circuits, their dynamics and interactions researchers often assume fixed connectivity, overlooking a crucial factor - the effect of neuromodulators. Neuromodulators can modulate circuit activity depending on several aspects, such as different brain states or sensory contexts. Therefore, considering the modulatory effects of neuromodulators on the functionality of neural circuits is an indispensable step towards a more complete picture of the brain’s ability to process information. Generally, this issue affects all neural systems; hence this thesis tries to address this with an experimental and computational approach to resolve neuromodulatory effects on cell type-level in a well-define system, the mouse retina. In the first study, we established and applied a machine-learning-based classification algorithm to identify individual functional retinal ganglion cell types, which enabled detailed cell type-resolved analyses. We applied the classifier to newly acquired data of light-evoked retinal ganglion cell responses and successfully identified their functional types. Here, the cell type-resolved analysis revealed that a particular principle of efficient coding applies to all types in a similar way. In a second study, we focused on the issue of inter-experimental variability that can occur during the process of pooling datasets. As a result, further downstream analyses may be complicated by the subtle variations between the individual datasets. To tackle this, we proposed a theoretical framework based on an adversarial autoencoder with the objective to remove inter-experimental variability from the pooled dataset, while preserving the underlying biological signal of interest. In the last study of this thesis, we investigated the functional effects of the neuromodulator nitric oxide on the retinal output signal. To this end, we used our previously developed retinal ganglion cell type classifier to unravel type-specific effects and established a paired recording protocol to account for type-specific time-dependent effects. We found that certain retinal ganglion cell types showed adaptational type-specific changes and that nitric oxide had a distinct modulation of a particular group of retinal ganglion cells. In summary, I first present several experimental and computational methods that allow to study functional neuromodulatory effects on the retinal output signal in a cell type-resolved manner and, second, use these tools to demonstrate their feasibility to study the neuromodulator nitric oxide

Computational modelling of neural mechanisms underlying natural speech perception

Humans are highly skilled at the analysis of complex auditory scenes. In particular, the human auditory system is characterized by incredible robustness to noise and can nearly effortlessly isolate the voice of a specific talker from even the busiest of mixtures. However, neural mechanisms underlying these remarkable properties remain poorly understood. This is mainly due to the inherent complexity of speech signals and multi-stage, intricate processing performed in the human auditory system. Understanding these neural mechanisms underlying speech perception is of interest for clinical practice, brain-computer interfacing and automatic speech processing systems. In this thesis, we developed computational models characterizing neural speech processing across different stages of the human auditory pathways. In particular, we studied the active role of slow cortical oscillations in speech-in-noise comprehension through a spiking neural network model for encoding spoken sentences. The neural dynamics of the model during noisy speech encoding reflected speech comprehension of young, normal-hearing adults. The proposed theoretical model was validated by predicting the effects of non-invasive brain stimulation on speech comprehension in an experimental study involving a cohort of volunteers. Moreover, we developed a modelling framework for detecting the early, high-frequency neural response to the uninterrupted speech in non-invasive neural recordings. We applied the method to investigate top-down modulation of this response by the listener's selective attention and linguistic properties of different words from a spoken narrative. We found that in both cases, the detected responses of predominantly subcortical origin were significantly modulated, which supports the functional role of feedback, between higher- and lower levels stages of the auditory pathways, in speech perception. The proposed computational models shed light on some of the poorly understood neural mechanisms underlying speech perception. The developed methods can be readily employed in future studies involving a range of experimental paradigms beyond these considered in this thesis.Open Acces

Biophysics-based modeling and data analysis of local field potential signal

Understanding the neurophysiological mechanisms of information processing within and across brain regions has always been a fundamental and challenging topic in neuroscience. Considerable works in the brain connectome and transcriptome have laid a profound foundation for understanding brain function by its structure. At the same time, the recent advance in recording techniques allows us to probe the nonstationary brain activity from various spatial and temporal scales. However, how to effectively build the dialogue between the anatomical structure and the dynamical brain signal still needs to be solved. To tackle the problem, we explore interpreting electrophysiology signals with mechanistic models. In chapter 2 we first segregate high-coherent brain signals into different pathways and then connect their dynamics to synaptic properties. Based on a state space model of LFP generation, we explore several preprocessing methods to bias the signal to the synaptic inputs and enhance the separatability of pathway-specific contributions. The separated sources are more reliable with the preprocessing methods, especially in highly coherent states, e.g., awake running. With reliably separated pathways, we further studied their synaptic properties and explored the local directional connections in the hippocampus. The estimated synaptic time constant and pathway connection agrees with well-established anatomical studies. In chapter 3 we explore establishing a simple model to capture the impulse response of passive neurons with detailed dendritic morphology. We validate Green’s function methods based on compartmentalized models by comparing them to numerical simulations and analytical solutions on continuous neuron membrane potentials. A parameterized model based on laminar Green’s function is further developed and helps to infer the anatomical properties, like the input current distribution and cell position, from their spatiotemporal response patterns. The effect of cell position and template are examed. Based on the model of chapter 3, we use the biophysical possible impulse response profile to regularize the source separation in the frequency domain in chapter 4. The components from different frequencies are clustered according to the same latent input distributions. The source separation in better-separated frequency bins from the same pathway helps separation in other highly contaminated frequencies. The optimization is formulated as a probabilistic model to maximize the negentropy as well as spatial likelihood. Similar to dipole approximation for EEG signals, Green’s function method provides an effective approximation to capture biologically possible spatiotemporal patterns and helps to guide the separation. We validated the method on real data with optogenetic stimulation. In chapter 5 we further separate the far-field signals from the local pathway activities according to their physiological properties. We propose a pipeline to reliably separate and automatically detect far-field signal components. Based on this, a toolbox is provided to remove the EMG artifacts and assess the cleaning performance. In the free-running animals, we show that EMG artifacts shadow the high-frequency oscillatory events detection, and EMG cleaning rescues this effect. Overall, this thesis explored multiple possibilities to incorporate neurophysiology knowledge to understand and model the electrical field potential signals.Das Verständnis der neurophysiologischen Mechanismen der Informationsverarbeitung innerhalb und zwischen Gehirnregionen war schon immer ein grundlegendes und herausforderndes Thema in den Neurowissenschaften. Weitreichende Arbeiten zum Konnektom und Transkriptom des Gehirns haben eine Grundlage für das Verständnis der Gehirnfunktion gelegt. Des Weiteren ermöglicht uns der derzeitige Fortschritt in der Aufnahmetechnik, die nicht stationäre Gehirnaktivität auf verschiedenen räumlichen und zeitlichen Skalen zu untersuchen. Wie jedoch die anatomischen Strukturen und die dynamischen Gehirnsignal effektiv zusammen wirken können, muss jedoch noch gelöst werden. Um dieses Problem anzugehen, untersuchen wir die Interpretation elektrophysiologischer Signale mit mechanistischen Modellen. In Kapitel 2 trennen wir zunächst die hochkohärenten Gehirnsignale in verschiedene Leitungsbahnen und verbinden dann die Dynamik mit synaptischen Eigenschaften. Basierend auf einem Zustandsraummodell zur Erzeugung lokaler Feldpotentiale (LFP) untersuchen wir verschiedene Vorverarbeitungsmethoden, die die Signale bestmöglich auf die synaptischen Eingangsströme ausrichten und die Trennbarkeit von leitungsbahnspezifischen Beiträgen verbessert. Die Trennung der Signalquellen ist durch das Vorverarbeitungsverfahren insbesondere während hochkohärenter Verhaltenszustände (z. B. laufen im Wachzustand) zuverlässiger. Mit zuverlässig getrennten Leitungsbahnen konnten wir die entsprechenden synaptischen Eigenschaften weiter untersuchen und die lokalen gerichteten Verbindungen im Hippocampus untersuchen. Die geschätzte synaptische Zeitkonstante und die Verbindungen der Leitungsbahnen stimmen mit etablierten anatomischen Studien überein. In Kapitel 3 untersuchen wir die Erstellung eines einfachen Modells zur Beschreibung der Impulsantwort passiver Neuronen mit detaillierter dendritischer Morphologie. Wir validieren Greensche Funktionsmethoden basierend auf kompartimentierten Modellen, indem wir sie mit numerischen Simulationen und analytischen Lösungen des kontinuierlichen Membranpotentials von Neuronen vergleichen. Ein parametrisiertes Modell, das auf der laminaren Greenschen Funktion basiert, wird weiterentwickelt. Es hilft dabei, die anatomischen Eigenschaften - die Verteilung des Eingangsstroms und die Zellposition - aus ihren raumzeitlichen Reaktionsmustern abzuleiten. Die Auswirkung der Zellposition und des Templates werden untersucht. Basierend auf dem Modell aus Kapitel 3 verwenden wir in Kapitel 4 das biophysikalisch mögliche Profil der Impulsantwort, um die Quellentrennung im Frequenzbereich festzulegen. Die Komponenten verschiedener Frequenzen werden nach derselben latenten Eingangsverteilungen geclustert. Die Quellentrennung in besser getrennten Frequenzbereichen derselben Leitungsbahn hilft bei der Quelltrennung in anderen stark kontaminierten Frequenzbereichen. Die Optimierung wird als probabilistisches Modell formuliert, um sowohl die Negentropie als auch die räumliche Wahrscheinlichkeit zu maximieren. Ähnlich wie die Dipolnäherungen für EEG-Signale bietet die Greensche Funktionsmethode eine effektive Annäherung, um biologisch mögliche raumzeitliche Muster zu erfassen, und hilft, die Quellen zu trennen. Wir haben die Methode an realen Daten mit optogenetischer Stimulation validiert. Im Kapitel 5 trennen wir weiter die Fernfeldsignale von den Signalen der lokalen Leitungsbahnen nach ihren physiologischen Eigenschaften. Wir schlagen eine Methode vor, die es erlaubt, Fernfeld-Signalkomponenten zuverlässig von lokaler Aktivitaet zu trennen und automatisch zu erkennen. Es wird eine Toolbox bereitgestellt, die EMG-Artefakte entfernt und die bereinigten Signale bewertet. In Ableitungen von freilaufenden Tieren zeigen wir, dass EMG-Artefakte die Erkennung von hochfrequenten Oszillationen beeintraechtigt, aber nach der Bereinigung des EMG-Signals erkannt werden kann. Insgesamt untersucht diese Dissertation mehrere Möglichkeiten die elektrischen Feldpotentiale neuronaler Aktivität unter Einbeziehung neurophysiologischen Wissens zu modellieren und zu verstehen