Training deep neural density estimators to identify mechanistic models of neural dynamics

Mechanistic modeling in neuroscience aims to explain observed phenomena in terms of underlying causes. However, determining which model parameters agree with complex and stochastic neural data presents a significant challenge. We address this challenge with a machine learning tool which uses deep neural density estimators-- trained using model simulations-- to carry out Bayesian inference and retrieve the full space of parameters compatible with raw data or selected data features. Our method is scalable in parameters and data features, and can rapidly analyze new data after initial training. We demonstrate the power and flexibility of our approach on receptive fields, ion channels, and Hodgkin-Huxley models. We also characterize the space of circuit configurations giving rise to rhythmic activity in the crustacean stomatogastric ganglion, and use these results to derive hypotheses for underlying compensation mechanisms. Our approach will help close the gap between data-driven and theory-driven models of neural dynamics

Dynamical principles in neuroscience

Dynamical modeling of neural systems and brain functions has a history of success over the last half century. This includes, for example, the explanation and prediction of some features of neural rhythmic behaviors. Many interesting dynamical models of learning and memory based on physiological experiments have been suggested over the last two decades. Dynamical models even of consciousness now exist. Usually these models and results are based on traditional approaches and paradigms of nonlinear dynamics including dynamical chaos. Neural systems are, however, an unusual subject for nonlinear dynamics for several reasons: (i) Even the simplest neural network, with only a few neurons and synaptic connections, has an enormous number of variables and control parameters. These make neural systems adaptive and flexible, and are critical to their biological function. (ii) In contrast to traditional physical systems described by well-known basic principles, first principles governing the dynamics of neural systems are unknown. (iii) Many different neural systems exhibit similar dynamics despite having different architectures and different levels of complexity. (iv) The network architecture and connection strengths are usually not known in detail and therefore the dynamical analysis must, in some sense, be probabilistic. (v) Since nervous systems are able to organize behavior based on sensory inputs, the dynamical modeling of these systems has to explain the transformation of temporal information into combinatorial or combinatorial-temporal codes, and vice versa, for memory and recognition. In this review these problems are discussed in the context of addressing the stimulating questions: What can neuroscience learn from nonlinear dynamics, and what can nonlinear dynamics learn from neuroscience?This work was supported by NSF Grant No. NSF/EIA-0130708, and Grant No. PHY 0414174; NIH Grant No. 1 R01 NS50945 and Grant No. NS40110; MEC BFI2003-07276, and Fundación BBVA

Training deep neural density estimators to identify mechanistic models of neural dynamics

Mechanistic modeling in neuroscience aims to explain observed phenomena in terms of underlying causes. However, determining which model parameters agree with complex and stochastic neural data presents a significant challenge. We address this challenge with a machine learning tool which uses deep neural density estimators—trained using model simulations—to carry out Bayesian inference and retrieve the full space of parameters compatible with raw data or selected data features. Our method is scalable in parameters and data features and can rapidly analyze new data after initial training. We demonstrate the power and flexibility of our approach on receptive fields, ion channels, and Hodgkin–Huxley models. We also characterize the space of circuit configurations giving rise to rhythmic activity in the crustacean stomatogastric ganglion, and use these results to derive hypotheses for underlying compensation mechanisms. Our approach will help close the gap between data-driven and theory-driven models of neural dynamics

Digital neural circuits : from ions to networks

PhD ThesisThe biological neural computational mechanism is always fascinating to human beings since it shows several state-of-the-art characteristics: strong fault tolerance, high power efficiency and self-learning capability. These behaviours lead the developing trend of designing the next-generation digital computation platform. Thus investigating and understanding how the neurons talk with each other is the key to replicating these calculation features. In this work I emphasize using tailor-designed digital circuits for exactly implementing bio-realistic neural network behaviours, which can be considered a novel approach to cognitive neural computation. The first advance is that biological real-time computing performances allow the presented circuits to be readily adapted for real-time closed-loop in vitro or in vivo experiments, and the second one is a transistor-based circuit that can be directly translated into an impalpable chip for high-level neurologic disorder rehabilitations. In terms of the methodology, first I focus on designing a heterogeneous or multiple-layer-based architecture for reproducing the finest neuron activities both in voltage-and calcium-dependent ion channels. In particular, a digital optoelectronic neuron is developed as a case study. Second, I focus on designing a network-on-chip architecture for implementing a very large-scale neural network (e.g. more than 100,000) with human cognitive functions (e.g. timing control mechanism). Finally, I present a reliable hybrid bio-silicon closed-loop system for central pattern generator prosthetics, which can be considered as a framework for digital neural circuit-based neuro-prosthesis implications. At the end, I present the general digital neural circuit design principles and the long-term social impacts of the presented work

Flexibility vs consistency: Quantifying differences in neuromodulatory elicited patterns of activity

Central pattern generating circuits underly fundamental behaviors such as respiration or locomotion and are under the influence of neuromodulators. The presence of neuromodulators is thought to confer flexibility to these circuits to generate distinct patterns of activity to meet distinct behavioral needs. Network output flexibility can be achieved by distinct classes of neuromodulators, those which have convergent cellular actions but divergent circuit actions or by those which have divergent cellular actions but convergent circuit actions. Both classes of neuromodulator exist in the stomatogastric nervous system of the crab Cancer borealis and influence the activity of a central pattern generating circuit in the stomatogastric ganglion, the pyloric network. The ability of both classes of neuromodulator, when applied individually, to generate qualitatively and quantitatively distinct patterns of activity has been demonstrated with respect to a baseline activity state. While it is assumed that each individual neuromodulator’s activity pattern is distinct, there has yet to be a fully quantitative description of the degree of difference between two modulated activity patterns. It is also unlikely that any single circuit will be under the influence of only a single neuromodulator at any point. Therefore, the possibility of generating distinct network outputs increases with each distinct combination of neuromodulators. While the actions of individual neuromodulators have been explored, the consequences of co-modulation on the pyloric network’s output are less understood. Previous attempts at quantifying the effects of a neuromodulator on the pyloric network output relied on evaluating only a single, often multi-dimensional, attribute of activity at a time and statistically testing the dependent parameters of that attribute with statistics that assume independence. This dissertation uses a new approach to quantify and statistically test how different one neuromodulator elicited pattern of activity is from another, preserving the inherent multi-dimensional nature of the attributes evaluated. The results of this dissertation show that the pyloric network output is able to generate statistically distinct network outputs with individual neuromodulators; however, flexibility is lost in favor of consistency under co-modulatory conditions

The molecular underpinnings of neuronal cell identity in the stomatogastric ganglion of cancer borealis

Throughout the life of an organism, the nervous system must be able to balance changing in response to environmental stimuli with the need to produce reliable, repeatable activity patterns to create stereotyped behaviors. Understanding the mechanisms responsible for this regulation requires a wealth of knowledge about the neural system, ranging from network connectivity and cell type identification to intrinsic neuronal excitability and transcriptomic expression. To make strides in this area, we have employed the well-described stomatogastric nervous system of the Jonah crab Cancer borealis to examine the molecular underpinnings and regulation of neuron cell identity. Several crustacean circuits, including the stomatogastric nervous system and the cardiac ganglion, continue to provide important new insights into circuit dynamics and modulation (Diehl, White, Stein, & Nusbaum, 2013; Marder, 2012; Marder & Bucher, 2007; Williams et al., 2013), but this work has been partially hampered by the lack of extensive molecular sequence knowledge in crustaceans. Here we generated de novo transcriptome assembly from central nervous system tissue for C. borealis producing 42,766 contigs, focusing on an initial identification, curation, and comparison of genes that will have the most profound impact on our understanding of circuit function in these species. This included genes for 34 distinct ion channel types, 17 biogenic amine and 5 GABA receptors, 28 major transmitter receptor subtypes including glutamate and acetylcholine receptors, and 6 gap junction proteins -- the Innexins. ... With this reference transcriptome and annotated sequences in hand, we sought to determine the strengths and limitations of using the neuronal molecular profile to classify them into cell types. ... Since the resulting activity of a neuron is the product of the expression of ion channel genes, we sought to further probe the expression profile of neurons across a range of cell types to understand how these patterns of mRNA abundance relate to the properties of individual cell types. ... Finally, we sought to better understand the molecular underpinnings of how these correlated patterns of mRNA expression are generated and maintained.Includes bibliographical reference

Mecanismos de codificación y procesamiento de información en redes basadas en firmas neuronales

Tesis doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Tecnología Electrónica y de las Comunicaciones. Fecha de lectura: 21-02-202

Determining how stable network oscillations arise from neuronal and synaptic mechanisms

Many animal behaviors involve the generation of rhythmic patterns and movements. These rhythmic patterns are commonly mediated by neural networks that produce an oscillatory activity pattern, where different neurons maintain a relative phase relationship. This thesis examines the relationships between the cellular and synaptic properties that give rise to stable activity in the form of phase maintenance, across different frequencies in a well-suited model system, the pyloric network of the crab Cancer borealis. The pyloric network has endogenously oscillating ‘pacemaker’ neurons that inhibit ‘follower’ neurons, which in turn feed back onto the pacemaker neurons. The focus of this thesis was to determine the methods by which phase maintenance is achieved in an oscillatory network. This thesis examines the idea that phase maintenance occurs through the actions of intrinsic properties of isolated neurons or through the dynamics of their synaptic connections or both. A combination of pharmacological and electrophysiological techniques a used to show how identified membrane properties and short-term synaptic plasticity are involved with phase maintenance over a range of biologically relevant oscillation frequencies. To examine whether network stability is due to the characteristic stable activity of the identified pyloric neuron types, the hypothesis that phase maintenance is an inherent property of synaptically-isolated individual neurons in the pyloric network was first tested. A set of parameters were determined (frequency-dependent activity profile) to define the response of each isolated pyloric neuron to sinusoidal input at different frequencies. The parameters that define the activity profile are: burst onset phase, burst end phase, resonance frequency and intra-burst spike frequency. Each pyloric neuron type was found to possess a unique activity profile, indicating that the individual neuron types are tuned to produce a particular activity pattern at different frequencies depending on their role in the network. To elucidate the biophysical properties underlying the frequency-dependent activity profiles of the neurons, the hyperpolarization activated current (Ih) was measured and found to possess frequency-dependent properties. This implies that Ih has a different influence on the activity phase of pyloric neurons at different frequencies. Additionally, it was found that the Ih contribution to the burst onset phase depends on the neuron type: in the pacemaker group neurons (PD) it had no influence on the burst onset phase at any frequency whereas in follower neurons it acted to advance the onset phase in one neuron type (LP) and, paradoxically, to delay it in a different neuron type (PY). The results from this part of the study provided evidence that stability is due in part to the intrinsic neuronal properties but that these intrinsic properties do not fully explain network stability. To address the contribution of pyloric synapses to network stability, the mechanisms by which synapses promote phase maintenance were investigated. An artificial synapse that mimicked the feedforward PD to LP synapse, was used so that the synaptic parameters could be varied in a controlled manner in order to examine the influence of the properties of this synapse on the postsynaptic LP neuron. It was found that a static synapse with fixed parameters (such as strength and peak phase) across frequencies cannot result in a constant activity phase in the LP neuron. However, if the synaptic strength decreases and the peak phase is delayed as a function of frequency, the LP neuron can maintain a constant activity phase across a large range of frequencies. These dynamic changes in the strength and peak phase of the PD to LP synapse are consistent with the short-term plasticity properties previously reported for this synapse. In the pyloric network, the follower neuron LP provides the sole transmitter-mediated feedback to the pacemaker neurons. To understand the role of this synapse in network stability, this synapse was blocked and replaced by an artificial synapse using the dynamic clamp technique. Different parameters of the artificial synapse, including strength, peak phase, duration and onset phase were found to affect the pyloric cycle period. The most effective parameters that influence cycle period were the synaptic duration and its onset phase. Overall this study demonstrated that both the intrinsic properties of individual neurons and the dynamic properties of the synapses are essential in producing stable activity phases in this oscillatory network. The insight obtained from this thesis can provide a general understanding of the contribution of intrinsic properties to neuronal activity phase and how short-term synaptic dynamics can act to promote phase maintenance in oscillatory networks

Homeostatic compensation and neuromodulation maintain synchronized motor neuron activity in the crustacean cardiac ganglion

Dissertation supervisor: Dr. David J. Schulz.Includes vita.Animals rely on the nervous system to produce appropriate behavior throughout their lives. In sending commands to the musculature for rhythmic motor behaviors such as breathing or walking, neural networks must be stable enough to send a reliable level of drive with the proper temporal coordination. Networks must also be flexible enough to meet changing environmental demands. A network's output ultimately arises from the intrinsic excitability of its constituent neurons and the synaptic connections between them. Interestingly, neurons and networks are able to produce highly conserved output from highly variable underlying intrinsic and synaptic properties. To explore the consequences of this variability, we have used the crustacean cardiac ganglion (CG) which consists of 9 neurons: 4 pacemaker cells that give excitatory input to 5 Large Cell motor neurons (LCs) which are responsible for driving the simultaneous contraction of the musculature that makes up the walls of the animal's single-chambered heart (Alexandrowicz, 1934; Hartline, 1967; Anderson and Cooke, 1971). The intact network can be dissected from the animal in physiological saline and it continues to produce robust, reliable, and rhythmic output (Welsh and Maynard, 1951; Cooke, 2002). LCs have virtually identical synchronized activity, but their intrinsic ionic conductances can be highly variable (Ransdell et al., 2013a). In Chapter 1, we exploit this variability by pharmacologically blocking a subset of their conductances to make LCs hyperexcitable and desynchronize their activity. We find that homeostatic compensation restores synchronized activity and excitability within one hour. This happens via two synergistic mechanisms: the membrane properties of each cell are re-tuned to converge on similar voltage activity, and increased conductance of the gap junctions between the cells helps to buffer away differences in their voltage activity. A separate but related study asked whether naturalistic perturbations of network activity would also result in desynchronization. Neuromodulation provides flexibility in the output of neural networks by altering a subset of their conductances. We hypothesized that this could also cause desynchronization. We found that modulation with serotonin and dopamine both increased the excitability of the CG. Interestingly, serotonin desynchronized the CG, but dopamine did not. We found that dopaminergic modulation directly increases gap junctional conductance. By co-applying these modulators, we found dopamine was able to prevent serotonin from desynchronizing the network without occluding its effects. It was also able to prevent the desynchronization caused by ion channel blockers. Finally, to fully understand the output of LCs, we must recognize that their activity arises not only from their intrinsic properties, but also from their synaptic drive from pacemaker cells. To address how variable this can be from one animal to the next, we analyze the activity of 131 animals taken over the course of approximately 5 years. We use this to address the fundamental question of how variable networks underlying a particular behavior can be across animals. We recognize two distinct classes of pacemaker inputs to LCs, and characterize bursting patterns for both types of pacemaker spike and LC output. We conclude that LCs from different animals receive different temporal patterns of pacemaker drive, which may have important functional implications. We also compare animals from winter and summer months, and find that temperature-independent seasonal effects may explain some of the variance in our data.Includes bibliographical references