A generative spike train model with time-structured higher order correlations

Emerging technologies are revealing the spiking activity in ever larger neural ensembles. Frequently, this spiking is far from independent, with correlations in the spike times of different cells. Understanding how such correlations impact the dynamics and function of neural ensembles remains an important open problem. Here we describe a new, generative model for correlated spike trains that can exhibit many of the features observed in data. Extending prior work in mathematical finance, this generalized thinning and shift (GTaS) model creates marginally Poisson spike trains with diverse temporal correlation structures. We give several examples which highlight the model's flexibility and utility. For instance, we use it to examine how a neural network responds to highly structured patterns of inputs. We then show that the GTaS model is analytically tractable, and derive cumulant densities of all orders in terms of model parameters. The GTaS framework can therefore be an important tool in the experimental and theoretical exploration of neural dynamics

Characterization and Compensation of Network-Level Anomalies in Mixed-Signal Neuromorphic Modeling Platforms

Advancing the size and complexity of neural network models leads to an ever increasing demand for computational resources for their simulation. Neuromorphic devices offer a number of advantages over conventional computing architectures, such as high emulation speed or low power consumption, but this usually comes at the price of reduced configurability and precision. In this article, we investigate the consequences of several such factors that are common to neuromorphic devices, more specifically limited hardware resources, limited parameter configurability and parameter variations. Our final aim is to provide an array of methods for coping with such inevitable distortion mechanisms. As a platform for testing our proposed strategies, we use an executable system specification (ESS) of the BrainScaleS neuromorphic system, which has been designed as a universal emulation back-end for neuroscientific modeling. We address the most essential limitations of this device in detail and study their effects on three prototypical benchmark network models within a well-defined, systematic workflow. For each network model, we start by defining quantifiable functionality measures by which we then assess the effects of typical hardware-specific distortion mechanisms, both in idealized software simulations and on the ESS. For those effects that cause unacceptable deviations from the original network dynamics, we suggest generic compensation mechanisms and demonstrate their effectiveness. Both the suggested workflow and the investigated compensation mechanisms are largely back-end independent and do not require additional hardware configurability beyond the one required to emulate the benchmark networks in the first place. We hereby provide a generic methodological environment for configurable neuromorphic devices that are targeted at emulating large-scale, functional neural networks

Network dynamics in the neural control of birdsong

Sequences of stereotyped actions are central to the everyday lives of humans and animals, from the kingfisher's dive to the performance of a piano concerto. Lashley asked how neural circuits managed this feat nearly 6 decades ago, and to this day it remains a fundamental question in neuroscience. Toward answering this question, vocal performance in the songbird was used as a model to study the performance of learned, stereotyped motor sequences. The first component of this work considers the song motor cortical zone HVC in the zebra finch, an area that sends precise timing signals to both the descending motor pathway, responsible for stereotyped vocal performance in the adult, and the basal ganglia, which is responsible for both motor variability and song learning. Despite intense interest in HVC, previous research has exclusively focused on describing the activity of small numbers of neurons recorded serially as the bird sings. To better understand HVC network dynamics, both single units and local field potentials were sampled across multiple electrodes simultaneously in awake behaving zebra finches. The local field potential and spiking data reveal a stereotyped spatio-temporal pattern of inhibition operating on a 30 ms time-scale that coordinates the neural sequences in principal cells underlying song. The second component addresses the resilience of the song circuit through cutting the motor cortical zone HVC in half along one axis. Despite this large-scale perturbation, the finch quickly recovers and sings a near-perfect song within a single day. These first two studies suggest that HVC is functionally organized to robustly generate neural dynamics that enable vocal performance. The final component concerns a statistical study of the complex, flexible songs of the domesticated canary. This study revealed that canary song is characterized by specific long-range correlations up to 7 seconds long-a time-scale more typical of human music than animal vocalizations. Thus, the neural sequences underlying birdsong must be capable of generating more structure and complexity than previously thought

Temporal Structure in Zebra Finch Song: Implications for the Motor Code and Learning Process

One of the touchstone questions in neuroscience is how the nervous system encodes complex behavioral sequences such as speech. With experience-dependent learning, well-defined anatomy and complex temporal organization, zebra finch song has served as an excellent model system for these questions. Male songs are learned from older males during a sensitive period that includes song memorization and vocal learning guided by auditory feedback. Once learned, song acoustics are hierarchically organized into syllables, continuous stretches of vocalization separated by silent gaps, which are arranged into stereotyped sequences termed motifs; on a finer scale, syllables are composed of one or more notes, vocalizations with a homogenous spectral profile. Although much is known about the song system, progress has been limited by conflicting data on the neural basis of the acoustic hierarchy and the role this organization plays during learning: While behavioral and electrophysiological studies have suggested separate circuits and learning stages for individual syllables and syllable sequence, these models have been challenged by physiological evidence that songs are actually driven by a clock-like mechanism that does not segment songs into different units. We have analyzed and modeled trial-to-trial timing variability in zebra finch song acoustics to investigate whether the hierarchy is in fact represented in the song system and learning process. Using automated template matching and dynamic time warping, we made millisecond-precise timing measurements in tens of thousands of recordings of both adult and juvenile song. In each adult song, we find rendition-to-rendition tempo variability that is spread across syllables and gaps; however syllable lengths stretch and compress with tempo changes proportionally less than gaps, \ie\ they are less ``elastic." Such non-uniformity is at odds with the simplest clock-based model in which songs are driven by a timing mechanism that paces song evenly across syllable-gap sequences. On the other hand, in a subsequent analysis we factored out tempo changes and used the remaining variability to investigate subsyllabic timescales that contradict the hierarchical model as well. Here, we find length variability that is specific to 10-msec song slices and independent of neighboring vocalization, yet correlated across motifs, providing the first behavioral evidence for a 5-10 msec timescale of song representation and an interaction with a neuromodulatory source operating on a much slower timescale. We have developed a model of song production constrained by the timing data; modeling suggests that adult song may be produced by an underlying chain of activity on a single 5-10 msec timescale, but with properties such as synaptic strength that do correspond to the acoustic hierarchy. Finally, we analyzed juvenile song within the same framework and investigated how the timing properties we modeled may develop during sensorimotor learning. The behavioral data indicate a period towards the end of learning in which syllable sequences become more stereotyped, tempo increases selectively among gaps, and independent timing variability falls two- to threefold across syllables and gaps. In remarkable contrast, over this same period we find no changes in patterns of global tempo variability or the fine timescale patterns indicative of chaining mechanisms. Overall, the developmental data suggest a final phase of song learning in which syllable-based representations are consolidated into the longer sequence-based chaining mechanisms proposed for the adult system. A similar process of linking simpler chains to form more functional activity patterns has been proposed for neocortex and other models of sequence learning in mammalian systems. In this respect, adult zebra finch song representations may be most analogous with procedural memory and overlearned sequences such as repetitive speech patterns

Functional Brain Oscillations: How Oscillations Facilitate Information Representation and Code Memories

The overall aim of the modelling works within this thesis is to lend theoretical evidence to empirical findings from the brain oscillations literature. We therefore hope to solidify and expand the notion that precise spike timing through oscillatory mechanisms facilitates communication, learning, information processing and information representation within the brain. The primary hypothesis of this thesis is that it can be shown computationally that neural de-synchronisations can allow information content to emerge. We do this using two neural network models, the first of which shows how differential rates of neuronal firing can indicate when a single item is being actively represented. The second model expands this notion by creating a complimentary timing mechanism, thus enabling the emergence of qualitive temporal information when a pattern of items is being actively represented. The secondary hypothesis of this thesis is that it can be also be shown computationally that oscillations might play a functional role in learning. Both of the models presented within this thesis propose a sparsely coded and fast learning hippocampal region that engages in the binding of novel episodic information. The first model demonstrates how active cortical representations enable learning to occur in their hippocampal counterparts via a phase-dependent learning rule. The second model expands this notion, creating hierarchical temporal sequences to encode the relative temporal position of cortical representations. We demonstrate in both of these models, how cortical brain oscillations might provide a gating function to the representation of information, whilst complimentary hippocampal oscillations might provide distinct phasic reference points for learning

Order-Based Representation in Random Networks of Cortical Neurons

The wide range of time scales involved in neural excitability and synaptic transmission might lead to ongoing change in the temporal structure of responses to recurring stimulus presentations on a trial-to-trial basis. This is probably the most severe biophysical constraint on putative time-based primitives of stimulus representation in neuronal networks. Here we show that in spontaneously developing large-scale random networks of cortical neurons in vitro the order in which neurons are recruited following each stimulus is a naturally emerging representation primitive that is invariant to significant temporal changes in spike times. With a relatively small number of randomly sampled neurons, the information about stimulus position is fully retrievable from the recruitment order. The effective connectivity that makes order-based representation invariant to time warping is characterized by the existence of stations through which activity is required to pass in order to propagate further into the network. This study uncovers a simple invariant in a noisy biological network in vitro; its applicability under in vivo constraints remains to be seen