Generalized reconfigurable memristive dynamical system (MDS) for neuromorphic applications

This study firstly presents (i) a novel general cellular mapping scheme for two dimensional neuromorphic dynamical systems such as bio-inspired neuron models, and (ii) an efficient mixed analog-digital circuit, which can be conveniently implemented on a hybrid memristor-crossbar/CMOS platform, for hardware implementation of the scheme. This approach employs 4n memristors and no switch for implementing an n-cell system in comparison with 2n2 memristors and 2n switches of a Cellular Memristive Dynamical System (CMDS). Moreover, this approach allows for dynamical variables with both analog and one-hot digital values opening a wide range of choices for interconnections and networking schemes. Dynamical response analyses show that this circuit exhibits various responses based on the underlying bifurcation scenarios which determine the main characteristics of the neuromorphic dynamical systems. Due to high programmability of the circuit, it can be applied to a variety of learning systems, real-time applications, and analytically indescribable dynamical systems. We simulate the FitzHugh-Nagumo (FHN), Adaptive Exponential (AdEx) integrate and fire, and Izhikevich neuron models on our platform, and investigate the dynamical behaviors of these circuits as case studies. Moreover, error analysis shows that our approach is suitably accurate. We also develop a simple hardware prototype for experimental demonstration of our approach.Unión Europea H2020 ECOMODE project under grant agreement 604102Unión Europea HBP project under grant number FP7-ICT-2013-FET-F-60410

Hardware-Amenable Structural Learning for Spike-based Pattern Classification using a Simple Model of Active Dendrites

This paper presents a spike-based model which employs neurons with functionally distinct dendritic compartments for classifying high dimensional binary patterns. The synaptic inputs arriving on each dendritic subunit are nonlinearly processed before being linearly integrated at the soma, giving the neuron a capacity to perform a large number of input-output mappings. The model utilizes sparse synaptic connectivity; where each synapse takes a binary value. The optimal connection pattern of a neuron is learned by using a simple hardware-friendly, margin enhancing learning algorithm inspired by the mechanism of structural plasticity in biological neurons. The learning algorithm groups correlated synaptic inputs on the same dendritic branch. Since the learning results in modified connection patterns, it can be incorporated into current event-based neuromorphic systems with little overhead. This work also presents a branch-specific spike-based version of this structural plasticity rule. The proposed model is evaluated on benchmark binary classification problems and its performance is compared against that achieved using Support Vector Machine (SVM) and Extreme Learning Machine (ELM) techniques. Our proposed method attains comparable performance while utilizing 10 to 50% less computational resources than the other reported techniques.Comment: Accepted for publication in Neural Computatio

Novel design methods of central nervous system of C. elegans and olfactory bulb model of mammal based on sequential logic and numerical integration

This study proposes a novel design method of a neuromorphic electronic circuit: design of a neuromorphic circuit based on appropriately selected hybrid dynamics of synchronous sequential logic, asynchronous sequential logic, and numerical integration. Based on the proposed design method, a novel central nervous system model of C. elegans, and an olfactory bulb model are presented. It is then shown that the presented models can realize typical responses of a conventional central nervous system model of C. elegans, and the observation of chaos in the olfactory bulb. Furthermore, the presented models are implemented by a field programmable gate array and the presented model of C.elegans is used to control a prototype robot of C. elegans body. Then, experiments validate that the presented central nervous system model enables the body robot to reproduce typical chemotaxis behaviors of the conventional C. elegans model. In addition, comparisons show that the presented model consumes fewer circuit elements and lower power compared to various central nervous system models of C. elegans based on synchronous sequential logic, asynchronous sequential logic, and numerical integration

Dimensions of Timescales in Neuromorphic Computing Systems

This article is a public deliverable of the EU project "Memory technologies with multi-scale time constants for neuromorphic architectures" (MeMScales, https://memscales.eu, Call ICT-06-2019 Unconventional Nanoelectronics, project number 871371). This arXiv version is a verbatim copy of the deliverable report, with administrative information stripped. It collects a wide and varied assortment of phenomena, models, research themes and algorithmic techniques that are connected with timescale phenomena in the fields of computational neuroscience, mathematics, machine learning and computer science, with a bias toward aspects that are relevant for neuromorphic engineering. It turns out that this theme is very rich indeed and spreads out in many directions which defy a unified treatment. We collected several dozens of sub-themes, each of which has been investigated in specialized settings (in the neurosciences, mathematics, computer science and machine learning) and has been documented in its own body of literature. The more we dived into this diversity, the more it became clear that our first effort to compose a survey must remain sketchy and partial. We conclude with a list of insights distilled from this survey which give general guidelines for the design of future neuromorphic systems

Influence of Autapses on Synchronization in Neural Networks With Chemical Synapses

A great deal of research has been devoted on the investigation of neural dynamics in various network topologies. However, only a few studies have focused on the influence of autapses, synapses from a neuron onto itself via closed loops, on neural synchronization. Here, we build a random network with adaptive exponential integrate-and-fire neurons coupled with chemical synapses, equipped with autapses, to study the effect of the latter on synchronous behavior. We consider time delay in the conductance of the pre-synaptic neuron for excitatory and inhibitory connections. Interestingly, in neural networks consisting of both excitatory and inhibitory neurons, we uncover that synchronous behavior depends on their synapse type. Our results provide evidence on the synchronous and desynchronous activities that emerge in random neural networks with chemical, inhibitory and excitatory synapses where neurons are equipped with autapses. © Copyright © 2020 Protachevicz, Iarosz, Caldas, Antonopoulos, Batista and Kurths

Dynamics and precursor signs for phase transitions in neural systems

This thesis investigates neural state transitions associated with sleep, seizure and anaesthesia. The aim is to address the question: How does a brain traverse the critical threshold between distinct cortical states, both healthy and pathological? Specifically we are interested in sub-threshold neural behaviour immediately prior to state transition. We use theoretical neural modelling (single spiking neurons, a network of these, and a mean-field continuum limit) and in vitro experiments to address this question. Dynamically realistic equations of motion for thalamic relay neuron, reticular nuclei, cortical pyramidal and cortical interneuron in different vigilance states are developed, based on the Izhikevich spiking neuron model. A network of cortical neurons is assembled to examine the behaviour of the gamma-producing cortical network and its transition to lower frequencies due to effect of anaesthesia. Then a three-neuron model for the thalamocortical loop for sleep spindles is presented. Numerical simulations of these networks confirms spiking consistent with reported in vivo measurement results, and provides supporting evidence for precursor indicators of imminent phase transition due to occurrence of individual spindles. To complement the spiking neuron networks, we study the Wilson–Cowan neural mass equations describing homogeneous cortical columns and a 1D spatial cluster of such columns. The abstract representation of cortical tissue by a pair of coupled integro-differential equations permits thorough linear stability, phase plane and bifurcation analyses. This model shows a rich set of spatial and temporal bifurcations marking the boundary to state transitions: saddle-node, Hopf, Turing, and mixed Hopf–Turing. Close to state transition, white-noise-induced subthreshold fluctuations show clear signs of critical slowing down with prolongation and strengthening of autocorrelations, both in time and space, irrespective of bifurcation type. Attempts at in vitro capture of these predicted leading indicators form the last part of the thesis. We recorded local field potentials (LFPs) from cortical and hippocampal slices of mouse brain. State transition is marked by the emergence and cessation of spontaneous seizure-like events (SLEs) induced by bathing the slices in an artificial cerebral spinal fluid containing no magnesium ions. Phase-plane analysis of the LFP time-series suggests that distinct bifurcation classes can be responsible for state change to seizure. Increased variance and growth of spectral power at low frequencies (f < 15 Hz) was observed in LFP recordings prior to initiation of some SLEs. In addition we demonstrated prolongation of electrically evoked potentials in cortical tissue, while forwarding the slice to a seizing regime. The results offer the possibility of capturing leading temporal indicators prior to seizure generation, with potential consequences for understanding epileptogenesis. Guided by dynamical systems theory this thesis captures evidence for precursor signs of phase transitions in neural systems using mathematical and computer-based modelling as well as in vitro experiments

LTS and FS Inhibitory Interneurons, Short-Term Synaptic Plasticity, and Cortical Circuit Dynamics

Somatostatin-expressing, low threshold-spiking (LTS) cells and fast-spiking (FS) cells are two common subtypes of inhibitory neocortical interneuron. Excitatory synapses from regular-spiking (RS) pyramidal neurons to LTS cells strongly facilitate when activated repetitively, whereas RS-to-FS synapses depress. This suggests that LTS neurons may be especially relevant at high rate regimes and protect cortical circuits against over-excitation and seizures. However, the inhibitory synapses from LTS cells usually depress, which may reduce their effectiveness at high rates. We ask: by which mechanisms and at what firing rates do LTS neurons control the activity of cortical circuits responding to thalamic input, and how is control by LTS neurons different from that of FS neurons? We study rate models of circuits that include RS cells and LTS and FS inhibitory cells with short-term synaptic plasticity. LTS neurons shift the RS firing-rate vs. current curve to the right at high rates and reduce its slope at low rates; the LTS effect is delayed and prolonged. FS neurons always shift the curve to the right and affect RS firing transiently. In an RS-LTS-FS network, FS neurons reach a quiescent state if they receive weak input, LTS neurons are quiescent if RS neurons receive weak input, and both FS and RS populations are active if they both receive large inputs. In general, FS neurons tend to follow the spiking of RS neurons much more closely than LTS neurons. A novel type of facilitation-induced slow oscillations is observed above the LTS firing threshold with a frequency determined by the time scale of recovery from facilitation. To conclude, contrary to earlier proposals, LTS neurons affect the transient and steady state responses of cortical circuits over a range of firing rates, not only during the high rate regime; LTS neurons protect against over-activation about as well as FS neurons

Investigating the role of fast-spiking interneurons in neocortical dynamics

PhD ThesisFast-spiking interneurons are the largest interneuronal population in neocortex. It is well documented that this population is crucial in many functions of the neocortex by subserving all aspects of neural computation, like gain control, and by enabling dynamic phenomena, like the generation of high frequency oscillations. Fast-spiking interneurons, which represent mainly the parvalbumin-expressing, soma-targeting basket cells, are also implicated in pathological dynamics, like the propagation of seizures or the impaired coordination of activity in schizophrenia. In the present thesis, I investigate the role of fast-spiking interneurons in such dynamic phenomena by using computational and experimental techniques. First, I introduce a neural mass model of the neocortical microcircuit featuring divisive inhibition, a gain control mechanism, which is thought to be delivered mainly by the soma-targeting interneurons. Its dynamics were analysed at the onset of chaos and during the phenomena of entrainment and long-range synchronization. It is demonstrated that the mechanism of divisive inhibition reduces the sensitivity of the network to parameter changes and enhances the stability and exibility of oscillations. Next, in vitro electrophysiology was used to investigate the propagation of activity in the network of electrically coupled fast-spiking interneurons. Experimental evidence suggests that these interneurons and their gap junctions are involved in the propagation of seizures. Using multi-electrode array recordings and optogenetics, I investigated the possibility of such propagating activity under the conditions of raised extracellular K+ concentration which applies during seizures. Propagated activity was recorded and the involvement of gap junctions was con rmed by pharmacological manipulations. Finally, the interaction between two oscillations was investigated. Two oscillations with di erent frequencies were induced in cortical slices by directly activating the pyramidal cells using optogenetics. Their interaction suggested the possibility of a coincidence detection mechanism at the circuit level. Pharmacological manipulations were used to explore the role of the inhibitory interneurons during this phenomenon. The results, however, showed that the observed phenomenon was not a result of synaptic activity. Nevertheless, the experiments provided some insights about the excitability of the tissue through scattered light while using optogenetics. This investigation provides new insights into the role of fast-spiking interneurons in the neocortex. In particular, it is suggested that the gain control mechanism is important for the physiological oscillatory dynamics of the network and that the gap junctions between these interneurons can potentially contribute to the inhibitory restraint during a seizure.Wellcome Trust