Neuromodulation of Neuromorphic Circuits

We present a novel methodology to enable control of a neuromorphic circuit in close analogy with the physiological neuromodulation of a single neuron. The methodology is general in that it only relies on a parallel interconnection of elementary voltage-controlled current sources. In contrast to controlling a nonlinear circuit through the parameter tuning of a state-space model, our approach is purely input-output. The circuit elements are controlled and interconnected to shape the current-voltage characteristics (I-V curves) of the circuit in prescribed timescales. In turn, shaping those I-V curves determines the excitability properties of the circuit. We show that this methodology enables both robust and accurate control of the circuit behavior and resembles the biophysical mechanisms of neuromodulation. As a proof of concept, we simulate a SPICE model composed of MOSFET transconductance amplifiers operating in the weak inversion regime.The research leading to these results has received funding from the European Research Council under the Advanced ERC Grant Agreement Switchlet n.67064

A bio-inspired bistable recurrent cell allows for long-lasting memory

Recurrent neural networks (RNNs) provide state-of-the-art performances in a wide variety of tasks that require memory. These performances can often be achieved thanks to gated recurrent cells such as gated recurrent units (GRU) and long short-term memory (LSTM). Standard gated cells share a layer internal state to store information at the network level, and long term memory is shaped by network-wide recurrent connection weights. Biological neurons on the other hand are capable of holding information at the cellular level for an arbitrary long amount of time through a process called bistability. Through bistability, cells can stabilize to different stable states depending on their own past state and inputs, which permits the durable storing of past information in neuron state. In this work, we take inspiration from biological neuron bistability to embed RNNs with long-lasting memory at the cellular level. This leads to the introduction of a new bistable biologically-inspired recurrent cell that is shown to strongly improves RNN performance on time-series which require very long memory, despite using only cellular connections (all recurrent connections are from neurons to themselves, i.e. a neuron state is not influenced by the state of other neurons). Furthermore, equipping this cell with recurrent neuromodulation permits to link them to standard GRU cells, taking a step towards the biological plausibility of GRU

The geometry of rest–spike bistability

Funder: Qualcomm; doi: http://dx.doi.org/10.13039/100005144Abstract: Morris–Lecar model is arguably the simplest dynamical model that retains both the slow–fast geometry of excitable phase portraits and the physiological interpretation of a conductance-based model. We augment this model with one slow inward current to capture the additional property of bistability between a resting state and a spiking limit cycle for a range of input current. The resulting dynamical system is a core structure for many dynamical phenomena such as slow spiking and bursting. We show how the proposed model combines physiological interpretation and mathematical tractability and we discuss the benefits of the proposed approach with respect to alternative models in the literature

Synthesis of neuromorphic circuits with neuromodulatory properties

The field of neuromorphic engineering shows great promise in delivering novel devices inspired by biological principles that would undertake sensory and processing tasks with an unprecedented level of efficiency. In order to achieve that, engineers are required to understand and implement the many complex biological regulatory mechanisms that allow the nervous system to robustly operate and adapt over scales covering many orders of magnitude, while at the same time using unreliable and noisy components. As a step towards that, this thesis aims at discussing and implementing the principles of neuromodulation in neuromorphic hardware, mechanisms which allow neurons to change and regulate their behaviour through the continuous control of their internal currents. We discuss how neural dynamics and its modulation can be broken down into four essential feedback loops, and we introduce a simplified model of the neural membrane respecting this fundamental structure. We present a novel methodology for controlling the neuron's behaviour through the shaping of its I-V curves in distinct timescales, thus characterising the behaviour of the neural circuit through its input-output properties. We show how modulation of the feedback loops affects the behaviour, and importantly, captures the transition between spiking and bursting oscillatory regimes, two major signalling modes of neurons. We then show how the architecture can be easily implemented using well-known neuromorphic building blocks based on subthreshold MOSFET circuits. Finally, we discuss how the excitability switch captured by the model can be exploited in simple network settings, thus opening up the possibility for future research into novel architectures where the control of cellular properties is utilised to shape the global behaviour of the network

Nonlinear resonance and excitability in interconnected systems

Engineering design amounts to develop components and interconnect them to obtain a desired behaviour. While in the context of equilibrium dynamics there is a well-developed theory that can account for robustness and optimality in this process, we still lack a corresponding methodology for nonequilibrium dynamics and in particular oscillatory behaviours. With the aim of fostering such a theory, this thesis studies two basic interconnections in the contexts of nonlinear resonance and excitability, two phenomena with the potential of encompassing a large number of applications. The first interconnection is considered in the context of vibration absorption. It corresponds to coupling two Duffing oscillators, the prototypical example of nonlinear resonator. Of primary interest is the frequency response of the system, which quantifies the behaviour in presence of harmonic forces. The analysis focuses on how isolated families of solutions appear and merge with a main one. Using singularity theory it is possible to organise these solutions in the space of parameters and delimit their presence through numerical methods. The second interconnection studied in this dissertation appears in the context of excitable circuits. Combining a fast excitable system and a slower oscillatory system that share a similar structure naturally leads to bursting. The resulting system has a slow-fast structure that can be leveraged in the analysis. The first step of this analysis is a novel slow-fast model of bistability between a rest state and a spiking attractor. Following this, the analysis moves to the complete interconnection, and in particular on how it can generate different patterns of bursting activity

Neuromodulation of Neuromorphic Circuits

We present a novel methodology to enable control of a neuromorphic circuit in close analogy with the physiological neuromodulation of a single neuron. The methodology is general in that it only relies on a parallel interconnection of elementary voltage-controlled current sources. In contrast to controlling a nonlinear circuit through the parameter tuning of a state-space model, our approach is purely input-output. The circuit elements are controlled and interconnected to shape the current-voltage characteristics (I-V curves) of the circuit in prescribed timescales. In turn, shaping those I-V curves determines the excitability properties of the circuit. We show that this methodology enables both robust and accurate control of the circuit behavior and resembles the biophysical mechanisms of neuromodulation. As a proof of concept, we simulate a SPICE model composed of MOSFET transconductance amplifiers operating in the weak inversion regime.The research leading to these results has received funding from the European Research Council under the Advanced ERC Grant Agreement Switchlet n.67064

Neuromodulation of Neuromorphic Circuits

We present a novel methodology to enable control of a neuromorphic circuit in close analogy with the physiological neuromodulation of a single neuron. The methodology is general in that it only relies on a parallel interconnection of elementary voltage-controlled current sources. In contrast to controlling a nonlinear circuit through the parameter tuning of a state-space model, our approach is purely input-output. The circuit elements are controlled and interconnected to shape the current-voltage characteristics (I-V curves) of the circuit in prescribed timescales. In turn, shaping those I-V curves determines the excitability properties of the circuit. We show that this methodology enables both robust and accurate control of the circuit behavior and resembles the biophysical mechanisms of neuromodulation. As a proof of concept, we simulate a SPICE model composed of MOSFET transconductance amplifiers operating in the weak inversion regime