Stochastic IMT (insulator-metal-transition) neurons: An interplay of thermal and threshold noise at bifurcation

Artificial neural networks can harness stochasticity in multiple ways to enable a vast class of computationally powerful models. Electronic implementation of such stochastic networks is currently limited to addition of algorithmic noise to digital machines which is inherently inefficient; albeit recent efforts to harness physical noise in devices for stochasticity have shown promise. To succeed in fabricating electronic neuromorphic networks we need experimental evidence of devices with measurable and controllable stochasticity which is complemented with the development of reliable statistical models of such observed stochasticity. Current research literature has sparse evidence of the former and a complete lack of the latter. This motivates the current article where we demonstrate a stochastic neuron using an insulator-metal-transition (IMT) device, based on electrically induced phase-transition, in series with a tunable resistance. We show that an IMT neuron has dynamics similar to a piecewise linear FitzHugh-Nagumo (FHN) neuron and incorporates all characteristics of a spiking neuron in the device phenomena. We experimentally demonstrate spontaneous stochastic spiking along with electrically controllable firing probabilities using Vanadium Dioxide (VO$_2$) based IMT neurons which show a sigmoid-like transfer function. The stochastic spiking is explained by two noise sources - thermal noise and threshold fluctuations, which act as precursors of bifurcation. As such, the IMT neuron is modeled as an Ornstein-Uhlenbeck (OU) process with a fluctuating boundary resulting in transfer curves that closely match experiments. As one of the first comprehensive studies of a stochastic neuron hardware and its statistical properties, this article would enable efficient implementation of a large class of neuro-mimetic networks and algorithms.Comment: Added sectioning, Figure 6, Table 1, and Section II.E Updated abstract, discussion and corrected typo

Stochastic IMT (insulator-metal-transition) neurons: An interplay of thermal and threshold noise at bifurcation

Artificial neural networks can harness stochasticity in multiple ways to enable a vast class of computationally powerful models. Electronic implementation of such stochastic networks is currently limited to addition of algorithmic noise to digital machines which is inherently inefficient; albeit recent efforts to harness physical noise in devices for stochasticity have shown promise. To succeed in fabricating electronic neuromorphic networks we need experimental evidence of devices with measurable and controllable stochasticity which is complemented with the development of reliable statistical models of such observed stochasticity. Current research literature has sparse evidence of the former and a complete lack of the latter. This motivates the current article where we demonstrate a stochastic neuron using an insulator-metal-transition (IMT) device, based on electrically induced phase-transition, in series with a tunable resistance. We show that an IMT neuron has dynamics similar to a piecewise linear FitzHugh-Nagumo (FHN) neuron and incorporates all characteristics of a spiking neuron in the device phenomena. We experimentally demonstrate spontaneous stochastic spiking along with electrically controllable firing probabilities using Vanadium Dioxide (VO$_2$) based IMT neurons which show a sigmoid-like transfer function. The stochastic spiking is explained by two noise sources - thermal noise and threshold fluctuations, which act as precursors of bifurcation. As such, the IMT neuron is modeled as an Ornstein-Uhlenbeck (OU) process with a fluctuating boundary resulting in transfer curves that closely match experiments. As one of the first comprehensive studies of a stochastic neuron hardware and its statistical properties, this article would enable efficient implementation of a large class of neuro-mimetic networks and algorithms.Comment: Added sectioning, Figure 6, Table 1, and Section II.E Updated abstract, discussion and corrected typo

A differential memristive synapse circuit for on-line learning in neuromorphic computing systems

Spike-based learning with memristive devices in neuromorphic computing architectures typically uses learning circuits that require overlapping pulses from pre- and post-synaptic nodes. This imposes severe constraints on the length of the pulses transmitted in the network, and on the network's throughput. Furthermore, most of these circuits do not decouple the currents flowing through memristive devices from the one stimulating the target neuron. This can be a problem when using devices with high conductance values, because of the resulting large currents. In this paper we propose a novel circuit that decouples the current produced by the memristive device from the one used to stimulate the post-synaptic neuron, by using a novel differential scheme based on the Gilbert normalizer circuit. We show how this circuit is useful for reducing the effect of variability in the memristive devices, and how it is ideally suited for spike-based learning mechanisms that do not require overlapping pre- and post-synaptic pulses. We demonstrate the features of the proposed synapse circuit with SPICE simulations, and validate its learning properties with high-level behavioral network simulations which use a stochastic gradient descent learning rule in two classification tasks.Comment: 18 Pages main text, 9 pages of supplementary text, 19 figures. Patente

Synthesis of neuromorphic circuits with neuromodulatory properties

The field of neuromorphic engineering shows great promise in delivering novel devices inspired by biological principles that would undertake sensory and processing tasks with an unprecedented level of efficiency. In order to achieve that, engineers are required to understand and implement the many complex biological regulatory mechanisms that allow the nervous system to robustly operate and adapt over scales covering many orders of magnitude, while at the same time using unreliable and noisy components. As a step towards that, this thesis aims at discussing and implementing the principles of neuromodulation in neuromorphic hardware, mechanisms which allow neurons to change and regulate their behaviour through the continuous control of their internal currents. We discuss how neural dynamics and its modulation can be broken down into four essential feedback loops, and we introduce a simplified model of the neural membrane respecting this fundamental structure. We present a novel methodology for controlling the neuron's behaviour through the shaping of its I-V curves in distinct timescales, thus characterising the behaviour of the neural circuit through its input-output properties. We show how modulation of the feedback loops affects the behaviour, and importantly, captures the transition between spiking and bursting oscillatory regimes, two major signalling modes of neurons. We then show how the architecture can be easily implemented using well-known neuromorphic building blocks based on subthreshold MOSFET circuits. Finally, we discuss how the excitability switch captured by the model can be exploited in simple network settings, thus opening up the possibility for future research into novel architectures where the control of cellular properties is utilised to shape the global behaviour of the network

Neuromodulation of Neuromorphic Circuits

We present a novel methodology to enable control of a neuromorphic circuit in close analogy with the physiological neuromodulation of a single neuron. The methodology is general in that it only relies on a parallel interconnection of elementary voltage-controlled current sources. In contrast to controlling a nonlinear circuit through the parameter tuning of a state-space model, our approach is purely input-output. The circuit elements are controlled and interconnected to shape the current-voltage characteristics (I-V curves) of the circuit in prescribed timescales. In turn, shaping those I-V curves determines the excitability properties of the circuit. We show that this methodology enables both robust and accurate control of the circuit behavior and resembles the biophysical mechanisms of neuromodulation. As a proof of concept, we simulate a SPICE model composed of MOSFET transconductance amplifiers operating in the weak inversion regime.The research leading to these results has received funding from the European Research Council under the Advanced ERC Grant Agreement Switchlet n.67064

Modeling Emerging Semiconductor Devices for Circuit Simulation

Circuit simulation is an indispensable part of modern IC design. The significant cost of fabrication has driven researchers to verify the chip functionality through simulation before submitting the design for final fabrication. With the impending end of Moore’s Law, researchers all over the world are looking for new devices with enhanced functionality. A plethora of promising emerging devices has been proposed in recent years. In order to leverage the full potential of such devices, circuit designers need fast, reliable models for SPICE simulation to explore different applications. Most of these new devices have complex underlying physical mechanism rendering the model development an extremely challenging task. For the models to be of practical use, they have to enable fast and accurate simulation that rules out the possibility of numerically solving a system of partial differential equations to arrive at a solution. In this chapter, we show how different modeling approaches can be used to simulate three emerging semiconductor devices namely, silicon- on- insulator four gate transistor(G4FET), perimeter gated single photon avalanche diode (PG-SPAD) and insulator-metal transistor (IMT) device with volatile memristance. All the models have been verified against experimental /TCAD data and implemented in commercial circuit simulator

Chemical Bionics - a novel design approach using ion sensitive field effect transistors

In the late 1980s Carver Mead introduced Neuromorphic engineering in which various aspects of the neural systems of the body were modelled using VLSI1 circuits. As a result most bio-inspired systems to date concentrate on modelling the electrical behaviour of neural systems such as the eyes, ears and brain. The reality is however that biological systems rely on chemical as well as electrical principles in order to function. This thesis introduces chemical bionics in which the chemically-dependent physiology of specific cells in the body is implemented for the development of novel bio-inspired therapeutic devices. The glucose dependent pancreatic beta cell is shown to be one such cell, that is designed and fabricated to form the first silicon metabolic cell. By replicating the bursting behaviour of biological beta cells, which respond to changes in blood glucose, a bio-inspired prosthetic for glucose homeostasis of Type I diabetes is demonstrated. To compliment this, research to further develop the Ion Sensitive Field Effect Transistor (ISFET) on unmodified CMOS is also presented for use as a monolithic sensor for chemical bionic systems. Problems arising by using the native passivation of CMOS as a sensing surface are described and methods of compensation are presented. A model for the operation of the device in weak inversion is also proposed for exploitation of its physical primitives to make novel monolithic solutions. Functional implementations in various technologies is also detailed to allow future implementations chemical bionic circuits. Finally the ISFET integrate and fire neuron, which is the first of its kind, is presented to be used as a chemical based building block for many existing neuromorphic circuits. As an example of this a chemical imager is described for spatio-temporal monitoring of chemical species and an acid base discriminator for monitoring changes in concentration around a fixed threshold is also proposed