Existence and exponential stability of solutions for quaternion-valued delayed hopfield neural networks by ξ-Norms

© 2013 IEEE. Recently, with the development of quaternion applications, quaternion-valued neural networks (QVNNs) have been presented and studied by more and more scholars. In this paper, the existence, uniqueness and exponential stability criteria of solutions for the quaternion-valued delayed Hopfield neural networks (QVDHNNs) are mainly investigated by means of the definitions of ξ-norms. In order to construct a ξ-norm, QVDHNNs system are decomposed into four real-number systems according to Hamilton rules. Then, taking advantage of ξ-norms, inequality technique and Cauchy's test for convergence, time-invariant delays and time-varying delays are considered successively to derive ξ-exponential type sufficient conditions. Based on these, several corollaries about the existence, uniqueness and exponential stability of solutions are obtained. Finally, two numerical examples with time-invariant delays and time-varying delays are given respectively. Their simulated images illustrate the effectiveness of the main theoretical results

Dynamical Systems in Spiking Neuromorphic Hardware

Dynamical systems are universal computers. They can perceive stimuli, remember, learn from feedback, plan sequences of actions, and coordinate complex behavioural responses. The Neural Engineering Framework (NEF) provides a general recipe to formulate models of such systems as coupled sets of nonlinear differential equations and compile them onto recurrently connected spiking neural networks – akin to a programming language for spiking models of computation. The Nengo software ecosystem supports the NEF and compiles such models onto neuromorphic hardware. In this thesis, we analyze the theory driving the success of the NEF, and expose several core principles underpinning its correctness, scalability, completeness, robustness, and extensibility. We also derive novel theoretical extensions to the framework that enable it to far more effectively leverage a wide variety of dynamics in digital hardware, and to exploit the device-level physics in analog hardware. At the same time, we propose a novel set of spiking algorithms that recruit an optimal nonlinear encoding of time, which we call the Delay Network (DN). Backpropagation across stacked layers of DNs dramatically outperforms stacked Long Short-Term Memory (LSTM) networks—a state-of-the-art deep recurrent architecture—in accuracy and training time, on a continuous-time memory task, and a chaotic time-series prediction benchmark. The basic component of this network is shown to function on state-of-the-art spiking neuromorphic hardware including Braindrop and Loihi. This implementation approaches the energy-efficiency of the human brain in the former case, and the precision of conventional computation in the latter case

ψ-type stability of reaction–diffusion neural networks with time-varying discrete delays and bounded distributed delays

In this paper, the ψ-type stability and robust ψ-type stability for reaction–diffusion neural networks (RDNNs) with Dirichlet boundary conditions, time-varying discrete delays and bounded distributed delays are investigated, respectively. Firstly, we analyze the ψ-type stability and robust ψ-type stability of RDNNs with time-varying discrete delays by means of ψ-type functions combined with some inequality techniques, and put forward several ψ-type stability criteria for the considered networks. Additionally, the models of RDNNs with bounded distributed delays are established and some sufficient conditions to guarantee the ψ-type stability and robust ψ-type stability are given. Lastly, two examples are provided to confirm the effectiveness of the derived results

Exponential Stability Analysis of Mixed Delayed Quaternion-Valued Neural Networks Via Decomposed Approach

© 2013 IEEE. With the application of quaternion in technology, quaternion-valued neural networks (QVNNs) have attracted many scholars' attention in recent years. For the existing results, dynamical behavior is an important studying side. In this paper, we mainly research the existence, uniqueness and exponential stability criteria of solutions for the QVNNs with discrete time-varying delays and distributed delays by means of generalized 2-norm. In order to avoid the noncommutativity of quaternion multiplication, the QVDNN system is firstly decomposed into four real-number systems by Hamilton rules. Then, we obtain the sufficient criteria for the existence, uniqueness and exponential stability of solutions by special Lyapunov-type functional, Cauchy convergence principle and monotone function. Furthermore, several corollaries are derived from the main results. Finally, we give one numerical example and its simulated figures to illustrate the effectiveness of the obtained conclusion

Breaking Implicit Assumptions of Physical Delay-Feedback Reservoir Computing

The Reservoir Computing (RC) paradigm is a supervised machine learning scheme using the natural computational ability of dynamical systems. Such dynamical systems incorporate time delays showcasing intricate dynamics. This richness in dynamics, particularly the system's transient response to external stimuli makes them suitable for RC. A subset of RCs, Delay-Feedback Reservoir Computing (DFRC), is distinguished by its unique features: a system that consists of a single nonlinear node and a delay-line, with `virtual' nodes defined along the delay-line by time-multiplexing procedure of the input. These characteristics render DFRC particularly useful for hardware integration. In this thesis, the aim is to break the implicit assumptions made in the design of physical DFRC based on Mackey-Glass dynamical system. The first assumption we address is the performance of DFRC is not affected by the attenuation in physcial delay-line as the nodes defined along it are 'virtual'. However, our experimental results contradict this. To mitigate the impact of losses along the delay line, we propose a methodology `Devirtualisation', which describes the procedure of directly tapping into the delay lines at the position of a `virtual' node, rather than at the delay line's end. It trade-offs the DFRC system's read-out frequency and the quantity of output lines. Masking plays a crucial role in DFRC, as it defines `virtual' nodes along the delay-line. The second assumption is that the mask used should randomly generated numbers uniformly distributed between [-u,u]. We experimentally compare Binary Weight Mask (BWM) vs. Random Weight Mask (RWM) under different scenarios; and investigate the randomness of BWM signal distribution's impact. The third implicit assumption is that, DFRC is designed to solve time series prediction tasks involving a single input and output with no external feedback. To break this assumption, we propose two approaches to mix multi-input signals into DFRC; to validate these approaches, a novel task for DFRC that inherently necessitates multiple inputs: the control of a forced Van der Pol oscillator system, is proposed

Backpropagation for Continuous Theta Neuron Networks

The Theta neuron model is a spiking neuron model which, unlike traditional Leaky-Integrate-and-Fire neurons, can model spike latencies, threshold adaptation, bistability of resting and tonic firing states, and more. Previous work on learning rules for networks of theta neurons includes the derivation of a spike-timing based backpropagation algorithm for multilayer feedforward networks. However, this learning rule is only applicable to a fixed number of spikes per neuron, and is unable to take into account the effects of synaptic dynamics. In this thesis a novel backpropagation learning rule for theta neuron networks is derived which incorporates synaptic dynamics, is applicable to changing numbers of spikes per neuron, and does not explicitly depend on spike-timing. The learning rule is successfully applied to XOR, cosine and sinc function mappings, and comparisons between other learning rules for spiking neural networks are made. The algorithm achieves 97.8 percent training performance and 96.7 percent test performance on the Fischer-Iris dataset, which is comparable to other spiking neural network learning rules. The algorithm also achieves 99.0 percent training performance and 99.14 percent test performance on the Wisconsin Breast Cancer dataset, which is better than the compared spiking neural network learning rules

Neuromorphic Engineering Editors' Pick 2021

This collection showcases well-received spontaneous articles from the past couple of years, which have been specially handpicked by our Chief Editors, Profs. André van Schaik and Bernabé Linares-Barranco. The work presented here highlights the broad diversity of research performed across the section and aims to put a spotlight on the main areas of interest. All research presented here displays strong advances in theory, experiment, and methodology with applications to compelling problems. This collection aims to further support Frontiers’ strong community by recognizing highly deserving authors

A Practical Investigation into Achieving Bio-Plausibility in Evo-Devo Neural Microcircuits Feasible in an FPGA

Many researchers has conjectured, argued, or in some cases demonstrated, that bio-plausibility can bring about emergent properties such as adaptability, scalability, fault-tolerance, self-repair, reliability, and autonomy to bio-inspired intelligent systems. Evolutionary-developmental (evo-devo) spiking neural networks are a very bio-plausible mixture of such bio-inspired intelligent systems that have been proposed and studied by a few researchers. However, the general trend is that the complexity and thus the computational cost grow with the bio-plausibility of the system. FPGAs (Field- Programmable Gate Arrays) have been used and proved to be one of the flexible and cost efficient hardware platforms for research' and development of such evo-devo systems. However, mapping a bio-plausible evo-devo spiking neural network to an FPGA is a daunting task full of different constraints and trade-offs that makes it, if not infeasible, very challenging. This thesis explores the challenges, trade-offs, constraints, practical issues, and some possible approaches in achieving bio-plausibility in creating evolutionary developmental spiking neural microcircuits in an FPGA through a practical investigation along with a series of case studies. In this study, the system performance, cost, reliability, scalability, availability, and design and testing time and complexity are defined as measures for feasibility of a system and structural accuracy and consistency with the current knowledge in biology as measures for bio-plausibility. Investigation of the challenges starts with the hardware platform selection and then neuron, cortex, and evo-devo models and integration of these models into a whole bio-inspired intelligent system are examined one by one. For further practical investigation, a new PLAQIF Digital Neuron model, a novel Cortex model, and a new multicellular LGRN evo-devo model are designed, implemented and tested as case studies. Results and their implications for the researchers, designers of such systems, and FPGA manufacturers are discussed and concluded in form of general trends, trade-offs, suggestions, and recommendations