H∞ state estimation for discrete-time memristive recurrent neural networks with stochastic time-delays

This paper deals with the robust (Formula presented.) state estimation problem for a class of memristive recurrent neural networks with stochastic time-delays. The stochastic time-delays under consideration are governed by a Bernoulli-distributed stochastic sequence. The purpose of the addressed problem is to design the robust state estimator such that the dynamics of the estimation error is exponentially stable in the mean square, and the prescribed (Formula presented.) performance constraint is met. By utilizing the difference inclusion theory and choosing a proper Lyapunov–Krasovskii functional, the existence condition of the desired estimator is derived. Based on it, the explicit expression of the estimator gain is given in terms of the solution to a linear matrix inequality. Finally, a numerical example is employed to demonstrate the effectiveness and applicability of the proposed estimation approach.This work was supported in part by the National Natural Science Foundation of China [grant number 61329301], [grant number 61134009], [grant number 61473076], [grant number 61503001]; the Shu Guang project of Shanghai Municipal Education Commission and Shanghai Education Development Foundation [grant number 13SG34]; the Natural Science Foundation of Universities in Anhui Province [grant number KJ2015A088], [grant number TSKJ2015B17]; the Fundamental Research Funds for the Central Universities, the DHU Distinguished Young Professor Program, and the Alexander von Humboldt Foundation of Germany

Lag Synchronization of Switched Neural Networks via Neural Activation Function and Applications in Image Encryption

© 2015 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.This paper investigates the problem of global exponential lag synchronization of a class of switched neural networks with time-varying delays via neural activation function and applications in image encryption. The controller is dependent on the output of the system in the case of packed circuits, since it is hard to measure the inner state of the circuits. Thus, it is critical to design the controller based on the neuron activation function. Comparing the results, in this paper, with the existing ones shows that we improve and generalize the results derived in the previous literature. Several examples are also given to illustrate the effectiveness and potential applications in image encryption

Avalanches and the edge-of-chaos in neuromorphic nanowire networks

The brain's efficient information processing is enabled by the interplay between its neuro-synaptic elements and complex network structure. This work reports on the neuromorphic dynamics of nanowire networks (NWNs), a brain-inspired system with synapse-like memristive junctions embedded within a recurrent neural network-like structure. Simulation and experiment elucidate how collective memristive switching gives rise to long-range transport pathways, drastically altering the network's global state via a discontinuous phase transition. The spatio-temporal properties of switching dynamics are found to be consistent with avalanches displaying power-law size and life-time distributions, with exponents obeying the crackling noise relationship, thus satisfying criteria for criticality. Furthermore, NWNs adaptively respond to time varying stimuli, exhibiting diverse dynamics tunable from order to chaos. Dynamical states at the edge-of-chaos are found to optimise information processing for increasingly complex learning tasks. Overall, these results reveal a rich repertoire of emergent, collective dynamics in NWNs which may be harnessed in novel, brain-inspired computing approaches

Dynamics meets Morphology: towards Dymorph Computation

In this dissertation, approaches are presented for both technically using and investigating biological principles with oscillators in the context of electrical engineering, in particular neuromorphic engineering. Thereby, dynamics as well as morphology as important neuronal principles were explicitly selected, which shape the information processing in the human brain and distinguish it from other technical systems. The aspects and principles selected here are adaptation during the encoding of stimuli, the comparatively low signal transmission speed, the continuous formation and elimination of connections, and highly complex, partly chaotic, dynamics. The selection of these phenomena and properties has led to the development of a sensory unit that is capable of encoding mechanical stress into a series of voltage pulses by the use of a MOSFET augmented by AlScN. The circuit is based on a leaky integrate and fire neuron model and features an adaptation of the pulse frequency. Furthermore, the slow signal transmission speed of biological systems was the motivation for the investigation of a temporal delay in the feedback of the output pulses of a relaxation oscillator. In this system stable pulse patterns which form due to so-called jittering bifurcations could be observed. In particular, switching between different stable pulse patterns was possible to induce. In the further course of the work, the first steps towards time-varying coupling of dynamic systems are investigated. It was shown that in a system consisting of dimethyl sulfoxid and zinc acetate, oscillators can be used to force the formation of filaments. The resulting filaments then lead to a change in the dynamics of the oscillators. Finally, it is shown that in a system with chaotic dynamics, the extension of it with a memristive device can lead to a transient stabilisation of the dynamics, a behaviour that can be identified as a repeated pass of Hopf bifurcations

Memristor based neural networks: Feasibility, theories and approaches

Memristor-based neural networks refer to the utilisation of memristors, the newly emerged nanoscale devices, in building neural networks. The memristor was first postulated by Leon Chua in 1971 as the fourth fundamental passive circuit element and experimentally validated by one of HP labs in 2008. Memristors, short for memory-resistor, have a peculiar memory effect which distinguishes them from resistors. By applying a bias voltage across it, the resistance of a memristor, namely memristance, is changed. In addition, the memristance is retained when the power supply is removed which demonstrates the non-volatility of the memristor. Memristor-based neural networks are currently being researched in order to replace complementary metal-oxide-semiconductor (CMOS) devices in neuromorphic circuits with memristors and to investigate their potential applications. Current research primarily focuses on the utilisation of memristors as synaptic connections between neurons, however in any application it may be possible to allow memristors to perform computation in a natural way which attempts to avoid additional CMOS devices. Examples of such methods utilised in neural networks are presented in this thesis, such as memristor-based cellular neural network (CNN) structures, the memristive spiking-time dependent plasticity (STDP) model and the exploration of their potential applications. This thesis presents manifold studies in the topic of memristor-based neural networks from theories and feasibility to approaches to implementations. Studies are divided into two parts which are the utilisation of memristors in non-spiking neural networks and spiking neural networks (SNNs). At the beginning of the thesis, fundamentals of neural networks and memristors are explored with the analysis of the physical properties and $v-i$ behaviour of memristors. In the studies of memristor-based non-spiking neural networks, a staircase memristor model is presented based on memristors which have multi-level resistive states and the delayed-switching effect. This model is adapted to CNNs and echo state networks (ESNs) as applications that benefit from memristive implementations. In the studies of memristor-based SNNs, a trace-based memristive STDP model is proposed and discussed to overcome the incompatibility issues of the previous model with all-to-all spike interaction. The work also presents applications of the trace-based memristive model in associative learning with retention loss and supervised learning. The computational results of experiments with different applications have shown that memristor-based neural networks will be advantageous in building synchronous or asynchronous parallel neuromorphic systems. The work presents several new findings on memristor modelling, memristor-based neural network structures and memristor-based associative learning. These studies address unexplored research areas in the context of memristor-based neural networks to the best of our knowledge, and therefore form original contributions

Analog Spiking Neuromorphic Circuits and Systems for Brain- and Nanotechnology-Inspired Cognitive Computing

Human society is now facing grand challenges to satisfy the growing demand for computing power, at the same time, sustain energy consumption. By the end of CMOS technology scaling, innovations are required to tackle the challenges in a radically different way. Inspired by the emerging understanding of the computing occurring in a brain and nanotechnology-enabled biological plausible synaptic plasticity, neuromorphic computing architectures are being investigated. Such a neuromorphic chip that combines CMOS analog spiking neurons and nanoscale resistive random-access memory (RRAM) using as electronics synapses can provide massive neural network parallelism, high density and online learning capability, and hence, paves the path towards a promising solution to future energy-efficient real-time computing systems. However, existing silicon neuron approaches are designed to faithfully reproduce biological neuron dynamics, and hence they are incompatible with the RRAM synapses, or require extensive peripheral circuitry to modulate a synapse, and are thus deficient in learning capability. As a result, they eliminate most of the density advantages gained by the adoption of nanoscale devices, and fail to realize a functional computing system. This dissertation describes novel hardware architectures and neuron circuit designs that synergistically assemble the fundamental and significant elements for brain-inspired computing. Versatile CMOS spiking neurons that combine integrate-and-fire, passive dense RRAM synapses drive capability, dynamic biasing for adaptive power consumption, in situ spike-timing dependent plasticity (STDP) and competitive learning in compact integrated circuit modules are presented. Real-world pattern learning and recognition tasks using the proposed architecture were demonstrated with circuit-level simulations. A test chip was implemented and fabricated to verify the proposed CMOS neuron and hardware architecture, and the subsequent chip measurement results successfully proved the idea. The work described in this dissertation realizes a key building block for large-scale integration of spiking neural network hardware, and then, serves as a step-stone for the building of next-generation energy-efficient brain-inspired cognitive computing systems

単層カーボンナノチューブ/ポルフィリン-ポリ酸ランダムネットワークを用いたマテリアルリザバー演算素子 —次世代機械知能への新規アプローチ

In a layman’s term, computation is defined as the execution of a given instruction through a programmable algorithm. History has it that starting from the simplest calculator to the sophisticated von Neumann machine, the above definition has been followed without a flaw. Logical operations for which a human takes a minute long to solve, is a matter of fraction of seconds for these gadgets. But contrastingly, when it comes to critical and analytical thinking that requires learning through observation like the human brain, these powerful machines falter and lag behind. Thus, inspired from the brain’s neural circuit, software models of neural networks (NN) integrated with high-speed supercomputers were developed as an alternative tool to implement machine intelligent tasks of function optimization, pattern, and voice recognition. But as device downscaling and transistor performance approaches the constant regime of Moore’s law due to high CMOS fabrication cost and large tunneling energy loss, training these algorithms over multiple hidden layers is turning out to be a grave concern for future applications. As a result, the interplay between faster performance and low computational power requirement for complex tasks deems highly disproportional. Therefore, alternative in terms of both NN models and conventional Neumann architecture needs to be addressed in today’s age for next-generation machine intelligence systems. Fortunately, through extensive research and studies, unconventional computing using a reservoir based neural network platform, called in-materio reservoir computing (RC) has come to the rescue. In-maerio RC uses physical, biological, chemical, cellular automata and other inanimate dynamical systems as a source of non-linear high dimensional spatio-temporal information processing unit to construct a specific target task. RC not only has a three-layer simplified neural architectural layer, but also imposes a cheap, fast, and simplified optimization of only the readout weights with machine intelligent regression algorithm to construct the supervised objective target via a weighted linear combination of the readouts. Thus, utilizing this idea, herein in this work we report such an in-materio RC with a dynamical random network of single walled carbon nanotube/porphyrin-polyoxometalate (SWNT/Por-POM) device. We begin with Chapter 1, which deals with the introduction covering the literature of ANN evolution and the shortcomings of von Neumann architecture and training models of these ANN, which leads us to adopt the in-materio RC architecture. We design the problem statement focused on extending the theoretical RC model of previously suggested SWNT/POM network to an experimental one and present the objective of fabricating a random network based on nanomaterials as they closely resemble the network structure of the brain. Finally, we conclude by stating the scope of this research work aiming towards validating the non-linear high dimensional reservoir property SWNT/Por-POM holds for it to explicitly demonstrate the RC benchmark tasks of optimization and classification. Chapter 2 describes the methodology including the chemical repository required for the facile synthesis of the material. The synthesis part is divided broadly into SWNT purification and then its dispersion with Por-POM to form the desired complex. It is then followed up with the microelectrode array fabrication and the consequent wet-transfer thin film deposition to give the ultimate reservoir architecture of input-output control read pads with SWNT/Por-POM reservoir. Finally we give a briefing of AFM, UV-Vis spectroscopy, FE-SEM characterization techniques of SWNT/Por-POM complex along with the electrical set-up interfaced with software algorithm to demonstrate the RC approach of in-materio machine intelligence. In Chapter 3, we study the current dynamics as a function of voltage and time and validate the non-linear information processing ability intrinsic to the device. The study reveals that the negative differential resistance (NDR) arising from redox nature of Por-POM results in oscillating random noise outputs giving rise to 1/f brain-like spatio-temporal information. We compute the memory capacity (MC) and prove that the device exhibits echo state property of fading memory, but remembers very little of the past information. The low MC and high non-linearity allowed us to choose mostly non-linear tasks of waveform generation, Boolean logic optimization and one-hot vector binary object classification as the RC benchmark. The Chapter 4 relates to the waveform generation task. Utilizing the high dimensional voltage readouts of varying amplitude, phase and higher harmonic frequencies, relative to input sine wave, a regression optimization was performed towards constructing cosine, triangular, square and sawtooth waves resulting in a high accuracy of around 95%. The task complexity of function optimization was further enhanced in Chapter 5 where two inputs were used to construct Boolean logic functions of OR, AND, XOR, NOR, NAND and XNOR. Similar to the waveform, accuracy over 95% could be achieved due to the presence of NDR nonlinearity. Furthermore, the device was also tested for classification problem in Chapter 6. Here we showed an off-line binary classification of four object toys; hedgehog, dog, block and bus, using the grasped tactile information of these objects as inputs obtained from the Toyota Human Support Robot. A one-ridge regression analysis to fit the hot vector supervised target was used to optimize the output weights for predicting the correct outcome. All the objects were successfully classified owing to the 1/f information processing factor. Lastly, we conclude the section in Chapter 7 with the future scope of extending the idea to fabricate a 3-D model of the same material as it opens up opportunity for higher memory capacity fruitful for future benchmark tasks of time-series prediction. Overall, our research marks a step stone in utilizing SWNT/Por-POM as the in-materio RC for the very first time thereby making it a desirable candidate for next-generation machine intelligence.九州工業大学博士学位論文 学位記番号：生工博甲第425号 学位授与年月日：令和3年12月27日1 Introduction and Literature review|2 Methodology|3 Reservoir dynamics emerging from an incidental structure of single-walled carbon nanotube/porphyrin-polyoxometalate complex|4 Fourier transform waveforms via in-materio reservoir computing from single-walled carbon nanotube/porphyrin-polyoxometalate complex|5 Room temperature demonstration of in-materio reservoir computing for optimizing Boolean function with single-walled carbon nanotube/porphyrin-polyoxometalate composite|6 Binary object classification with tactile sensory input information of via single-walled carbon nanotube/porphyrin-polyoxometalate network as in-materio reservoir computing|7 Future scope and Conclusion九州工業大学令和3年