Information processing using a single dynamical node as complex system

L. Appeltant... et al.Novel methods for information processing are highly desired in our information-driven society. Inspired by the brain's ability to process information, the recently introduced paradigm known as 'reservoir computing' shows that complex networks can efficiently perform computation. Here we introduce a novel architecture that reduces the usually required large number of elements to a single nonlinear node with delayed feedback. Through an electronic implementation, we experimentally and numerically demonstrate excellent performance in a speech recognition benchmark. Complementary numerical studies also show excellent performance for a time series prediction benchmark. These results prove that delay-dynamical systems, even in their simplest manifestation, can perform efficient information processing. This finding paves the way to feasible and resource-efficient technological implementations of reservoir computing.This research was partially supported by the Belgian Science Policy Office, under grant IAP P6-10 'photonics@be', by FWO and FRS–FNRS (Belgium), MICINN (Spain) under projects FISICOS (FIS2007-60327) and DeCoDicA (TEC2009-14101) and by the European project PHOCUS (EU FET-Open grant: 240763). L.A. and G.VdS. are a PhD Fellow and a Postdoctoral Fellow of the Research Foundation-Flanders (FWO).Peer reviewe

Reservoir computing based on delay-dynamical systems

Today, except for mathematical operations, our brain functions much faster and more efficient than any supercomputer. It is precisely this form of information processing in neural networks that inspires researchers to create systems that mimic the brain’s information processing capabilities. In this thesis we propose a novel approach to implement these alternative computer architectures, based on delayed feedback. We show that one single nonlinear node with delayed feedback can replace a large network of nonlinear nodes. First we numerically investigate the architecture and performance of delayed feedback systems as information processing units. Then we elaborate on electronic and opto-electronic implementations of the concept. Next to evaluating their performance for standard benchmarks, we also study task independent properties of the system, extracting information on how to further improve the initial scheme. Finally, some simple modifications are suggested, yielding improvements in terms of speed or performanc

Reservoir computing using a delayed feedback system: towards photonic implementations

Delayed feedback systems are known to exhibit a rich dynamical behavior, showing a wide variety of dynamical regimes. We use this richness to implement reservoir computing, a processing concept in machine learning. In this paper we demonstrate the proof of principle on an electronic system, however the approach is readily transferable to photonics, promising fast and computationally efficient all-optical processing. Using only one single node with delayed feedback instead of an entire network of nodes, we succeed in obtaining state-of-the-art results on benchmarks such as spoken digit recognition and system identification

Information processing using a single dynamical node as complex system

L. Appeltant... et al.Novel methods for information processing are highly desired in our information-driven society. Inspired by the brain's ability to process information, the recently introduced paradigm known as 'reservoir computing' shows that complex networks can efficiently perform computation. Here we introduce a novel architecture that reduces the usually required large number of elements to a single nonlinear node with delayed feedback. Through an electronic implementation, we experimentally and numerically demonstrate excellent performance in a speech recognition benchmark. Complementary numerical studies also show excellent performance for a time series prediction benchmark. These results prove that delay-dynamical systems, even in their simplest manifestation, can perform efficient information processing. This finding paves the way to feasible and resource-efficient technological implementations of reservoir computing.This research was partially supported by the Belgian Science Policy Office, under grant IAP P6-10 'photonics@be', by FWO and FRS–FNRS (Belgium), MICINN (Spain) under projects FISICOS (FIS2007-60327) and DeCoDicA (TEC2009-14101) and by the European project PHOCUS (EU FET-Open grant: 240763). L.A. and G.VdS. are a PhD Fellow and a Postdoctoral Fellow of the Research Foundation-Flanders (FWO).Peer reviewe

Reservoir computing based on delay-dynamical systems

Today, except for mathematical operations, our brain functions much faster and more efficient than any supercomputer. It is precisely this form of information processing in neural networks that inspires researchers to create systems that mimic the brain’s information processing capabilities. In this thesis we propose a novel approach to implement these alternative computer architectures, based on delayed feedback. We show that one single nonlinear node with delayed feedback can replace a large network of nonlinear nodes. First we numerically investigate the architecture and performance of delayed feedback systems as information processing units. Then we elaborate on electronic and opto-electronic implementations of the concept. Next to evaluating their performance for standard benchmarks, we also study task independent properties of the system, extracting information on how to further improve the initial scheme. Finally, some simple modifications are suggested, yielding improvements in terms of speed or performance

Constructing optimized binary masks for reservoir computing with delay systems

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.Reservoir computing is a novel bio-inspired computing method, capable of solving complex tasks in a computationally efficient way. It has recently been successfully implemented using delayed feedback systems, allowing to reduce the hardware complexity of brain-inspired computers drastically. In this approach, the pre-processing procedure relies on the definition of a temporal mask which serves as a scaled time-mutiplexing of the input. Originally, random masks had been chosen, motivated by the random connectivity in reservoirs. This random generation can sometimes fail. Moreover, for hardware implementations random generation is not ideal due to its complexity and the requirement for trial and error. We outline a procedure to reliably construct an optimal mask pattern in terms of multipurpose performance, derived from the concept of maximum length sequences. Not only does this ensure the creation of the shortest possible mask that leads to maximum variability in the reservoir states for the given reservoir, it also allows for an interpretation of the statistical significance of the provided training samples for the task at hand.This research was partially supported by the Belgian Science Policy Office, under grant IAP P7/35 Photonics@be, by FWO(Belgium), MICINN (Spain), Comunitat Autonoma de les Illes Balears, FEDER, and the European Commission under Projects TEC2012-36335 (TRIPHOP), Grups Competitius and EC FP7 Projects PHOCUS (Grant No. 240763).Peer Reviewe

Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing

Many information processing challenges are difficult to solve with traditional Turing or von Neumann approaches. Implementing unconventional computational methods is therefore essential and optics provides promising opportunities. Here we experimentally demonstrate optical information processing using a nonlinear optoelectronic oscillator subject to delayed feedback.We implement a neuro-inspired concept, called Reservoir Computing, proven to possess universal computational capabilities. We particularly exploit the transient response of a complex dynamical system to an input data stream. We employ spoken digit recognition and time series prediction tasks as benchmarks, achieving competitive processing figures of merit.The project PHOCUS acknowledges the financial support of the Future and Emerging Technologies (FET) programme within the Seventh Frame- work Programme for Research of the European Commission, under FET-Open grant number: 240763. Moreover, this work was supported by MICINN (Spain), and FEDER, under Projects TEC2009-14101 (DeCoDicA), FIS2007-60327 (FISICOS), and 0200950I190 (Proyecto Intra- murales Especiales).Peer reviewe

Delay-based reservoir computing: Noise effects in a combined analog and digital implementation

Reservoir computing is a paradigm in machine learning whose processing capabilities rely on the dynamical behavior of recurrent neural networks. We present a mixed analog and digital implementation of this concept with a nonlinear analog electronic circuit as a main computational unit. In our approach, the reservoir network can be replaced by a single nonlinear element with delay via time-multiplexing. We analyze the influence of noise on the performance of the system for two benchmark tasks: 1) a classification problem and 2) a chaotic time-series prediction task. Special attention is given to the role of quantization noise, which is studied by varying the resolution in the conversion interface between the analog and digital worlds.This work was supported in part by MINECO, Spain, in part by the Comunitat Autònoma de les Illes Balears, in part by FEDER, in part by the European Commission under Project TEC2012-38864 and Project TEC2012-36335, in part by Grups Competitius, in part by the EC FP7 Project PHOCUS under Grant 240763, in part by the Interuniversity Attraction Pole Photonics@be, Belgian Science Policy Office, and in part by the Flemish Research Foundation.Peer Reviewe

Reservoir computing using a delayed feedback system: towards photonic implementations

Delayed feedback systems are known to exhibit a rich dynamical behavior, showing a wide variety of dynamical regimes. We use this richness to implement reservoir computing, a processing concept in machine learning. In this paper we demonstrate the proof of principle on an electronic system, however the approach is readily transferable to photonics, promising fast and computationally efficient all-optical processing. Using only one single node with delayed feedback instead of an entire network of nodes, we succeed in obtaining state-of-the-art results on benchmarks such as spoken digit recognition and system identification.info:eu-repo/semantics/publishe

Computing using delayed feedback systems: towards photonics

info:eu-repo/semantics/publishe