67 research outputs found
Reservoir Computing Approach to Robust Computation using Unreliable Nanoscale Networks
As we approach the physical limits of CMOS technology, advances in materials
science and nanotechnology are making available a variety of unconventional
computing substrates that can potentially replace top-down-designed
silicon-based computing devices. Inherent stochasticity in the fabrication
process and nanometer scale of these substrates inevitably lead to design
variations, defects, faults, and noise in the resulting devices. A key
challenge is how to harness such devices to perform robust computation. We
propose reservoir computing as a solution. In reservoir computing, computation
takes place by translating the dynamics of an excited medium, called a
reservoir, into a desired output. This approach eliminates the need for
external control and redundancy, and the programming is done using a
closed-form regression problem on the output, which also allows concurrent
programming using a single device. Using a theoretical model, we show that both
regular and irregular reservoirs are intrinsically robust to structural noise
as they perform computation
Dynamic clustering of time series with Echo State Networks
In this paper we introduce a novel methodology for unsupervised analysis of time series, based upon the iterative implementation of a clustering algorithm embedded into the evolution of a recurrent Echo State Network. The main features of the temporal data are captured by the dynamical evolution of the network states, which are then subject to a clustering procedure. We apply the proposed algorithm to time series coming from records of eye movements, called saccades, which are recorded for diagnosis of a neurodegenerative form of ataxia. This is a hard classification problem, since saccades from patients at an early stage of the disease are practically indistinguishable from those coming from healthy subjects. The unsupervised clustering algorithm implanted within the recurrent network produces more compact clusters, compared to conventional clustering of static data, and provides a source of information that could aid diagnosis and assessment of the disease.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tec
Algorithms and Results of Eye Tissues Differentiation Based on RF Ultrasound
Algorithms and software were developed for analysis of B-scan ultrasonic signals acquired from commercial diagnostic ultrasound system. The algorithms process raw ultrasonic signals in backscattered spectrum domain, which is obtained using two time-frequency methods: short-time Fourier and Hilbert-Huang transformations. The signals from selected regions of eye tissues are characterized by parameters: B-scan envelope amplitude, approximated spectral slope, approximated spectral intercept, mean instantaneous frequency, mean instantaneous bandwidth, and parameters of Nakagami distribution characterizing Hilbert-Huang transformation output. The backscattered ultrasound signal parameters characterizing intraocular and orbit tissues were processed by decision tree data mining algorithm. The pilot trial proved that applied methods are able to correctly classify signals from corpus vitreum blood, extraocular muscle, and orbit tissues. In 26 cases of ocular tissues classification, one error occurred, when tissues were classified into classes of corpus vitreum blood, extraocular muscle, and orbit tissue. In this pilot classification parameters of spectral intercept and Nakagami parameter for instantaneous frequencies distribution of the 1st intrinsic mode function were found specific for corpus vitreum blood, orbit and extraocular muscle tissues. We conclude that ultrasound data should be further collected in clinical database to establish background for decision support system for ocular tissue noninvasive differentiation
Physics-Informed Echo State Networks for Chaotic Systems Forecasting
We propose a physics-informed Echo State Network (ESN) to predict the
evolution of chaotic systems. Compared to conventional ESNs, the
physics-informed ESNs are trained to solve supervised learning tasks while
ensuring that their predictions do not violate physical laws. This is achieved
by introducing an additional loss function during the training of the ESNs,
which penalizes non-physical predictions without the need of any additional
training data. This approach is demonstrated on a chaotic Lorenz system, where
the physics-informed ESNs improve the predictability horizon by about two
Lyapunov times as compared to conventional ESNs. The proposed framework shows
the potential of using machine learning combined with prior physical knowledge
to improve the time-accurate prediction of chaotic dynamical systems.Comment: 7 pages, 3 figure
Physics-Informed Echo State Networks for Chaotic Systems Forecasting
We propose a physics-informed Echo State Network (ESN)
to predict the evolution of chaotic systems. Compared to conventional
ESNs, the physics-informed ESNs are trained to solve supervised learning
tasks while ensuring that their predictions do not violate physical laws.
This is achieved by introducing an additional loss function during the
training of the ESNs, which penalizes non-physical predictions without
the need of any additional training data. This approach is demonstrated
on a chaotic Lorenz system, where the physics-informed ESNs improve
the predictability horizon by about two Lyapunov times as compared to
conventional ESNs. The proposed framework shows the potential of using
machine learning combined with prior physical knowledge to improve the
time-accurate prediction of chaotic dynamical systems
Optoelectronic Reservoir Computing
Reservoir computing is a recently introduced, highly efficient bio-inspired
approach for processing time dependent data. The basic scheme of reservoir
computing consists of a non linear recurrent dynamical system coupled to a
single input layer and a single output layer. Within these constraints many
implementations are possible. Here we report an opto-electronic implementation
of reservoir computing based on a recently proposed architecture consisting of
a single non linear node and a delay line. Our implementation is sufficiently
fast for real time information processing. We illustrate its performance on
tasks of practical importance such as nonlinear channel equalization and speech
recognition, and obtain results comparable to state of the art digital
implementations.Comment: Contains main paper and two Supplementary Material
ReservoirPy: an Efficient and User-Friendly Library to Design Echo State Networks
International audienceWe present a simple user-friendly library called ReservoirPy based on Python scientific modules. It provides a flexible interface to implement efficient Reservoir Computing (RC) architectures with a particular focus on Echo State Networks (ESN). Advanced features of ReservoirPy allow to improve up to 87.9% of computation time efficiency on a simple laptop compared to basic Python implementation. Overall, we provide tutorials for hyperparameters tuning, offline and online training, fast spectral initialization, parallel and sparse matrix computation on various tasks (MackeyGlass and audio recognition tasks). In particular, we provide graphical tools to easily explore hyperparameters using random search with the help of the hyperopt library
The role of structure and complexity on Reservoir Computing quality
We explore the effect of structure and connection complexity on the dynamical behaviour of Reservoir Computers (RC). At present, considerable effort is taken to design and hand-craft physical reservoir computers. Both structure and physical complexity are often pivotal to task performance, however, assessing their overall importance is challenging. Using a recently proposed framework, we evaluate and compare the dynamical freedom (referring to quality) of neural network structures, as an analogy for physical systems. The results quantify how structure affects the range of behaviours exhibited by these networks. It highlights that high quality reached by more complex structures is often also achievable in simpler structures with greater network size. Alternatively, quality is often improved in smaller networks by adding greater connection complexity. This work demonstrates the benefits of using abstract behaviour representation, rather than evaluation through benchmark tasks, to assess the quality of computing substrates, as the latter typically has biases, and often provides little insight into the complete computing quality of physical systems
Reservoir Topology in Deep Echo State Networks
Deep Echo State Networks (DeepESNs) recently extended the applicability of
Reservoir Computing (RC) methods towards the field of deep learning. In this
paper we study the impact of constrained reservoir topologies in the
architectural design of deep reservoirs, through numerical experiments on
several RC benchmarks. The major outcome of our investigation is to show the
remarkable effect, in terms of predictive performance gain, achieved by the
synergy between a deep reservoir construction and a structured organization of
the recurrent units in each layer. Our results also indicate that a
particularly advantageous architectural setting is obtained in correspondence
of DeepESNs where reservoir units are structured according to a permutation
recurrent matrix.Comment: Preprint of the paper published in the proceedings of ICANN 201
Reservoir computing quality : connectivity and topology
We explore the effect of connectivity and topology on the dynamical behaviour of Reservoir Computers. At present, considerable effort is taken to design and hand-craft physical reservoir computers. Both structure and physical complexity are often pivotal to task performance, however, assessing their overall importance is challenging. Using a recently developed framework, we evaluate and compare the dynamical freedom (referring to quality) of neural network structures, as an analogy for physical systems. The results quantify how structure affects the behavioural range of networks. It demonstrates how high quality reached by more complex structures is often also achievable in simpler structures with greater network size. Alternatively, quality is often improved in smaller networks by adding greater connection complexity. This work demonstrates the benefits of using dynamical behaviour to assess the quality of computing substrates, rather than evaluation through benchmark tasks that often provide a narrow and biased insight into the computing quality of physical systems
- …