907 research outputs found
Product Reservoir Computing: Time-Series Computation with Multiplicative Neurons
Echo state networks (ESN), a type of reservoir computing (RC) architecture,
are efficient and accurate artificial neural systems for time series processing
and learning. An ESN consists of a core of recurrent neural networks, called a
reservoir, with a small number of tunable parameters to generate a
high-dimensional representation of an input, and a readout layer which is
easily trained using regression to produce a desired output from the reservoir
states. Certain computational tasks involve real-time calculation of high-order
time correlations, which requires nonlinear transformation either in the
reservoir or the readout layer. Traditional ESN employs a reservoir with
sigmoid or tanh function neurons. In contrast, some types of biological neurons
obey response curves that can be described as a product unit rather than a sum
and threshold. Inspired by this class of neurons, we introduce a RC
architecture with a reservoir of product nodes for time series computation. We
find that the product RC shows many properties of standard ESN such as
short-term memory and nonlinear capacity. On standard benchmarks for chaotic
prediction tasks, the product RC maintains the performance of a standard
nonlinear ESN while being more amenable to mathematical analysis. Our study
provides evidence that such networks are powerful in highly nonlinear tasks
owing to high-order statistics generated by the recurrent product node
reservoir
Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere
Fourier Neural Operators (FNOs) have proven to be an efficient and effective
method for resolution-independent operator learning in a broad variety of
application areas across scientific machine learning. A key reason for their
success is their ability to accurately model long-range dependencies in
spatio-temporal data by learning global convolutions in a computationally
efficient manner. To this end, FNOs rely on the discrete Fourier transform
(DFT), however, DFTs cause visual and spectral artifacts as well as pronounced
dissipation when learning operators in spherical coordinates since they
incorrectly assume a flat geometry. To overcome this limitation, we generalize
FNOs on the sphere, introducing Spherical FNOs (SFNOs) for learning operators
on spherical geometries. We apply SFNOs to forecasting atmospheric dynamics,
and demonstrate stable auto\-regressive rollouts for a year of simulated time
(1,460 steps), while retaining physically plausible dynamics. The SFNO has
important implications for machine learning-based simulation of climate
dynamics that could eventually help accelerate our response to climate change
Photonic reservoir computing enabled by stimulated Brillouin scattering
Artificial Intelligence (AI) drives the creation of future technologies that
disrupt the way humans live and work, creating new solutions that change the
way we approach tasks and activities, but it requires a lot of data processing,
large amounts of data transfer, and computing speed. It has led to a growing
interest of research in developing a new type of computing platform which is
inspired by the architecture of the brain specifically those that exploit the
benefits offered by photonic technologies, fast, low-power, and larger
bandwidth. Here, a new computing platform based on the photonic reservoir
computing architecture exploiting the non-linear wave-optical dynamics of the
stimulated Brillouin scattering is reported. The kernel of the new photonic
reservoir computing system is constructed of an entirely passive optical
system. Moreover, it is readily suited for use in conjunction with high
performance optical multiplexing techniques to enable real-time artificial
intelligence. Here, a methodology to optimise the operational condition of the
new photonic reservoir computing is described which is found to be strongly
dependent on the dynamics of the stimulated Brillouin scattering system. The
new architecture described here offers a new way of realising AI-hardware which
highlight the application of photonics for AI.Comment: 8 pages, 6 figures, research articl
- …