36,523 research outputs found
Computational Capacity and Energy Consumption of Complex Resistive Switch Networks
Resistive switches are a class of emerging nanoelectronics devices that
exhibit a wide variety of switching characteristics closely resembling
behaviors of biological synapses. Assembled into random networks, such
resistive switches produce emerging behaviors far more complex than that of
individual devices. This was previously demonstrated in simulations that
exploit information processing within these random networks to solve tasks that
require nonlinear computation as well as memory. Physical assemblies of such
networks manifest complex spatial structures and basic processing capabilities
often related to biologically-inspired computing. We model and simulate random
resistive switch networks and analyze their computational capacities. We
provide a detailed discussion of the relevant design parameters and establish
the link to the physical assemblies by relating the modeling parameters to
physical parameters. More globally connected networks and an increased network
switching activity are means to increase the computational capacity linearly at
the expense of exponentially growing energy consumption. We discuss a new
modular approach that exhibits higher computational capacities and energy
consumption growing linearly with the number of networks used. The results show
how to optimize the trade-off between computational capacity and energy
efficiency and are relevant for the design and fabrication of novel computing
architectures that harness random assemblies of emerging nanodevices
Storage of phase-coded patterns via STDP in fully-connected and sparse network: a study of the network capacity
We study the storage and retrieval of phase-coded patterns as stable
dynamical attractors in recurrent neural networks, for both an analog and a
integrate-and-fire spiking model. The synaptic strength is determined by a
learning rule based on spike-time-dependent plasticity, with an asymmetric time
window depending on the relative timing between pre- and post-synaptic
activity. We store multiple patterns and study the network capacity.
For the analog model, we find that the network capacity scales linearly with
the network size, and that both capacity and the oscillation frequency of the
retrieval state depend on the asymmetry of the learning time window. In
addition to fully-connected networks, we study sparse networks, where each
neuron is connected only to a small number z << N of other neurons. Connections
can be short range, between neighboring neurons placed on a regular lattice, or
long range, between randomly chosen pairs of neurons. We find that a small
fraction of long range connections is able to amplify the capacity of the
network. This imply that a small-world-network topology is optimal, as a
compromise between the cost of long range connections and the capacity
increase.
Also in the spiking integrate and fire model the crucial result of storing
and retrieval of multiple phase-coded patterns is observed. The capacity of the
fully-connected spiking network is investigated, together with the relation
between oscillation frequency of retrieval state and window asymmetry
Lifelong Sequential Modeling with Personalized Memorization for User Response Prediction
User response prediction, which models the user preference w.r.t. the
presented items, plays a key role in online services. With two-decade rapid
development, nowadays the cumulated user behavior sequences on mature Internet
service platforms have become extremely long since the user's first
registration. Each user not only has intrinsic tastes, but also keeps changing
her personal interests during lifetime. Hence, it is challenging to handle such
lifelong sequential modeling for each individual user. Existing methodologies
for sequential modeling are only capable of dealing with relatively recent user
behaviors, which leaves huge space for modeling long-term especially lifelong
sequential patterns to facilitate user modeling. Moreover, one user's behavior
may be accounted for various previous behaviors within her whole online
activity history, i.e., long-term dependency with multi-scale sequential
patterns. In order to tackle these challenges, in this paper, we propose a
Hierarchical Periodic Memory Network for lifelong sequential modeling with
personalized memorization of sequential patterns for each user. The model also
adopts a hierarchical and periodical updating mechanism to capture multi-scale
sequential patterns of user interests while supporting the evolving user
behavior logs. The experimental results over three large-scale real-world
datasets have demonstrated the advantages of our proposed model with
significant improvement in user response prediction performance against the
state-of-the-arts.Comment: SIGIR 2019. Reproducible codes and datasets:
https://github.com/alimamarankgroup/HPM
Slow synaptic dynamics in a network: from exponential to power-law forgetting
We investigate a mean-field model of interacting synapses on a directed
neural network. Our interest lies in the slow adaptive dynamics of synapses,
which are driven by the fast dynamics of the neurons they connect. Cooperation
is modelled from the usual Hebbian perspective, while competition is modelled
by an original polarity-driven rule. The emergence of a critical manifold
culminating in a tricritical point is crucially dependent on the presence of
synaptic competition. This leads to a universal power-law relaxation of
the mean synaptic strength along the critical manifold and an equally universal
relaxation at the tricritical point, to be contrasted with the
exponential relaxation that is otherwise generic. In turn, this leads to the
natural emergence of long- and short-term memory from different parts of
parameter space in a synaptic network, which is the most novel and important
result of our present investigations.Comment: 12 pages, 8 figures. Phys. Rev. E (2014) to appea
Spatio-temporal Learning with Arrays of Analog Nanosynapses
Emerging nanodevices such as resistive memories are being considered for
hardware realizations of a variety of artificial neural networks (ANNs),
including highly promising online variants of the learning approaches known as
reservoir computing (RC) and the extreme learning machine (ELM). We propose an
RC/ELM inspired learning system built with nanosynapses that performs both
on-chip projection and regression operations. To address time-dynamic tasks,
the hidden neurons of our system perform spatio-temporal integration and can be
further enhanced with variable sampling or multiple activation windows. We
detail the system and show its use in conjunction with a highly analog
nanosynapse device on a standard task with intrinsic timing dynamics- the TI-46
battery of spoken digits. The system achieves nearly perfect (99%) accuracy at
sufficient hidden layer size, which compares favorably with software results.
In addition, the model is extended to a larger dataset, the MNIST database of
handwritten digits. By translating the database into the time domain and using
variable integration windows, up to 95% classification accuracy is achieved. In
addition to an intrinsically low-power programming style, the proposed
architecture learns very quickly and can easily be converted into a spiking
system with negligible loss in performance- all features that confer
significant energy efficiency.Comment: 6 pages, 3 figures. Presented at 2017 IEEE/ACM Symposium on Nanoscale
architectures (NANOARCH
- …