2,461 research outputs found
Storage Capacity of Extremely Diluted Hopfield Model
The storage capacity of the extremely diluted Hopfield Model is studied by
using Monte Carlo techniques. In this work, instead of diluting the synapses
according to a given distribution, the dilution of the synapses is obtained
systematically by retaining only the synapses with dominant contributions. It
is observed that by using the prescribed dilution method the critical storage
capacity of the system increases with decreasing number of synapses per neuron
reaching almost the value obtained from mean-field calculations. It is also
shown that the increase of the storage capacity of the diluted system depends
on the storage capacity of the fully connected Hopfield Model and the fraction
of the diluted synapses.Comment: Latex, 14 pages, 4 eps figure
Spontaneous structure formation in a network of chaotic units with variable connection strengths
As a model of temporally evolving networks, we consider a globally coupled
logistic map with variable connection weights. The model exhibits
self-organization of network structure, reflected by the collective behavior of
units. Structural order emerges even without any inter-unit synchronization of
dynamics. Within this structure, units spontaneously separate into two groups
whose distinguishing feature is that the first group possesses many
outwardly-directed connections to the second group, while the second group
possesses only few outwardly-directed connections to the first. The relevance
of the results to structure formation in neural networks is briefly discussed.Comment: 4 pages, 3 figures, REVTe
Bump formation in a binary attractor neural network
This paper investigates the conditions for the formation of local bumps in
the activity of binary attractor neural networks with spatially dependent
connectivity. We show that these formations are observed when asymmetry between
the activity during the retrieval and learning is imposed. Analytical
approximation for the order parameters is derived. The corresponding phase
diagram shows a relatively large and stable region, where this effect is
observed, although the critical storage and the information capacities
drastically decrease inside that region. We demonstrate that the stability of
the network, when starting from the bump formation, is larger than the
stability when starting even from the whole pattern. Finally, we show a very
good agreement between the analytical results and the simulations performed for
different topologies of the network.Comment: about 14 page
Memory Aware Synapses: Learning what (not) to forget
Humans can learn in a continuous manner. Old rarely utilized knowledge can be
overwritten by new incoming information while important, frequently used
knowledge is prevented from being erased. In artificial learning systems,
lifelong learning so far has focused mainly on accumulating knowledge over
tasks and overcoming catastrophic forgetting. In this paper, we argue that,
given the limited model capacity and the unlimited new information to be
learned, knowledge has to be preserved or erased selectively. Inspired by
neuroplasticity, we propose a novel approach for lifelong learning, coined
Memory Aware Synapses (MAS). It computes the importance of the parameters of a
neural network in an unsupervised and online manner. Given a new sample which
is fed to the network, MAS accumulates an importance measure for each parameter
of the network, based on how sensitive the predicted output function is to a
change in this parameter. When learning a new task, changes to important
parameters can then be penalized, effectively preventing important knowledge
related to previous tasks from being overwritten. Further, we show an
interesting connection between a local version of our method and Hebb's
rule,which is a model for the learning process in the brain. We test our method
on a sequence of object recognition tasks and on the challenging problem of
learning an embedding for predicting triplets.
We show state-of-the-art performance and, for the first time, the ability to
adapt the importance of the parameters based on unlabeled data towards what the
network needs (not) to forget, which may vary depending on test conditions.Comment: ECCV 201
Apollo 7 retrofire and reentry of service propulsion module. Further study of Intelsat 2 F-2 apogee burn
Photography of Apollo 7 retrofire and service propulsion module reentry and apogee burn of Intelsat 2 F-2 satellit
Reflection Positivity and Monotonicity
We prove general reflection positivity results for both scalar fields and
Dirac fields on a Riemannian manifold, and comment on applications to quantum
field theory. As another application, we prove the inequality
between Dirichlet and Neumann covariance operators on a manifold with a
reflection.Comment: 11 page
Effects of Synaptic and Myelin Plasticity on Learning in a Network of Kuramoto Phase Oscillators
Models of learning typically focus on synaptic plasticity. However, learning
is the result of both synaptic and myelin plasticity. Specifically, synaptic
changes often co-occur and interact with myelin changes, leading to complex
dynamic interactions between these processes. Here, we investigate the
implications of these interactions for the coupling behavior of a system of
Kuramoto oscillators. To that end, we construct a fully connected,
one-dimensional ring network of phase oscillators whose coupling strength
(reflecting synaptic strength) as well as conduction velocity (reflecting
myelination) are each regulated by a Hebbian learning rule. We evaluate the
behavior of the system in terms of structural (pairwise connection strength and
conduction velocity) and functional connectivity (local and global
synchronization behavior). We find that for conditions in which a system
limited to synaptic plasticity develops two distinct clusters both structurally
and functionally, additional adaptive myelination allows for functional
communication across these structural clusters. Hence, dynamic conduction
velocity permits the functional integration of structurally segregated
clusters. Our results confirm that network states following learning may be
different when myelin plasticity is considered in addition to synaptic
plasticity, pointing towards the relevance of integrating both factors in
computational models of learning.Comment: 39 pages, 15 figures This work is submitted in Chaos: An
Interdisciplinary Journal of Nonlinear Scienc
Rotation of Late-Type Stars in Praesepe with K2
We have Fourier analyzed 941 K2 light curves of likely members of Praesepe,
measuring periods for 86% and increasing the number of rotation periods (P) by
nearly a factor of four. The distribution of P vs. (V-K), a mass proxy, has
three different regimes: (V-K)<1.3, where the rotation rate rapidly slows as
mass decreases; 1.3<(V-K)<4.5, where the rotation rate slows more gradually as
mass decreases; and (V-K)>4.5, where the rotation rate rapidly increases as
mass decreases. In this last regime, there is a bimodal distribution of
periods, with few between 2 and 10 days. We interpret this to mean
that once M stars start to slow down, they do so rapidly. The K2 period-color
distribution in Praesepe (790 Myr) is much different than in the Pleiades
(125 Myr) for late F, G, K, and early-M stars; the overall distribution
moves to longer periods, and is better described by 2 line segments. For mid-M
stars, the relationship has similarly broad scatter, and is steeper in
Praesepe. The diversity of lightcurves and of periodogram types is similar in
the two clusters; about a quarter of the periodic stars in both clusters have
multiple significant periods. Multi-periodic stars dominate among the higher
masses, starting at a bluer color in Praesepe ((V-K)1.5) than in the
Pleiades ((V-K)2.6). In Praesepe, there are relatively more light curves
that have two widely separated periods, 6 days. Some of these could
be examples of M star binaries where one star has spun down but the other has
not.Comment: Accepted by Ap
Learning by message-passing in networks of discrete synapses
We show that a message-passing process allows to store in binary "material"
synapses a number of random patterns which almost saturates the information
theoretic bounds. We apply the learning algorithm to networks characterized by
a wide range of different connection topologies and of size comparable with
that of biological systems (e.g. ). The algorithm can be
turned into an on-line --fault tolerant-- learning protocol of potential
interest in modeling aspects of synaptic plasticity and in building
neuromorphic devices.Comment: 4 pages, 3 figures; references updated and minor corrections;
accepted in PR
- …