29,860 research outputs found
A Spike-Timing Pattern Based Neural Network Model for the Study of Memory Dynamics
It is well accepted that the brain's computation relies on spatiotemporal activity of neural networks. In particular, there is growing evidence of the importance of continuously and precisely timed spiking activity. Therefore, it is important to characterize memory states in terms of spike-timing patterns that give both reliable memory of firing activities and precise memory of firing timings. The relationship between memory states and spike-timing patterns has been studied empirically with large-scale recording of neuron population in recent years. Here, by using a recurrent neural network model with dynamics at two time scales, we construct a dynamical memory network model which embeds both fast neural and synaptic variation and slow learning dynamics. A state vector is proposed to describe memory states in terms of spike-timing patterns of neural population, and a distance measure of state vector is defined to study several important phenomena of memory dynamics: partial memory recall, learning efficiency, learning with correlated stimuli. We show that the distance measure can capture the timing difference of memory states. In addition, we examine the influence of network topology on learning ability, and show that local connections can increase the network's ability to embed more memory states. Together theses results suggest that the proposed system based on spike-timing patterns gives a productive model for the study of detailed learning and memory dynamics
Nanophotonic reservoir computing with photonic crystal cavities to generate periodic patterns
Reservoir computing (RC) is a technique in machine learning inspired by neural systems. RC has been used successfully to solve complex problems such as signal classification and signal generation. These systems are mainly implemented in software, and thereby they are limited in speed and power efficiency. Several optical and optoelectronic implementations have been demonstrated, in which the system has signals with an amplitude and phase. It is proven that these enrich the dynamics of the system, which is beneficial for the performance. In this paper, we introduce a novel optical architecture based on nanophotonic crystal cavities. This allows us to integrate many neurons on one chip, which, compared with other photonic solutions, closest resembles a classical neural network. Furthermore, the components are passive, which simplifies the design and reduces the power consumption. To assess the performance of this network, we train a photonic network to generate periodic patterns, using an alternative online learning rule called first-order reduced and corrected error. For this, we first train a classical hyperbolic tangent reservoir, but then we vary some of the properties to incorporate typical aspects of a photonics reservoir, such as the use of continuous-time versus discrete-time signals and the use of complex-valued versus real-valued signals. Then, the nanophotonic reservoir is simulated and we explore the role of relevant parameters such as the topology, the phases between the resonators, the number of nodes that are biased and the delay between the resonators. It is important that these parameters are chosen such that no strong self-oscillations occur. Finally, our results show that for a signal generation task a complex-valued, continuous-time nanophotonic reservoir outperforms a classical (i.e., discrete-time, real-valued) leaky hyperbolic tangent reservoir (normalized root-mean-square errors = 0.030 versus NRMSE = 0.127)
Optimising the topology of complex neural networks
In this paper, we study instances of complex neural networks, i.e. neural
netwo rks with complex topologies. We use Self-Organizing Map neural networks
whose n eighbourhood relationships are defined by a complex network, to
classify handwr itten digits. We show that topology has a small impact on
performance and robus tness to neuron failures, at least at long learning
times. Performance may howe ver be increased (by almost 10%) by artificial
evolution of the network topo logy. In our experimental conditions, the evolved
networks are more random than their parents, but display a more heterogeneous
degree distribution
Neuronal avalanches of a self-organized neural network with active-neuron-dominant structure
Neuronal avalanche is a spontaneous neuronal activity which obeys a power-law
distribution of population event sizes with an exponent of -3/2. It has been
observed in the superficial layers of cortex both \emph{in vivo} and \emph{in
vitro}. In this paper we analyze the information transmission of a novel
self-organized neural network with active-neuron-dominant structure. Neuronal
avalanches can be observed in this network with appropriate input intensity. We
find that the process of network learning via spike-timing dependent plasticity
dramatically increases the complexity of network structure, which is finally
self-organized to be active-neuron-dominant connectivity. Both the entropy of
activity patterns and the complexity of their resulting post-synaptic inputs
are maximized when the network dynamics are propagated as neuronal avalanches.
This emergent topology is beneficial for information transmission with high
efficiency and also could be responsible for the large information capacity of
this network compared with alternative archetypal networks with different
neural connectivity.Comment: Non-final version submitted to Chao
Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A survey
Dynamic networks are used in a wide range of fields, including social network
analysis, recommender systems, and epidemiology. Representing complex networks
as structures changing over time allow network models to leverage not only
structural but also temporal patterns. However, as dynamic network literature
stems from diverse fields and makes use of inconsistent terminology, it is
challenging to navigate. Meanwhile, graph neural networks (GNNs) have gained a
lot of attention in recent years for their ability to perform well on a range
of network science tasks, such as link prediction and node classification.
Despite the popularity of graph neural networks and the proven benefits of
dynamic network models, there has been little focus on graph neural networks
for dynamic networks. To address the challenges resulting from the fact that
this research crosses diverse fields as well as to survey dynamic graph neural
networks, this work is split into two main parts. First, to address the
ambiguity of the dynamic network terminology we establish a foundation of
dynamic networks with consistent, detailed terminology and notation. Second, we
present a comprehensive survey of dynamic graph neural network models using the
proposed terminologyComment: 28 pages, 9 figures, 8 table
Short-Term Memory Through Persistent Activity: Evolution of Self-Stopping and Self-Sustaining Activity in Spiking Neural Networks
Memories in the brain are separated in two categories: short-term and
long-term memories. Long-term memories remain for a lifetime, while short-term
ones exist from a few milliseconds to a few minutes. Within short-term memory
studies, there is debate about what neural structure could implement it.
Indeed, mechanisms responsible for long-term memories appear inadequate for the
task. Instead, it has been proposed that short-term memories could be sustained
by the persistent activity of a group of neurons. In this work, we explore what
topology could sustain short-term memories, not by designing a model from
specific hypotheses, but through Darwinian evolution in order to obtain new
insights into its implementation. We evolved 10 networks capable of retaining
information for a fixed duration between 2 and 11s. Our main finding has been
that the evolution naturally created two functional modules in the network: one
which sustains the information containing primarily excitatory neurons, while
the other, which is responsible for forgetting, was composed mainly of
inhibitory neurons. This demonstrates how the balance between inhibition and
excitation plays an important role in cognition.Comment: 28 page
- …