Critical dynamics in homeostatic memory networks

Critical behavior in neural networks characterized by scale-free event distributions and brought about by self-regulatory mechanisms such as short-term synaptic dynamics or homeostatic plasticity, is believed to optimize sensitivity to input and information transfer in the system. Although theoretical predictions of the spike distributions have been confirmed by in-vitro experiments, in-vivo data yield a more complex picture which might be due to the in-homogeneity of the network structure, leakage in currents or massive driving inputs which has so far not been comprehensively covered by analytical or numerical studies.

We address these questions by the study of a neural model of memory that allows for storage and retrieval of patterns and for recombining such patterns as needed for search in problem solving. The model features critical dynamics in the neural assembly as a result of the interplay of synaptic depression and facilitation (Levina e.a 2007, 2009). Model simulations show that the prolonged consolidation of memory patterns induces a bias towards the memories which affects the scale-free spike-frequency distribution. However, selective modification of neuronal circuitry in the form of controlled homeostatic regulation in the form of recalibration of the synaptic weights towards the critical value preserved criticality although characterized by fluctuations between learned random patterns, as observed by the dynamics of stored pattern retrieval quality. The resulting spike statistics depends on the assumed coding scheme, but even sparse or orthogonal memory patterns introduce a typical event size which is incompatible with critical dynamics below the maximal memory capacity. Specifically results obtained for de-correlated patterns show an immediate jump from the sub-critical regime to a state of super-criticality in contrast to a more structured wave-like formation in the avalanche dynamics obtained from a general set of random patterns, pointing towards an eventual evolution of the network connectivity and the optimization of the critical regime. Specifically results obtained for de-correlated patterns show an immediate jump from the sub-critical regime to a state of super-criticality in contrast to a more structured wave-like formation in the avalanche dynamics obtained from a general set of random patterns, pointing towards an eventual evolution of the network connectivity and the optimization of the critical regime (Pearlmutter and Houghton, 2009).

The combination of memory and ongoing dynamics in the model was chosen for its implications in the context of cognitive aging. Following the paradigm of aging as a multi-criteria optimization process, we posit aging effects as a result of an increasing incompatibility of learning goals. In aging, a shift from fluid intelligence (flexibility to recombine memory content) towards crystalline intelligence (optimal memory organization) appears as a lifelong trend against the general decrease of resources. We show that in young age memory and criticality can be maintained simultaneously by a homeostatic leveling of the synaptic conductances. This balance is lost in the aging brain where the memory attractors cannot be kept sufficiently shallow due to neural and synaptic loss, a reduction of activity while experiencing a growth in memories. The value of the memory organization is therefore protected on the cost of the partial loss of the capability of recombining memory patterns in a task-dependent way

Dimensions of Timescales in Neuromorphic Computing Systems

This article is a public deliverable of the EU project "Memory technologies with multi-scale time constants for neuromorphic architectures" (MeMScales, https://memscales.eu, Call ICT-06-2019 Unconventional Nanoelectronics, project number 871371). This arXiv version is a verbatim copy of the deliverable report, with administrative information stripped. It collects a wide and varied assortment of phenomena, models, research themes and algorithmic techniques that are connected with timescale phenomena in the fields of computational neuroscience, mathematics, machine learning and computer science, with a bias toward aspects that are relevant for neuromorphic engineering. It turns out that this theme is very rich indeed and spreads out in many directions which defy a unified treatment. We collected several dozens of sub-themes, each of which has been investigated in specialized settings (in the neurosciences, mathematics, computer science and machine learning) and has been documented in its own body of literature. The more we dived into this diversity, the more it became clear that our first effort to compose a survey must remain sketchy and partial. We conclude with a list of insights distilled from this survey which give general guidelines for the design of future neuromorphic systems

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

Reservoir computing (RC), first applied to temporal signal processing, is a recurrent neural network in which neurons are randomly connected. Once initialized, the connection strengths remain unchanged. Such a simple structure turns RC into a non-linear dynamical system that maps low-dimensional inputs into a high-dimensional space. The model's rich dynamics, linear separability, and memory capacity then enable a simple linear readout to generate adequate responses for various applications. RC spans areas far beyond machine learning, since it has been shown that the complex dynamics can be realized in various physical hardware implementations and biological devices. This yields greater flexibility and shorter computation time. Moreover, the neuronal responses triggered by the model's dynamics shed light on understanding brain mechanisms that also exploit similar dynamical processes. While the literature on RC is vast and fragmented, here we conduct a unified review of RC's recent developments from machine learning to physics, biology, and neuroscience. We first review the early RC models, and then survey the state-of-the-art models and their applications. We further introduce studies on modeling the brain's mechanisms by RC. Finally, we offer new perspectives on RC development, including reservoir design, coding frameworks unification, physical RC implementations, and interaction between RC, cognitive neuroscience and evolution.Comment: 51 pages, 19 figures, IEEE Acces

25th Annual Computational Neuroscience Meeting: CNS-2016

Abstracts of the 25th Annual Computational Neuroscience Meeting: CNS-2016 Seogwipo City, Jeju-do, South Korea. 2–7 July 201

25th annual computational neuroscience meeting: CNS-2016

The same neuron may play different functional roles in the neural circuits to which it belongs. For example, neurons in the Tritonia pedal ganglia may participate in variable phases of the swim motor rhythms [1]. While such neuronal functional variability is likely to play a major role the delivery of the functionality of neural systems, it is difficult to study it in most nervous systems. We work on the pyloric rhythm network of the crustacean stomatogastric ganglion (STG) [2]. Typically network models of the STG treat neurons of the same functional type as a single model neuron (e.g. PD neurons), assuming the same conductance parameters for these neurons and implying their synchronous firing [3, 4]. However, simultaneous recording of PD neurons shows differences between the timings of spikes of these neurons. This may indicate functional variability of these neurons. Here we modelled separately the two PD neurons of the STG in a multi-neuron model of the pyloric network. Our neuron models comply with known correlations between conductance parameters of ionic currents. Our results reproduce the experimental finding of increasing spike time distance between spikes originating from the two model PD neurons during their synchronised burst phase. The PD neuron with the larger calcium conductance generates its spikes before the other PD neuron. Larger potassium conductance values in the follower neuron imply longer delays between spikes, see Fig. 17.Neuromodulators change the conductance parameters of neurons and maintain the ratios of these parameters [5]. Our results show that such changes may shift the individual contribution of two PD neurons to the PD-phase of the pyloric rhythm altering their functionality within this rhythm. Our work paves the way towards an accessible experimental and computational framework for the analysis of the mechanisms and impact of functional variability of neurons within the neural circuits to which they belong

26th Annual Computational Neuroscience Meeting (CNS*2017): Part 3 - Meeting Abstracts - Antwerp, Belgium. 15–20 July 2017

This work was produced as part of the activities of FAPESP Research,\ud Disseminations and Innovation Center for Neuromathematics (grant\ud 2013/07699-0, S. Paulo Research Foundation). NLK is supported by a\ud FAPESP postdoctoral fellowship (grant 2016/03855-5). ACR is partially\ud supported by a CNPq fellowship (grant 306251/2014-0)