The necessity of connection structures in neural models of variable binding

In his review of neural binding problems, Feldman (Cogn Neurodyn 7:1-11, 2013) addressed two types of models as solutions of (novel) variable binding. The one type uses labels such as phase synchrony of activation. The other (‘connectivity based') type uses dedicated connections structures to achieve novel variable binding. Feldman argued that label (synchrony) based models are the only possible candidates to handle novel variable binding, whereas connectivity based models lack the flexibility required for that. We argue and illustrate that Feldman's analysis is incorrect. Contrary to his conclusion, connectivity based models are the only viable candidates for models of novel variable binding because they are the only type of models that can produce behavior. We will show that the label (synchrony) based models analyzed by Feldman are in fact examples of connectivity based models. Feldman's analysis that novel variable binding can be achieved without existing connection structures seems to result from analyzing the binding problem in a wrong frame of reference, in particular in an outside instead of the required inside frame of reference. Connectivity based models can be models of novel variable binding when they possess a connection structure that resembles a small-world network, as found in the brain. We will illustrate binding with this type of model with episode binding and the binding of words, including novel words, in sentence structures

The role of recurrent networks in neural architectures of grounded cognition: learning of control

Recurrent networks have been used as neural models of language processing, with mixed results. Here, we discuss the role of recurrent networks in a neural architecture of grounded cognition. In particular, we discuss how the control of binding in this architecture can be learned. We trained a simple recurrent network (SRN) and a feedforward network (FFN) for this task. The results show that information from the architecture is needed as input for these networks to learn control of binding. Thus, both control systems are recurrent. We found that the recurrent system consisting of the architecture and an SRN or an FFN as a "core" can learn basic (but recursive) sentence structures. Problems with control of binding arise when the system with the SRN is tested on number of new sentence structures. In contrast, control of binding for these structures succeeds with the FFN. Yet, for some structures with (unlimited) embeddings, difficulties arise due to dynamical binding conflicts in the architecture itself. In closing, we discuss potential future developments of the architecture presented here

The state of MIIND

MIIND (Multiple Interacting Instantiations of Neural Dynamics) is a highly modular multi-level C++ framework, that aims to shorten the development time for models in Cognitive Neuroscience (CNS). It offers reusable code modules (libraries of classes and functions) aimed at solving problems that occur repeatedly in modelling, but tries not to impose a specific modelling philosophy or methodology. At the lowest level, it offers support for the implementation of sparse networks. For example, the library SparseImplementationLib supports sparse random networks and the library LayerMappingLib can be used for sparse regular networks of filter-like operators. The library DynamicLib, which builds on top of the library SparseImplementationLib, offers a generic framework for simulating network processes. Presently, several specific network process implementations are provided in MIIND: the Wilson–Cowan and Ornstein–Uhlenbeck type, and population density techniques for leaky-integrate-and-fire neurons driven by Poisson input. A design principle of MIIND is to support detailing: the refinement of an originally simple model into a form where more biological detail is included. Another design principle is extensibility: the reuse of an existing model in a larger, more extended one. One of the main uses of MIIND so far has been the instantiation of neural models of visual attention. Recently, we have added a library for implementing biologically-inspired models of artificial vision, such as HMAX and recent successors. In the long run we hope to be able to apply suitably adapted neuronal mechanisms of attention to these artificial models

To reverse engineer an entire nervous system

There are many theories of how behavior may be controlled by neurons. Testing and refining these theories would be greatly facilitated if we could correctly simulate an entire nervous system so we could replicate the brain dynamics in response to any stimuli or contexts. Besides, simulating a nervous system is in itself one of the big dreams in systems neuroscience. However, doing so requires us to identify how each neuron's output depends on its inputs, a process we call reverse engineering. Current efforts at this focus on the mammalian nervous system, but these brains are mind-bogglingly complex, allowing only recordings of tiny subsystems. Here we argue that the time is ripe for systems neuroscience to embark on a concerted effort to reverse engineer a smaller system and that Caenorhabditis elegans is the ideal candidate system as the established optophysiology techniques can capture and control each neuron's activity and scale to hundreds of thousands of experiments. Data across populations and behaviors can be combined because across individuals the nervous system is largely conserved in form and function. Modern machine-learning-based modeling should then enable a simulation of C. elegans' impressive breadth of brain states and behaviors. The ability to reverse engineer an entire nervous system will benefit the design of artificial intelligence systems and all of systems neuroscience, enabling fundamental insights as well as new approaches for investigations of progressively larger nervous systems.Comment: 23 pages, 2 figures, opinion pape

Routing brain traffic through the von Neumann bottleneck: Efficient cache usage in spiking neural network simulation code on general purpose computers

Simulation is a third pillar next to experiment and theory in the study of complex dynamic systems such as biological neural networks. Contemporary brain-scale networks correspond to directed graphs of a few million nodes, each with an in-degree and out-degree of several thousands of edges, where nodes and edges correspond to the fundamental biological units, neurons and synapses, respectively. When considering a random graph, each node's edges are distributed across thousands of parallel processes. The activity in neuronal networks is also sparse. Each neuron occasionally transmits a brief signal, called spike, via its outgoing synapses to the corresponding target neurons. This spatial and temporal sparsity represents an inherent bottleneck for simulations on conventional computers: Fundamentally irregular memory-access patterns cause poor cache utilization. Using an established neuronal network simulation code as a reference implementation, we investigate how common techniques to recover cache performance such as software-induced prefetching and software pipelining can benefit a real-world application. The algorithmic changes reduce simulation time by up to 50%. The study exemplifies that many-core systems assigned with an intrinsically parallel computational problem can overcome the von Neumann bottleneck of conventional computer architectures

Efficient Numerical Population Density Techniques with an Application in Spinal Cord Modelling

MIIND is a neural simulator which uses an innovative numerical population density technique to simulate the behaviour of multiple interacting populations of neurons under the influence of noise. Recent efforts have produced similar techniques but they are often limited to a single neuron model or type of behaviour. Extensions to these require a great deal of further work and specialist knowledge. The technique used in MIIND overcomes this limitation by being agnostic to the underlying neuron model of each population. However, earlier versions of MIIND still required a high level of technical knowledge to set up the software and involved an often time-consuming manual pre-simulation process. It was also limited to only two-dimensional neuron models. This thesis presents the development of an alternative population density technique, based on that already in MIIND, which reduces the pre-simulation step to an automated process. The new technique is much more flexible and has no limit on the number of time-dependent variables in the underlying neuron model. For the first time, the population density over the state space of the Hodgkin-Huxley neuron model can be observed in an efficient manner on a single PC. The technique allows simulation time to be significantly reduced by gracefully degrading the accuracy without losing important behavioural features. The MIIND software itself has also been simplified, reducing technical barriers to entry, so that it can now be run from a Python script and installed as a Python module. With the improved usability, a model of neural populations in the spinal cord was simulated in MIIND. It showed how afferent signals can be integrated into common reflex circuits to produce observed patterns of muscle activation during an isometric knee extension task. The influence of proprioception in motor control is not fully understood as it can be both task and subject-specific. The results of this study show that afferent signals have a significant effect on sub-maximal muscle contractions even when the limb remains static. Such signals should be considered when developing methods to improve motor control in activities of daily living via therapeutic or mechanical means