Generating functionals for computational intelligence: the Fisher information as an objective function for self-limiting Hebbian learning rules

Generating functionals may guide the evolution of a dynamical system and constitute a possible route for handling the complexity of neural networks as relevant for computational intelligence. We propose and explore a new objective function, which allows to obtain plasticity rules for the afferent synaptic weights. The adaption rules are Hebbian, self-limiting, and result from the minimization of the Fisher information with respect to the synaptic flux. We perform a series of simulations examining the behavior of the new learning rules in various circumstances. The vector of synaptic weights aligns with the principal direction of input activities, whenever one is present. A linear discrimination is performed when there are two or more principal directions; directions having bimodal firing-rate distributions, being characterized by a negative excess kurtosis, are preferred. We find robust performance and full homeostatic adaption of the synaptic weights results as a by-product of the synaptic flux minimization. This self-limiting behavior allows for stable online learning for arbitrary durations. The neuron acquires new information when the statistics of input activities is changed at a certain point of the simulation, showing however, a distinct resilience to unlearn previously acquired knowledge. Learning is fast when starting with randomly drawn synaptic weights and substantially slower when the synaptic weights are already fully adapted

Slowness: An Objective for Spike-Timing-Dependent Plasticity?

Slow Feature Analysis (SFA) is an efficient algorithm for learning input-output functions that extract the most slowly varying features from a quickly varying signal. It has been successfully applied to the unsupervised learning of translation-, rotation-, and other invariances in a model of the visual system, to the learning of complex cell receptive fields, and, combined with a sparseness objective, to the self-organized formation of place cells in a model of the hippocampus. In order to arrive at a biologically more plausible implementation of this learning rule, we consider analytically how SFA could be realized in simple linear continuous and spiking model neurons. It turns out that for the continuous model neuron SFA can be implemented by means of a modified version of standard Hebbian learning. In this framework we provide a connection to the trace learning rule for invariance learning. We then show that for Poisson neurons spike-timing-dependent plasticity (STDP) with a specific learning window can learn the same weight distribution as SFA. Surprisingly, we find that the appropriate learning rule reproduces the typical STDP learning window. The shape as well as the timescale are in good agreement with what has been measured experimentally. This offers a completely novel interpretation for the functional role of spike-timing-dependent plasticity in physiological neurons

Generating functionals for autonomous latching dynamics in attractor relict networks

Coupling local, slowly adapting variables to an attractor network allows to destabilize all attractors, turning them into attractor ruins. The resulting attractor relict network may show ongoing autonomous latching dynamics. We propose to use two generating functionals for the construction of attractor relict networks, a Hopfield energy functional generating a neural attractor network and a functional based on information-theoretical principles, encoding the information content of the neural firing statistics, which induces latching transition from one transiently stable attractor ruin to the next. We investigate the influence of stress, in terms of conflicting optimization targets, on the resulting dynamics. Objective function stress is absent when the target level for the mean of neural activities is identical for the two generating functionals and the resulting latching dynamics is then found to be regular. Objective function stress is present when the respective target activity levels differ, inducing intermittent bursting latching dynamics

Self-directedness, integration and higher cognition

In this paper I discuss connections between self-directedness, integration and higher cognition. I present a model of self-directedness as a basis for approaching higher cognition from a situated cognition perspective. According to this model increases in sensorimotor complexity create pressure for integrative higher order control and learning processes for acquiring information about the context in which action occurs. This generates complex articulated abstractive information processing, which forms the major basis for higher cognition. I present evidence that indicates that the same integrative characteristics found in lower cognitive process such as motor adaptation are present in a range of higher cognitive process, including conceptual learning. This account helps explain situated cognition phenomena in humans because the integrative processes by which the brain adapts to control interaction are relatively agnostic concerning the source of the structure participating in the process. Thus, from the perspective of the motor control system using a tool is not fundamentally different to simply controlling an arm

Spiking Neural Networks for Inference and Learning: A Memristor-based Design Perspective

On metrics of density and power efficiency, neuromorphic technologies have the potential to surpass mainstream computing technologies in tasks where real-time functionality, adaptability, and autonomy are essential. While algorithmic advances in neuromorphic computing are proceeding successfully, the potential of memristors to improve neuromorphic computing have not yet born fruit, primarily because they are often used as a drop-in replacement to conventional memory. However, interdisciplinary approaches anchored in machine learning theory suggest that multifactor plasticity rules matching neural and synaptic dynamics to the device capabilities can take better advantage of memristor dynamics and its stochasticity. Furthermore, such plasticity rules generally show much higher performance than that of classical Spike Time Dependent Plasticity (STDP) rules. This chapter reviews the recent development in learning with spiking neural network models and their possible implementation with memristor-based hardware

The Future of Emotional Harm

Why should tort law treat claims for emotional harm as a second-class citizen? Judicial skepticism about these claims is long entrenched, justified by an amalgam of perceived problems ranging from proof difficulties for causation and the need to constrain fraudulent claims, to the ubiquity of the injury, and a concern about open-ended liability. To address this jumble of justifications, the law has developed a series of duty limitations to curb the claims and preclude them from reaching the jury for individualized analysis. The limited duty approach to emotional harm is maintained by the latest iteration of the Restatement (Third) of Torts. This Article argues that many of the justifications for curtailing this tort have been discredited by scientific developments. In particular, the rapid advances in neuroscience give greater insight into the changes that occur in the brain from emotional harm. Limited duty tests should no longer be used as proxies for validity or justified by the presumed untrustworthiness of the claim. Instead, validity evidence for emotional harm claims—like evidence of physical harm—should be entrusted to juries. This approach will reassert the jury’s role as the traditional factfinder, promote corrective justice and deterrence values, and lead to greater equity for negligent infliction of emotional distress (NIED) claimants. The traditional limitations on tort recovery, including the rules of evidence and causation, are more than adequate to avoid opening the floodgates to emotional distress claims

The Future of Emotional Harm

Why should tort law treat claims for emotional harm as a second-class citizen? Judicial skepticism about these claims is long entrenched, justified by an amalgam of perceived problems ranging from proof difficulties for causation and the need to constrain fraudulent claims, to the ubiquity of the injury, and a concern about open-ended liability. To address this jumble of justifications, the law has developed a series of duty limitations to curb the claims and preclude them from reaching the jury for individualized analysis. The limited duty approach to emotional harm is maintained by the latest iteration of the Restatement (Third) of Torts. This Article argues that many of the justifications for curtailing this tort have been discredited by scientific developments. In particular, the rapid advances in neuroscience give greater insight into the changes that occur in the brain from emotional harm. Limited duty tests should no longer be used as proxies for validity or justified by the presumed untrustworthiness of the claim. Instead, validity evidence for emotional harm claims—like evidence of physical harm—should be entrusted to juries. This approach will reassert the jury’s role as the traditional factfinder, promote corrective justice and deterrence values, and lead to greater equity for negligent infliction of emotional distress (NIED) claimants. The traditional limitations on tort recovery, including the rules of evidence and causation, are more than adequate to avoid opening the floodgates to emotional distress claims