Time representation in reinforcement learning models of the basal ganglia

Reinforcement learning (RL) models have been influential in understanding many aspects of basal ganglia function, from reward prediction to action selection. Time plays an important role in these models, but there is still no theoretical consensus about what kind of time representation is used by the basal ganglia. We review several theoretical accounts and their supporting evidence. We then discuss the relationship between RL models and the timing mechanisms that have been attributed to the basal ganglia. We hypothesize that a single computational system may underlie both RL and interval timing—the perception of duration in the range of seconds to hours. This hypothesis, which extends earlier models by incorporating a time-sensitive action selection mechanism, may have important implications for understanding disorders like Parkinson's disease in which both decision making and timing are impaired

Numerical characterization of intraoperative and chronic electrodes in deep brain stimulation

An intraoperative electrode (microelectrode) is used in the deep bra In stImulation (DBS) technique to pinpoint the brain target and to choose the best parameters for the electrical stimulus. However, when the intraoperative electrode is replaced with the chronic one (macroelectrode), the observed effects do not always coincide with predictions. To investigate the causes of such discrepancies, a 3D model of the basal ganglia has been considered and realistic models of both intraoperative and chronic electrodes have been developed and numerically solved. Results of simulations of the electric potential (V) and the activating function (AF) along neuronal fibers show that the different geometries and sizes of the two electrodes do not change the distributions and polarities of these functions, but rather the amplitudes. This effect is similar to the one produced by the presence of different tissue layers (edema or glial tissue) in the pen-electrode space. Conversely, an inaccurate positioning of the chronic electrode with respect to the intraoperative one (electric centers not coincident) may induce a completely different electric stimulation in some groups of fibers

Is there a semantic system for abstract words?

Two views on the semantics of concrete words are that their core mental representations are feature-based or are reconstructions of sensory experience. We argue that neither of these approaches is capable of representing the semantics of abstract words, which involve the representation of possibly hypothetical physical and mental states, the binding of entities within a structure, and the possible use of embedding (or recursion) in such structures. Brain based evidence in the form of dissociations between deficits related to concrete and abstract semantics corroborates the hypothesis. Neuroimaging evidence suggests that left lateral inferior frontal cortex supports those processes responsible for the representation of abstract words

TMS-induced Neural Noise in Sensory Cortex Interferes with Short-term Memory Storage in Prefrontal Cortex

In a previous study, Harris et al. (2002) found disruption of vibrotactile short-term memory after applying single-pulse transcranial magnetic stimulation (TMS) to primary somatosensory cortex (SI) early in the maintenance period, and suggested that this demonstrated a role for SI in vibrotactile memory storage. While such a role is compatible with recent suggestions that sensory cortex is the storage substrate for working memory, it stands in contrast to a relatively large body of evidence from human EEG and single-cell recording in primates that instead points to prefrontal cortex as the storage substrate for vibrotactile memory. In the present study, we use computational methods to demonstrate how Harris et al.\u27s results can be reproduced by TMS-induced activity in sensory cortex and subsequent feedforward interference with memory traces stored in prefrontal cortex, thereby reconciling discordant findings in the tactile memory literature

The influence of the noradrenergic system on optimal control of neural plasticity

Decision making under uncertainty is challenging for any autonomous agent. The challenge increases when the environment’s stochastic properties change over time, i.e., when the environment is volatile. In order to efficiently adapt to volatile environments, agents must primarily rely on recent outcomes to quickly change their decision strategies; in other words, they need to increase their knowledge plasticity. On the contrary, in stable environments, knowledge stability must be preferred to preserve useful information against noise. Here we propose that in mammalian brain, the locus coeruleus (LC) is one of the nuclei involved in volatility estimation and in the subsequent control of neural plasticity. During a reinforcement learning task, LC activation, measured by means of pupil diameter, coded both for environmental volatility and learning rate. We hypothesize that LC could be responsible, through norepinephrinic modulation, for adaptations to optimize decision making in volatile environments. We also suggest a computational model on the interaction between the anterior cingulate cortex (ACC) and LC for volatility estimation

From genes to behavior: placing cognitive models in the context of biological pathways.

Connecting neural mechanisms of behavior to their underlying molecular and genetic substrates has important scientific and clinical implications. However, despite rapid growth in our knowledge of the functions and computational properties of neural circuitry underlying behavior in a number of important domains, there has been much less progress in extending this understanding to their molecular and genetic substrates, even in an age marked by exploding availability of genomic data. Here we describe recent advances in analytical strategies that aim to overcome two important challenges associated with studying the complex relationship between genes and behavior: (i) reducing distal behavioral phenotypes to a set of molecular, physiological, and neural processes that render them closer to the actions of genetic forces, and (ii) striking a balance between the competing demands of discovery and interpretability when dealing with genomic data containing up to millions of markers. Our proposed approach involves linking, on one hand, models of neural computations and circuits hypothesized to underlie behavior, and on the other hand, the set of the genes carrying out biochemical processes related to the functioning of these neural systems. In particular, we focus on the specific example of value-based decision-making, and discuss how such a combination allows researchers to leverage existing biological knowledge at both neural and genetic levels to advance our understanding of the neurogenetic mechanisms underlying behavior

Interactions between motion and form processing in the human visual system

The predominant view of motion and form processing in the human visual system assumes that these two attributes are handled by separate and independent modules. Motion processing involves filtering by direction-selective sensors, followed by integration to solve the aperture problem. Form processing involves filtering by orientation-selective and size-selective receptive fields, followed by integration to encode object shape. It has long been known that motion signals can influence form processing in the well-known Gestalt principle of common fate; texture elements which share a common motion property are grouped into a single contour or texture region. However, recent research in psychophysics and neuroscience indicates that the influence of form signals on motion processing is more extensive than previously thought. First, the salience and apparent direction of moving lines depends on how the local orientation and direction of motion combine to match the receptive field properties of motion-selective neurons. Second, orientation signals generated by “motion-streaks” influence motion processing; motion sensitivity, apparent direction and adaptation are affected by simultaneously present orientation signals. Third, form signals generated by human body shape influence biological motion processing, as revealed by studies using point-light motion stimuli. Thus, form-motion integration seems to occur at several different levels of cortical processing, from V1 to STS