The Influence of Markov Decision Process Structure on the Possible Strategic Use of Working Memory and Episodic Memory

Researchers use a variety of behavioral tasks to analyze the effect of biological manipulations on memory function. This research will benefit from a systematic mathematical method for analyzing memory demands in behavioral tasks. In the framework of reinforcement learning theory, these tasks can be mathematically described as partially-observable Markov decision processes. While a wealth of evidence collected over the past 15 years relates the basal ganglia to the reinforcement learning framework, only recently has much attention been paid to including psychological concepts such as working memory or episodic memory in these models. This paper presents an analysis that provides a quantitative description of memory states sufficient for correct choices at specific decision points. Using information from the mathematical structure of the task descriptions, we derive measures that indicate whether working memory (for one or more cues) or episodic memory can provide strategically useful information to an agent. In particular, the analysis determines which observed states must be maintained in or retrieved from memory to perform these specific tasks. We demonstrate the analysis on three simplified tasks as well as eight more complex memory tasks drawn from the animal and human literature (two alternation tasks, two sequence disambiguation tasks, two non-matching tasks, the 2-back task, and the 1-2-AX task). The results of these analyses agree with results from quantitative simulations of the task reported in previous publications and provide simple indications of the memory demands of the tasks which can require far less computation than a full simulation of the task. This may provide a basis for a quantitative behavioral stoichiometry of memory tasks

Dot Display Affects Approximate Number System Acuity and Relationships with Mathematical Achievement and Inhibitory Control

Much research has investigated the relationship between the Approximate Number System (ANS) and mathematical achievement, with continued debate surrounding the existence of such a link. The use of different stimulus displays may account for discrepancies in the findings. Indeed, closer scrutiny of the literature suggests that studies supporting a link between ANS acuity and mathematical achievement in adults have mostly measured the ANS using spatially intermixed displays (e.g. of blue and yellow dots), whereas those failing to replicate a link have primarily used spatially separated dot displays. The current study directly compared ANS acuity when using intermixed or separate dots, investigating how such methodological variation mediated the relationship between ANS acuity and mathematical achievement. ANS acuity was poorer and less reliable when measured with intermixed displays, with performance during both conditions related to inhibitory control. Crucially, mathematical achievement was significantly related to ANS accuracy difference (accuracy on congruent trials minus accuracy on incongruent trials) when measured with intermixed displays, but not with separate displays. The findings indicate that methodological variation affects ANS acuity outcomes, as well as the apparent relationship between the ANS and mathematical achievement. Moreover, the current study highlights the problem of low reliabilities of ANS measures. Further research is required to construct ANS measures with improved reliability, and to understand which processes may be responsible for the increased likelihood of finding a correlation between the ANS and mathematical achievement when using intermixed displays

Grid Cells, Place Cells, and Geodesic Generalization for Spatial Reinforcement Learning

Reinforcement learning (RL) provides an influential characterization of the brain's mechanisms for learning to make advantageous choices. An important problem, though, is how complex tasks can be represented in a way that enables efficient learning. We consider this problem through the lens of spatial navigation, examining how two of the brain's location representations—hippocampal place cells and entorhinal grid cells—are adapted to serve as basis functions for approximating value over space for RL. Although much previous work has focused on these systems' roles in combining upstream sensory cues to track location, revisiting these representations with a focus on how they support this downstream decision function offers complementary insights into their characteristics. Rather than localization, the key problem in learning is generalization between past and present situations, which may not match perfectly. Accordingly, although neural populations collectively offer a precise representation of position, our simulations of navigational tasks verify the suggestion that RL gains efficiency from the more diffuse tuning of individual neurons, which allows learning about rewards to generalize over longer distances given fewer training experiences. However, work on generalization in RL suggests the underlying representation should respect the environment's layout. In particular, although it is often assumed that neurons track location in Euclidean coordinates (that a place cell's activity declines “as the crow flies” away from its peak), the relevant metric for value is geodesic: the distance along a path, around any obstacles. We formalize this intuition and present simulations showing how Euclidean, but not geodesic, representations can interfere with RL by generalizing inappropriately across barriers. Our proposal that place and grid responses should be modulated by geodesic distances suggests novel predictions about how obstacles should affect spatial firing fields, which provides a new viewpoint on data concerning both spatial codes

The role of cognitive inhibition in different components of arithmetic

This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Research has established that executive functions, the skills required to monitor and control thought and action, are related to achievement in mathematics. Until recently research has focused on working memory, but studies are beginning to show that inhibition skills—the ability to suppress distracting information and unwanted responses—are also important for mathematics. However, these studies employed general mathematics tests and therefore are unable to pinpoint how inhibition skills relate to specific components of mathematics. We explored how inhibition skills are related to overall achievement as well as factual, procedural and conceptual knowledge in 209 participants aged 11–12, 13–14 and adults. General mathematics achievement was more strongly related to inhibition measured in numerical compared with non-numerical contexts. Inhibition skills were related to conceptual knowledge in older participants, but procedural skills in younger participants. These differing relationships can shed light on the mechanisms by which inhibition is involved in mathematics

Independent adaptation mechanisms for numerosity and size perception provide evidence against a common sense of magnitude

Abstract How numerical quantity is processed is a central issue for cognition. On the one hand the “number sense theory” claims that numerosity is perceived directly, and may represent an early precursor for acquisition of mathematical skills. On the other, the “theory of magnitude” notes that numerosity correlates with many continuous properties such as size and density, and may therefore not exist as an independent feature, but be part of a more general system of magnitude. In this study we examined interactions in sensitivity between numerosity and size perception. In a group of children, we measured psychophysically two sensory parameters: perceptual adaptation and discrimination thresholds for both size and numerosity. Neither discrimination thresholds nor adaptation strength for numerosity and size correlated across participants. This clear lack of correlation (confirmed by Bayesian analyses) suggests that numerosity and size interference effects are unlikely to reflect a shared sensory representation. We suggest these small interference effects may rather result from top-down phenomena occurring at late decisional levels rather than a primary “sense of magnitude”