The Influence of Markov Decision Process Structure on the Possible Strategic Use of Working Memory and Episodic Memory

Researchers use a variety of behavioral tasks to analyze the effect of biological manipulations on memory function. This research will benefit from a systematic mathematical method for analyzing memory demands in behavioral tasks. In the framework of reinforcement learning theory, these tasks can be mathematically described as partially-observable Markov decision processes. While a wealth of evidence collected over the past 15 years relates the basal ganglia to the reinforcement learning framework, only recently has much attention been paid to including psychological concepts such as working memory or episodic memory in these models. This paper presents an analysis that provides a quantitative description of memory states sufficient for correct choices at specific decision points. Using information from the mathematical structure of the task descriptions, we derive measures that indicate whether working memory (for one or more cues) or episodic memory can provide strategically useful information to an agent. In particular, the analysis determines which observed states must be maintained in or retrieved from memory to perform these specific tasks. We demonstrate the analysis on three simplified tasks as well as eight more complex memory tasks drawn from the animal and human literature (two alternation tasks, two sequence disambiguation tasks, two non-matching tasks, the 2-back task, and the 1-2-AX task). The results of these analyses agree with results from quantitative simulations of the task reported in previous publications and provide simple indications of the memory demands of the tasks which can require far less computation than a full simulation of the task. This may provide a basis for a quantitative behavioral stoichiometry of memory tasks

Theorems on existence and global dynamics for the Einstein equations

This article is a guide to theorems on existence and global dynamics of solutions of the Einstein equations. It draws attention to open questions in the field. The local-in-time Cauchy problem, which is relatively well understood, is surveyed. Global results for solutions with various types of symmetry are discussed. A selection of results from Newtonian theory and special relativity that offer useful comparisons is presented. Treatments of global results in the case of small data and results on constructing spacetimes with prescribed singularity structure or late-time asymptotics are given. A conjectural picture of the asymptotic behaviour of general cosmological solutions of the Einstein equations is built up. Some miscellaneous topics connected with the main theme are collected in a separate section.Comment: Submitted to Living Reviews in Relativity, major update of Living Rev. Rel. 5 (2002)

Theorems on existence and global dynamics for the Einstein equations

This article is a guide to theorems on existence and global dynamics of solutions of the Einstein equations. It draws attention to open questions in the field. The local in time Cauchy problem, which is relatively well understood, is surveyed. Global results for solutions with various types of symmetry are discussed. A selection of results from Newtonian theory and special relativity which offer useful comparisons is presented. Treatments of global results in the case of small data and results on constructing spacetimes with prescribed singularity structure are given. A conjectural picture of the asymptotic behaviour of general cosmological solutions of the Einstein equations is built up. Some miscellaneous topics connected with the main theme are collected in a separate section.Comment: 54 pages, submitted to Living Reviews in Relativit

Abundance, population structure and claw morphology of the semi-terrestrial crab Pachygrapsus marmoratus (Fabricius, 1787) on shores of differing wave exposure

Wave action is known to influence the abundance and distribution of intertidal organisms. Wave action will also determine the duration and suitability of various foraging windows (high-tide and low-tide, day and night)for predation and can also affect predator behaviour, both directly by impeding prey handling and indirectly by influencing prey abundance. It remains uncertain whether semiterrestrial mobile predators such as crabs which can access intertidal prey during emersion when the effects of wave action are minimal, are influenced by exposure. Here, we assessed the effect of wave action on the abundance andpopulation structure (size and gender) of the semi-terrestrial intertidal crab Pachygrapsus marmoratus on rocky shores in Portugal. The activity of P. marmoratus with the tidal cycle on sheltered and exposed shores was established using baited pots at high-tide to examine whether there was activity during intertidal immersion and by low-tide searches. Because prey abundance varies along a wave exposure gradient on most Portuguese shores and because morphology of crab chelipeds are known to be related to diet composition, we further tested the hypothesis that predator stomach contents reflected differences in prey abundance along the horizontal gradient in wave exposure and that this would be correlated with the crab cheliped morphology. Thus, we examined phenotypic variation in P. marmoratus chelipeds across shores of differing exposure to wave action. P. marmoratus was only active during lowtide. Patterns of abundance and population structure of crabs did not vary with exposure to wave action. Stomach contents, however, varied significantly between shores of differing exposure with a higher consumption of hardshelled prey (mussels) on exposed locations, where this type of prey is more abundant, and a higher consumption of barnacles on sheltered shores. Multivariate geometric analysis of crab claws showed that claws were signifocantly larger on exposed shores. There was a significant correlation between animals with larger claws and the abundance of mussels in their stomach. Variation in cheliped size may have resulted from differing food availability on shelteredand exposed shores