Analyses of Markov Decision Process Structure Regarding the Possible Strategic use of Interacting Memory Systems

Behavioral tasks are often used to study the different memory systems present in humans and animals. Such tasks are usually designed to isolate and measure some aspect of a single memory system. However, it is not necessarily clear that any given task actually does isolate a system or that the strategy used by a subject in the experiment is the one desired by the experimenter. We have previously shown that when tasks are written mathematically as a form of partially observable Markov decision processes, the structure of the tasks provide information regarding the possible utility of certain memory systems. These previous analyses dealt with the disambiguation problem: given a specific ambiguous observation of the environment, is there information provided by a given memory strategy that can disambiguate that observation to allow a correct decision? Here we extend this approach to cases where multiple memory systems can be strategically combined in different ways. Specifically, we analyze the disambiguation arising from three ways by which episodic-like memory retrieval might be cued (by another episodic-like memory, by a semantic association, or by working memory for some earlier observation). We also consider the disambiguation arising from holding earlier working memories, episodic-like memories or semantic associations in working memory. From these analyses we can begin to develop a quantitative hierarchy among memory systems in which stimulus-response memories and semantic associations provide no disambiguation while the episodic memory system provides the most flexible disambiguation, with working memory at an intermediate level

Path dependence, its critics and the quest for ‘historical economics’

The concept of path dependence refers to a property of contingent, non- reversible dynamical processes, including a wide array of biological and social processes that can properly be described as 'evolutionary.' To dispell existing confusions in the literature, and clarify the meaning and significance of path dependence for economists, the paper formulates definitions that relate the phenomenon to the property of non-ergodicity in stochastic processes; it examines the nature of the relationship between between path dependence and 'market failure,' and discusses the meaning of 'lock-in.' Unlike tests for the presence of non-ergodicity, assessments of the economic significance of path dependence are shown to involve difficult issues of counterfactual specification, and the welfare evaluation of alternative dynamic paths rather than terminal states. The policy implications of the existence of path dependence are shown to be more subtle and, as a rule, quite different from those which have been presumed by critics of the concept. A concluding section applies the notion of 'lock-in' reflexively to the evolution of economic analysis, suggesting that resistence to historical economics is a manifestation of 'sunk cost hysteresis' in the sphere of human cognitive development.path dependence, non-ergodicity, irreversibility, lock-in, counterfactual analysis

The Influence of Markov Decision Process Structure on the Possible Strategic Use of Working Memory and Episodic Memory

Researchers use a variety of behavioral tasks to analyze the effect of biological manipulations on memory function. This research will benefit from a systematic mathematical method for analyzing memory demands in behavioral tasks. In the framework of reinforcement learning theory, these tasks can be mathematically described as partially-observable Markov decision processes. While a wealth of evidence collected over the past 15 years relates the basal ganglia to the reinforcement learning framework, only recently has much attention been paid to including psychological concepts such as working memory or episodic memory in these models. This paper presents an analysis that provides a quantitative description of memory states sufficient for correct choices at specific decision points. Using information from the mathematical structure of the task descriptions, we derive measures that indicate whether working memory (for one or more cues) or episodic memory can provide strategically useful information to an agent. In particular, the analysis determines which observed states must be maintained in or retrieved from memory to perform these specific tasks. We demonstrate the analysis on three simplified tasks as well as eight more complex memory tasks drawn from the animal and human literature (two alternation tasks, two sequence disambiguation tasks, two non-matching tasks, the 2-back task, and the 1-2-AX task). The results of these analyses agree with results from quantitative simulations of the task reported in previous publications and provide simple indications of the memory demands of the tasks which can require far less computation than a full simulation of the task. This may provide a basis for a quantitative behavioral stoichiometry of memory tasks

Learning in evolutionary environments

Not availabl

The Grip of History and the Scope for Novelty: Some Results and Open Questions on Path Dependence in Economic Processes

-Path dependence, irreversibility, increasing returns, learning, lock-in.

Community detection of agents interacting through regional innovation policies: structures, processes and functions

The detection of communities of agents that interacted over time through regional innovation policies is analyzed through the application of three methodologies: Clique Percolation Method (CPM) by Palla et al. (2005), Infomap by Rosvall and Bergstrom (2008), and Dynamic Cluster Index analysis (DCI) by Villani et al. (2013). In a economic context of analysis centered on such a complex object as innovation, the three methodologies are applied to investigate different specific aspects of community organizations aimed at developing innovative activities. The investigation of relational structures (through CPM), of shared processes (through Infomap) and of integrated behaviors (through DCI analysis) allowed the identification of communities that reveal, respectively, meaningful characterizations in terms of agents\u2019 participations in specific waves of the policy, of agents\u2019 participations in projects operating in particular technological domains, and in terms of agents\u2019 institutional typologies. The case study regards the policy interventions implemented by region Tuscany (Italy) in 2000-2006 with the aim of supporting innovative network projects among local actors

Three Puzzles on Mathematics, Computation, and Games

In this lecture I will talk about three mathematical puzzles involving mathematics and computation that have preoccupied me over the years. The first puzzle is to understand the amazing success of the simplex algorithm for linear programming. The second puzzle is about errors made when votes are counted during elections. The third puzzle is: are quantum computers possible?Comment: ICM 2018 plenary lecture, Rio de Janeiro, 36 pages, 7 Figure