How Suitable Is Your Naturalistic Dataset for Theory-based User Modeling?

Peer reviewe

Modeling Risky Choices in Unknown Environments

Decision-theoretic models explain human behavior in choice problems involving uncertainty, in terms of individual tendencies such as risk aversion. However, many classical models of risk require knowing the distribution of possible outcomes (rewards) for all options, limiting their applicability outside of controlled experiments. We study the task of learning such models in contexts where the modeler does not know the distributions but instead can only observe the choices and their outcomes for a user familiar with the decision problems, for example a skilled player playing a digital game. We propose a framework combining two separate components, one for modeling the unknown decision-making environment and another for the risk behavior. By using environment models capable of learning distributions we are able to infer classical models of decision-making under risk from observations of the user's choices and outcomes alone, and we also demonstrate alternative models for predictive purposes. We validate the approach on artificial data and demonstrate a practical use case in modeling risk attitudes of professional esports teams.Peer reviewe

An Empirical Investigation of the Emergence of Money: Contrasting Temporal Difference and Opportunity Cost Reinforcement Learning

Money is a fundamental and ubiquitous institution in modern economies. However, the question of its emergence remains a central one for economists. The monetary search-theoretic approach studies the conditions under which commodity money emerges as a solution to override frictions inherent to inter-individual exchanges in a decentralized economy. Although among these conditions, agents' rationality is classically essential and a prerequisite to any theoretical monetary equilibrium, human subjects often fail to adopt optimal strategies in tasks implementing a search-theoretic paradigm when these strategies are speculative, i.e., involve the use of a costly medium of exchange to increase the probability of subsequent and successful trades. In the present work, we hypothesize that implementing such speculative behaviors relies on reinforcement learning instead of lifetime utility calculations, as supposed by classical economic theory. To test this hypothesis, we operationalized the Kiyotaki and Wright paradigm of money emergence in a multi-step exchange task and fitted behavioral data regarding human subjects performing this task with two reinforcement learning models. Each of them implements a distinct cognitive hypothesis regarding the weight of future or counterfactual rewards in current decisions. We found that both models outperformed theoretical predictions about subjects' behaviors regarding the implementation of speculative strategies and that the latter relies on the degree of the opportunity costs consideration in the learning process. Speculating about the marketability advantage of money thus seems to depend on mental simulations of counterfactual events that agents are performing in exchange situations

Coordination over a unique medium of exchange under information scarcity

International audienceSeveral micro-founded macroeconomic models with rational expectations address the issue of money emergence, by characterizing it as a coordination game. ese models have in common the use of agents who dispose of perfect or near-perfect information on the global state of the economy and who display full-edged computational abilities. Several experimental studies have shown that a simple trial-and-error learning process could constitute an explanation for how agents coordinate on a single mean of exchange (Brown, 1996; Du y, 2001; Kindler et al., 2017; Lefebvre et al., 2018). However, these studies provide subjects with full information regarding the state of the economy while restricting the number of goods in circulation to three. In this study, by the mean of multi-agent simulations and human experiments, we test the hypothesis according to which coordination over a unique medium of exchange is possible in the context of information scarcity. In our experimental design, subjects and arti cial agents are only aware of the outcome of their own decisions. We provide results for economies with 3 and 4 goods to evaluate to which extent it is possible to generalize results obtained with 3 goods to n goods. Our ndings show that in an economyà la Iwai (1996), commodity money can emerge under drastic information restrictions with 3 goods in circulation, but generalization to 4 or more goods is not guaranteed

The adaptive value of probability distortion and risk-seeking in macaques' decision-making

International audienceIn humans, the attitude toward risk is not neutral and is dissimilar between bets involving gains and bets involving losses. The existence and prevalence of these decision features in non-human primates are unclear. In addition, only a few studies have tried to simulate the evolution of agents based on their attitude toward risk. Therefore, we still ignore to which extent Prospect theory’s claims are evolutionary rooted. To shed light on this issue, we collected data in 9 macaques that performed bets involving gains or losses. We confirmed that their overall behaviour is coherent with Prospect theory’s claims. In parallel, we used a genetic algorithm to simulate the evolution of a population of agents across several generations. We showed that the algorithm selects progressively agents that exhibit risk-seeking and an inverted S-shape distorted perception of probability. We compared these two results and found that monkeys' attitude toward risk when facing losses only is congruent with the simulation. This result is consistent with the idea that gambling in the loss domain is analogous to deciding in a context of life-threatening challenges where a certain level of risk-seeking behaviours and probability distortions may be adaptive

Exercer son libre arbitre : un processus décisionnel

La question de savoir s'il est possible d'agir selon son propre libre arbitre continue de faire l’objet de nombreuses discussions, formelles et plus informelles, sérieuses et moins sérieuses et les positions adoptées par les protagonistes de telles discussions font souvent preuve d'une grande diversité. Une manière de situer ces positions les unes par rapport aux autres est de considérer l'attitude prise par rapport à l'argument suivant qui défend une incompatibilité entre exercice du libre a..

De l’homo-œconomicus au primate non-humain : trois études de cas sur les micro-fondations cognitives de l’économie

Through three studies, this thesis aims to explore the cognitive micro-foundations of economics. In a first study, I investigate the role of the information for coordination on a unique medium of exchange, that is to say money emergence. Relying on the search theoretical models (Kiyotaki & Wright, 1989, and Iwai, 1996), the goal of this study is to challenge the assumption that an exhaustive information is a necessary condition for money emergence. In a second study, I tackle the role of the information in duopoly competition. Using a model a-la-Hotelling (1929), we test the hypothesis that varying the amount of information available by consumers substantially impacts market’s dynamics. In a third study, I am interested in decision-making under risk in rhesus monkeys. Based on the prospect theory (Kahneman & Tversky, 1989, 1992), the main purpose is to assess to what extent macaques exhibit an asymmetric treatment of gains and losses similar to that of humans.A travers trois études, cette thèse vise à explorer les micro-fondations cognitives de l'économie. Dans une première étude, j'examine le rôle de l'information dans la coordination sur un moyen d'échange unique, c'est-à-dire l'émergence d'une monnaie. En s'appuyant sur les modèles de prospection monétaire (Kiyotaki & Wright, 1989, et Iwai, 1996), l'objectif de cette étude est de remettre en question l'hypothèse selon laquelle une information exhaustive est une condition nécessaire à l'émergence d'une monnaie. Dans une deuxième étude, j'aborde le rôle de l'information dans une situation de concurrence duopolistique. À l'aide d'un modèle à la Hotelling (1929), nous vérifions l'hypothèse selon laquelle la variation de la quantité d'information accessible aux consommateurs influe sur la dynamique du marché. Dans une troisième étude, je m'intéresse à la prise de décision dans le risque chez le macaque rhésus. Prenant appui sur la théorie des perspectives (Kahneman & Tversky, 1989, 1992), l'objectif principal est d'examiner dans quelle mesure les macaques font preuve d'un traitement asymétrique des gains et des pertes similaire à celui des humains

[Re] Faster Teaching via POMDP Planning

Peer reviewe

Les attitudes réactives et le déterminisme : une étude expérimentale

1. Introduction Le fait d’attribuer à un agent une responsabilité pour ses actions, comme de le punir pour un crime qu’il a commis, ou de le louer pour une action bonne, implique-t-il qu’il ait agi de manière non déterminée ? Selon le point de vue incompatibiliste, la réponse à cette question est positive : dans un monde où les actions humaines sont soumises au déterminisme, l’attribution d’éloge ou de blâme perdrait son fondement, car la notion même de responsabilité serait transférée de l’a..