6,380 research outputs found
Segregation Dynamics with Reinforcement Learning and Agent Based Modeling
Societies are complex. Properties of social systems can be explained by the
interplay and weaving of individual actions. Incentives are key to understand
people's choices and decisions. For instance, individual preferences of where
to live may lead to the emergence of social segregation. In this paper, we
combine Reinforcement Learning (RL) with Agent Based Models (ABM) in order to
address the self-organizing dynamics of social segregation and explore the
space of possibilities that emerge from considering different types of
incentives. Our model promotes the creation of interdependencies and
interactions among multiple agents of two different kinds that want to
segregate from each other. For this purpose, agents use Deep Q-Networks to make
decisions based on the rules of the Schelling Segregation model and the
Predator-Prey model. Despite the segregation incentive, our experiments show
that spatial integration can be achieved by establishing interdependencies
among agents of different kinds. They also reveal that segregated areas are
more probable to host older people than diverse areas, which attract younger
ones. Through this work, we show that the combination of RL and ABMs can create
an artificial environment for policy makers to observe potential and existing
behaviors associated to incentives.Comment: 14 pages, 4 figures + supplemental material, in revie
ACE Models of Endogenous Interactions
Various approaches used in Agent-based Computational Economics (ACE) to model endogenously determined interactions between agents are discussed. This concerns models in which agents not only (learn how to) play some (market or other) game, but also (learn to) decide with whom to do that (or not).Endogenous interaction, Agent-based Computational Economics (ACE)
Techniques to Understand Computer Simulations: Markov Chain Analysis
The aim of this paper is to assist researchers in understanding the dynamics of simulation models that have been implemented and can be run in a computer, i.e. computer models. To do that, we start by explaining (a) that computer models are just input-output functions, (b) that every computer model can be re-implemented in many different formalisms (in particular in most programming languages), leading to alternative representations of the same input-output relation, and (c) that many computer models in the social simulation literature can be usefully represented as time-homogeneous Markov chains. Then we argue that analysing a computer model as a Markov chain can make apparent many features of the model that were not so evident before conducting such analysis. To prove this point, we present the main concepts needed to conduct a formal analysis of any time-homogeneous Markov chain, and we illustrate the usefulness of these concepts by analysing 10 well-known models in the social simulation literature as Markov chains. These models are: • Schelling\'s (1971) model of spatial segregation • Epstein and Axtell\'s (1996) Sugarscape • Miller and Page\'s (2004) standing ovation model • Arthur\'s (1989) model of competing technologies • Axelrod\'s (1986) metanorms models • Takahashi\'s (2000) model of generalized exchange • Axelrod\'s (1997) model of dissemination of culture • Kinnaird\'s (1946) truels • Axelrod and Bennett\'s (1993) model of competing bimodal coalitions • Joyce et al.\'s (2006) model of conditional association In particular, we explain how to characterise the transient and the asymptotic dynamics of these computer models and, where appropriate, how to assess the stochastic stability of their absorbing states. In all cases, the analysis conducted using the theory of Markov chains has yielded useful insights about the dynamics of the computer model under study.Computer Modelling, Simulation, Markov, Stochastic Processes, Analysis, Re-Implementation
The impact of moving expenses on social segregation: a simulation with RL and ABM
Over the past decades, breakthroughs such as Reinforcement Learning (RL) and
Agent-based modeling (ABM) have made simulations of economic models feasible.
Recently, there has been increasing interest in applying ABM to study the
impact of residential preferences on neighborhood segregation in the Schelling
Segregation Model. In this paper, RL is combined with ABM to simulate a
modified Schelling Segregation model, which incorporates moving expenses as an
input parameter. In particular, deep Q network (DQN) is adopted as RL agents'
learning algorithm to simulate the behaviors of households and their
preferences. This paper studies the impact of moving expenses on the overall
segregation pattern and its role in social integration. A more comprehensive
simulation of the segregation model is built for policymakers to forecast the
potential consequences of their policies.Comment: 7 pages with 1 table and 1 figur
Opinion Polarization by Learning from Social Feedback
We explore a new mechanism to explain polarization phenomena in opinion
dynamics in which agents evaluate alternative views on the basis of the social
feedback obtained on expressing them. High support of the favored opinion in
the social environment, is treated as a positive feedback which reinforces the
value associated to this opinion. In connected networks of sufficiently high
modularity, different groups of agents can form strong convictions of competing
opinions. Linking the social feedback process to standard equilibrium concepts
we analytically characterize sufficient conditions for the stability of
bi-polarization. While previous models have emphasized the polarization effects
of deliberative argument-based communication, our model highlights an affective
experience-based route to polarization, without assumptions about negative
influence or bounded confidence.Comment: Presented at the Social Simulation Conference (Dublin 2017
Dynamical selection of Nash equilibria using Experience Weighted Attraction Learning: emergence of heterogeneous mixed equilibria
We study the distribution of strategies in a large game that models how
agents choose among different double auction markets. We classify the possible
mean field Nash equilibria, which include potentially segregated states where
an agent population can split into subpopulations adopting different
strategies. As the game is aggregative, the actual equilibrium strategy
distributions remain undetermined, however. We therefore compare with the
results of Experience-Weighted Attraction (EWA) learning, which at long times
leads to Nash equilibria in the appropriate limits of large intensity of
choice, low noise (long agent memory) and perfect imputation of missing scores
(fictitious play). The learning dynamics breaks the indeterminacy of the Nash
equilibria. Non-trivially, depending on how the relevant limits are taken, more
than one type of equilibrium can be selected. These include the standard
homogeneous mixed and heterogeneous pure states, but also \emph{heterogeneous
mixed} states where different agents play different strategies that are not all
pure. The analysis of the EWA learning involves Fokker-Planck modeling combined
with large deviation methods. The theoretical results are confirmed by
multi-agent simulations.Comment: 35 pages, 16 figure
- …