Fashion, Cooperation, and Social Interactions

Fashion plays such a crucial rule in the evolution of culture and society that it is regarded as a second nature to the human being. Also, its impact on economy is quite nontrivial. On what is fashionable, interestingly, there are two viewpoints that are both extremely widespread but almost opposite: conformists think that what is popular is fashionable, while rebels believe that being different is the essence. Fashion color is fashionable in the first sense, and Lady Gaga in the second. We investigate a model where the population consists of the afore-mentioned two groups of people that are located on social networks (a spatial cellular automata network and small-world networks). This model captures two fundamental kinds of social interactions (coordination and anti-coordination) simultaneously, and also has its own interest to game theory: it is a hybrid model of pure competition and pure cooperation. This is true because when a conformist meets a rebel, they play the zero sum matching pennies game, which is pure competition. When two conformists (rebels) meet, they play the (anti-) coordination game, which is pure cooperation. Simulation shows that simple social interactions greatly promote cooperation: in most cases people can reach an extraordinarily high level of cooperation, through a selfish, myopic, naive, and local interacting dynamic (the best response dynamic). We find that degree of synchronization also plays a critical role, but mostly on the negative side. Four indices, namely cooperation degree, average satisfaction degree, equilibrium ratio and complete ratio, are defined and applied to measure people's cooperation levels from various angles. Phase transition, as well as emergence of many interesting geographic patterns in the cellular automata network, is also observed.Comment: 21 pages, 12 figure

Learning with Opponent-Learning Awareness

Multi-agent settings are quickly gathering importance in machine learning. This includes a plethora of recent work on deep multi-agent reinforcement learning, but also can be extended to hierarchical RL, generative adversarial networks and decentralised optimisation. In all these settings the presence of multiple learning agents renders the training problem non-stationary and often leads to unstable training or undesired final results. We present Learning with Opponent-Learning Awareness (LOLA), a method in which each agent shapes the anticipated learning of the other agents in the environment. The LOLA learning rule includes a term that accounts for the impact of one agent's policy on the anticipated parameter update of the other agents. Results show that the encounter of two LOLA agents leads to the emergence of tit-for-tat and therefore cooperation in the iterated prisoners' dilemma, while independent learning does not. In this domain, LOLA also receives higher payouts compared to a naive learner, and is robust against exploitation by higher order gradient-based methods. Applied to repeated matching pennies, LOLA agents converge to the Nash equilibrium. In a round robin tournament we show that LOLA agents successfully shape the learning of a range of multi-agent learning algorithms from literature, resulting in the highest average returns on the IPD. We also show that the LOLA update rule can be efficiently calculated using an extension of the policy gradient estimator, making the method suitable for model-free RL. The method thus scales to large parameter and input spaces and nonlinear function approximators. We apply LOLA to a grid world task with an embedded social dilemma using recurrent policies and opponent modelling. By explicitly considering the learning of the other agent, LOLA agents learn to cooperate out of self-interest. The code is at github.com/alshedivat/lola

Discrete stochastic processes, replicator and Fokker-Planck equations of coevolutionary dynamics in finite and infinite populations

Finite-size fluctuations in coevolutionary dynamics arise in models of biological as well as of social and economic systems. This brief tutorial review surveys a systematic approach starting from a stochastic process discrete both in time and state. The limit $N\to \infty$ of an infinite population can be considered explicitly, generally leading to a replicator-type equation in zero order, and to a Fokker-Planck-type equation in first order in $1/\sqrt{N}$. Consequences and relations to some previous approaches are outlined.Comment: Banach Center publications, in pres