Numerical analysis of a reinforcement learning model with the dynamic aspiration level in the iterated Prisoner's Dilemma

Humans and other animals can adapt their social behavior in response to environmental cues including the feedback obtained through experience. Nevertheless, the effects of the experience-based learning of players in evolution and maintenance of cooperation in social dilemma games remain relatively unclear. Some previous literature showed that mutual cooperation of learning players is difficult or requires a sophisticated learning model. In the context of the iterated Prisoner's Dilemma, we numerically examine the performance of a reinforcement learning model. Our model modifies those of Karandikar et al. (1998), Posch et al. (1999), and Macy and Flache (2002) in which players satisfice if the obtained payoff is larger than a dynamic threshold. We show that players obeying the modified learning mutually cooperate with high probability if the dynamics of threshold is not too fast and the association between the reinforcement signal and the action in the next round is sufficiently strong. The learning players also perform efficiently against the reactive strategy. In evolutionary dynamics, they can invade a population of players adopting simpler but competitive strategies. Our version of the reinforcement learning model does not complicate the previous model and is sufficiently simple yet flexible. It may serve to explore the relationships between learning and evolution in social dilemma situations.Comment: 7 figure

Consistent individual differences in human social learning strategies

Social learning has allowed humans to build up extensive cultural repertoires, enabling them to adapt to a wide variety of environmental and social conditions. However, it is unclear which social learning strategies people use, especially in social contexts where their payoffs depend on the behaviour of others. Here we show experimentally that individuals differ in their social learning strategies and that they tend to employ the same learning strategy irrespective of the interaction context. Payoff-based learners focus on their peers’ success, while decision-based learners disregard payoffs and exclusively focus on their peers’ past behaviour. These individual differences may be of considerable importance for cultural evolution. By means of a simple model, we demonstrate that groups harbouring individuals with different learning strategies may be faster in adopting technological innovations and can be more efficient through successful role differentiation. Our study highlights the importance of individual variation for human interactions and sheds new light on the dynamics of cultural evolution

Learning strategies in modelling economic growth

Cornerstone economic growth models as the Solow-Swan model and their modern extensions normally assume the rate of population growth as exogenous without any explanation of the links between economic growth and most important demographic variables. Recently, some articles have presented models to explain many phenomena of population dynamics, including evolution and ageing. This paper is a first exercise to include endogenous population dynamics and learning strategies as ingredients of an economic growth model. The model includes two ways of learning that determinate economic growth: individual and social learning. We study the dynamics through computer simulations and we show that the model reflects some features of real economies.Economic Growth, Learning Strategies, Human Capital, Penna model

Rapid Evolution of Social Learning

Culture is widely thought to be beneficial when social learning is less costly than individual learning and thus may explain the enormous ecological success of humans. Rogers (1988. Does biology constrain culture. Am. Anthropol. 90: 819–831) contradicted this common view by showing that the evolution of social learning does not necessarily increase the net benefits of learned behaviours in a variable environment. Using simulation experiments, we re-analysed extensions of Rogers' model after relaxing the assumption that genetic evolution is much slower than cultural evolution. Our results show that this assumption is crucial for Rogers' finding. For many parameter settings, genetic and cultural evolution occur on the same time scale, and feedback effects between genetic and cultural dynamics increase the net benefits. Thus, by avoiding the costs of individual learning, social learning can increase ecological success. Furthermore, we found that rapid evolution can limit the evolution of complex social learning strategies, which have been proposed to be widespread in animals.Human Evolutionary Biolog

Social Preference, Incomplete Information, and the Evolution of Ultimatum Game in the Small World Networks: An Agent-Based Approach

Certain social preference models have been proposed to explain fairness behavior in experimental games. Existing bodies of research on evolutionary games, however, explain the evolution of fairness merely through the self-interest agents. This paper attempts to analyze the ultimatum game's evolution on complex networks when a number of agents display social preference. Agents' social preference is modeled in three forms: fairness consideration or maintaining a minimum acceptable money level, inequality aversion, and social welfare preference. Different from other spatial ultimatum game models, the model in this study assumes that agents have incomplete information on other agents' strategies, so the agents need to learn and develop their own strategies in this unknown environment. Genetic Algorithm Learning Classifier System algorithm is employed to address the agents' learning issue. Simulation results reveal that raising the minimum acceptable level or including fairness consideration in a game does not always promote fairness level in ultimatum games in a complex network. If the minimum acceptable money level is high and not all agents possess a social preference, the fairness level attained may be considerably lower. However, the inequality aversion social preference has negligible effect on the results of evolutionary ultimatum games in a complex network. Social welfare preference promotes the fairness level in the ultimatum game. This paper demonstrates that agents' social preference is an important factor in the spatial ultimatum game, and different social preferences create different effects on fairness emergence in the spatial ultimatum game.Spatial Ultimatum Game, Complex Network, Social Preference, Agent Based Modeling

Behavioral Modernity and the Cultural Transmission of Structured Information: The Semantic Axelrod Model

Cultural transmission models are coming to the fore in explaining increases in the Paleolithic toolkit richness and diversity. During the later Paleolithic, technologies increase not only in terms of diversity but also in their complexity and interdependence. As Mesoudi and O'Brien (2008) have shown, selection broadly favors social learning of information that is hierarchical and structured, and multiple studies have demonstrated that teaching within a social learning environment can increase fitness. We believe that teaching also provides the scaffolding for transmission of more complex cultural traits. Here, we introduce an extension of the Axelrod (1997} model of cultural differentiation in which traits have prerequisite relationships, and where social learning is dependent upon the ordering of those prerequisites. We examine the resulting structure of cultural repertoires as learning environments range from largely unstructured imitation, to structured teaching of necessary prerequisites, and we find that in combination with individual learning and innovation, high probabilities of teaching prerequisites leads to richer cultural repertoires. Our results point to ways in which we can build more comprehensive explanations of the archaeological record of the Paleolithic as well as other cases of technological change.Comment: 24 pages, 7 figures. Submitted to "Learning Strategies and Cultural Evolution during the Paleolithic", edited by Kenichi Aoki and Alex Mesoudi, and presented at the 79th Annual Meeting of the Society for American Archaeology, Austin TX. Revised 5/14/1

To copy or to innovate? The role of personality and social networks on children's learning strategies

In our technologically complex world, children frequently have problems to solve and skills to learn. They can develop solutions through learning strategies involving social learning or asocial endeavors. While evidence is emerging that children may differ individually in their propensity to adopt different learning strategies, little is known about what underlies these differences. In this article, we reflect on recent research with children, adults, and nonhuman animals regarding individual differences in learning strategies. We suggest that characteristics of children's personalities and children's positions in their social networks are pertinent to individual differences in their learning strategies. These are likely pivotal factors in the learning strategies children adopt, and thus can help us understand who copies and who innovates, an important question for cultural evolution. We also discuss how methodological issues constrain developmental researchers in this field and provide suggestions for ongoing work

The dynamics of Machiavellian intelligence

The "Machiavellian intelligence" hypothesis (or the "social brain" hypothesis) posits that large brains and distinctive cognitive abilities of humans have evolved via intense social competition in which social competitors developed increasingly sophisticated "Machiavellian" strategies as a means to achieve higher social and reproductive success. Here we build a mathematical model aiming to explore this hypothesis. In the model, genes control brains which invent and learn strategies (memes) which are used by males to gain advantage in competition for mates. We show that the dynamics of intelligence has three distinct phases. During the dormant phase only newly invented memes are present in the population. During the cognitive explosion phase the population's meme count and the learning ability, cerebral capacity (controlling the number of different memes that the brain can learn and use), and Machiavellian fitness of individuals increase in a runaway fashion. During the saturation phase natural selection resulting from the costs of having large brains checks further increases in cognitive abilities. Overall, our results suggest that the mechanisms underlying the "Machiavellian intelligence" hypothesis can indeed result in the evolution of significant cognitive abilities on the time scale of 10 to 20 thousand generations. We show that cerebral capacity evolves faster and to a larger degree than learning ability. Our model suggests that there may be a tendency toward a reduction in cognitive abilities (driven by the costs of having a large brain) as the reproductive advantage of having a large brain decreases and the exposure to memes increases in modern societies.Comment: A revised version has been published by PNA

Local and Global Interactions in an Evolutionary Resource Game

Conditions for the emergence of cooperation in a spatial common-pool resource game are studied. This combines in a unique way local and global interactions. A fixed number of harvesters are located on a spatial grid. Harvesters choose among three strategies: defection, cooperation, and enforcement. Individual payoffs are affected by both global factors, namely, aggregate harvest and resource stock level, and local factors, such as the imposition of sanctions on neighbors by enforcers. The evolution of strategies in the population is driven by social learning through imitation. Numerous types of equilibria exist in these settings. An important new finding is that clusters of cooperators and enforcers can survive among large groups of defectors. We discuss how the results contrast with the non-spatial, but otherwise similar, game of Sethi and Somanathan (1996).Common property, Cooperation, Evolutionary game theory, Global interactions, Local interactions, Social norms

The Evolution of Facultative Conformity Based on Similarity

Conformist social learning can have a pronounced impact on the cultural evolution of human societies, and it can shape both the genetic and cultural evolution of human social behavior more broadly. Conformist social learning is beneficial when the social learner and the demonstrators from whom she learns are similar in the sense that the same behavior is optimal for both. Otherwise, the social learner's optimum is likely to be rare among demonstrators, and conformity is costly. The trade-off between these two situations has figured prominently in the longstanding debate about the evolution of conformity, but the importance of the trade-off can depend critically on the flexibility of one's social learning strategy. We developed a gene-culture coevolutionary model that allows cognition to encode and process information about the similarity between naive learners and experienced demonstrators. Facultative social learning strategies that condition on perceived similarity evolve under certain circumstances. When this happens, facultative adjustments are often asymmetric. Asymmetric adjustments mean that the tendency to follow the majority when learners perceive demonstrators as similar is stronger than the tendency to follow the minority when learners perceive demonstrators as different. In an associated incentivized experiment, we found that social learners adjusted how they used social information based on perceived similarity, but adjustments were symmetric. The symmetry of adjustments completely eliminated the commonly assumed trade-off between cases in which learners and demonstrators share an optimum versus cases in which they do not. In a second experiment that maximized the potential for social learners to follow their preferred strategies, a few social learners exhibited an inclination to follow the majority. Most, however, did not respond systematically to social information. Additionally, in the complete absence of information about their similarity to demonstrators, social learners were unwilling to make assumptions about whether they shared an optimum with demonstrators. Instead, social learners simply ignored social information even though this was the only information available. Our results suggest that social cognition equips people to use conformity in a discriminating fashion that moderates the evolutionary trade-offs that would occur if conformist social learning was rigidly applied