Global adaptation in networks of selfish components: emergent associative memory at the system scale

In some circumstances complex adaptive systems composed of numerous self-interested agents can self-organise into structures that enhance global adaptation, efficiency or function. However, the general conditions for such an outcome are poorly understood and present a fundamental open question for domains as varied as ecology, sociology, economics, organismic biology and technological infrastructure design. In contrast, sufficient conditions for artificial neural networks to form structures that perform collective computational processes such as associative memory/recall, classification, generalisation and optimisation, are well-understood. Such global functions within a single agent or organism are not wholly surprising since the mechanisms (e.g. Hebbian learning) that create these neural organisations may be selected for this purpose, but agents in a multi-agent system have no obvious reason to adhere to such a structuring protocol or produce such global behaviours when acting from individual self-interest. However, Hebbian learning is actually a very simple and fully-distributed habituation or positive feedback principle. Here we show that when self-interested agents can modify how they are affected by other agents (e.g. when they can influence which other agents they interact with) then, in adapting these inter-agent relationships to maximise their own utility, they will necessarily alter them in a manner homologous with Hebbian learning. Multi-agent systems with adaptable relationships will thereby exhibit the same system-level behaviours as neural networks under Hebbian learning. For example, improved global efficiency in multi-agent systems can be explained by the inherent ability of associative memory to generalise by idealising stored patterns and/or creating new combinations of sub-patterns. Thus distributed multi-agent systems can spontaneously exhibit adaptive global behaviours in the same sense, and by the same mechanism, as the organisational principles familiar in connectionist models of organismic learning

If you can't be with the one you love, love the one you're with: How individual habituation of agent interactions improves global utility

Simple distributed strategies that modify the behaviour of selfish individuals in a manner that enhances cooperation or global efficiency have proved difficult to identify. We consider a network of selfish agents who each optimise their individual utilities by coordinating (or anti-coordinating) with their neighbours, to maximise the pay-offs from randomly weighted pair-wise games. In general, agents will opt for the behaviour that is the best compromise (for them) of the many conflicting constraints created by their neighbours, but the attractors of the system as a whole will not maximise total utility. We then consider agents that act as 'creatures of habit' by increasing their preference to coordinate (anti-coordinate) with whichever neighbours they are coordinated (anti-coordinated) with at the present moment. These preferences change slowly while the system is repeatedly perturbed such that it settles to many different local attractors. We find that under these conditions, with each perturbation there is a progressively higher chance of the system settling to a configuration with high total utility. Eventually, only one attractor remains, and that attractor is very likely to maximise (or almost maximise) global utility. This counterintutitve result can be understood using theory from computational neuroscience; we show that this simple form of habituation is equivalent to Hebbian learning, and the improved optimisation of global utility that is observed results from wellknown generalisation capabilities of associative memory acting at the network scale. This causes the system of selfish agents, each acting individually but habitually, to collectively identify configurations that maximise total utility

Promotion of cooperation induced by the interplay between structure and game dynamics

We consider the coupled dynamics of the adaption of network structure and the evolution of strategies played by individuals occupying the network vertices. We propose a computational model in which each agent plays a $n$-round Prisoner's Dilemma game with its immediate neighbors, after that, based upon self-interest, partial individuals may punish their defective neighbors by dismissing the social tie to the one who defects the most times, meanwhile seek for a new partner at random from the neighbors of the punished agent. It is found that the promotion of cooperation is attributed to the entangled evolution of individual strategy and network structure. Moreover, we show that the emerging social networks exhibit high heterogeneity and disassortative mixing pattern. For a given average connectivity of the population and the number of rounds, there is a critical value for the fraction of individuals adapting their social interactions, above which cooperators wipe out defectors. Besides, the effects of the average degree, the number of rounds, and the intensity of selection are investigated by extensive numerical simulations. Our results to some extent reflect the underlying mechanism promoting cooperation.Comment: 13 pages, 6 figure

Investigating social interaction strategies for bootstrapping lexicon development

This paper investigates how different modes of social interactions influence the bootstrapping and evolution of lexicons. This is done by comparing three language game models that differ in the type of social interactions they use. The simulations show that the language games which use either joint attention or corrective feedback as a source of contextual input are better capable of bootstrapping a lexicon than the game without such directed interactions. The simulation of the latter game, however, does show that it is possible to develop a lexicon without using directed input when the lexicon is transmitted from generation to generation

Cooperation and Self-Regulation in a Model of Agents Playing Different Games

A simple model for cooperation between "selfish" agents, which play an extended version of the Prisoner's Dilemma(PD) game, in which they use arbitrary payoffs, is presented and studied. A continuous variable, representing the probability of cooperation, $p_k(t) \in$ [0,1], is assigned to each agent $k$ at time $t$. At each time step $t$ a pair of agents, chosen at random, interact by playing the game. The players update their $p_k(t)$ using a criteria based on the comparison of their utilities with the simplest estimate for expected income. The agents have no memory and use strategies not based on direct reciprocity nor 'tags'. Depending on the payoff matrix, the systems self-organizes - after a transient - into stationary states characterized by their average probability of cooperation $\bar{p}_{eq}$ and average equilibrium per-capita-income $\bar{p}_{eq},\bar{U}_\infty$. It turns out that the model exhibit some results that contradict the intuition. In particular, some games which - {\it a priory}- seems to favor defection most, may produce a relatively high degree of cooperation. Conversely, other games, which one would bet that lead to maximum cooperation, indeed are not the optimal for producing cooperation.Comment: 11 pages, 3 figures, keybords: Complex adaptive systems, Agent-based models, Social system

Individual and global adaptation in networks

The structure of complex biological and socio-economic networks affects the selective pressures or behavioural incentives of components in that network, and reflexively, the evolution/behaviour of individuals in those networks changes the structure of such networks over time. Such ‘adaptive networks’ underlie how gene-regulation networks evolve, how ecological networks self-organise, and how networks of strategic agents co-create social organisations. Although such domains are different in the details, they can each be characterised as networks of self-interested agents where agents alter network connections in the direction that increases their individual utility. Recent work shows that such dynamics are equivalent to associative learning, well-understood in the context of neural networks. Associative learning in neural substrates is the result of mandated learning rules (e.g. Hebbian learning), but in networks of autonomous agents ‘associative induction’ occurs as a result of local individual incentives to alter connections. Using results from a number of recent studies, here we review the theoretical principles that can be transferred between disciplines as a result of this isomorphism, and the implications for the organisation of genetic, social and ecological networks

Exploring the cooperative regimes in a model of agents without memory or "tags": indirect reciprocity vs. selfish incentives

The self-organization in cooperative regimes in a simple mean-field version of a model based on "selfish" agents which play the Prisoner's Dilemma (PD) game is studied. The agents have no memory and use strategies not based on direct reciprocity nor 'tags'. Two variables are assigned to each agent $i$ at time $t$, measuring its capital $C(i;t)$ and its probability of cooperation $p(i;t)$. At each time step $t$ a pair of agents interact by playing the PD game. These 2 agents update their probability of cooperation $p(i)$ as follows: they compare the profits they made in this interaction $\delta C(i;t)$ with an estimator $\epsilon(i;t)$ and, if $\delta C(i;t) \ge \epsilon(i;t)$, agent $i$ increases its $p(i;t)$ while if $\delta C(i;t) < \epsilon(i;t)$ the agent decreases $p(i;t)$. The 4!=24 different cases produced by permuting the four Prisoner's Dilemma canonical payoffs 3, 0, 1, and 5 - corresponding,respectively, to $R$ (reward), $S$ (sucker's payoff), $T$ (temptation to defect) and $P$ (punishment) - are analyzed. It turns out that for all these 24 possibilities, after a transient,the system self-organizes into a stationary state with average equilibrium probability of cooperation $\bar{p}_\infty$ = constant $> 0$.Depending on the payoff matrix, there are different equilibrium states characterized by their average probability of cooperation and average equilibrium per-capita-income ($\bar{p}_\infty,\bar{\delta C}_\infty$).Comment: 11 pages, 5 figure

Endogenous Social Preferences, Heterogeneity and Cooperation

We set up an analytical framework focusing on the problem of interaction over time when economic agents are characterized by various types of distributional social preferences. We develop an evolutionary approach in which individual preferences are endogenous and account for the evolution of cooperation when all the players are initially entirely selfish. In particular, within motivationally heterogeneous agents embedded in a social network, we adopt a variant of the indirect evolutionary approach, where material payoffs play a critical role, and assume that a coevolutionary process occurs in which subjective preferences gradually evolve due to a key mechanism involving behavioral choices, relational intensity and degree of social openness. The simulations we carried out led to strongly consistent results with regard to the evolution of player types, the dynamics of material payoffs, the creation of significant interpersonal relationships among agents and the frequency of cooperation. In the long run, cooperation turns out to be the strategic choice that obtains the best performances, in terms of material payoffs, and "nice guys", far from finishing last, succeed in coming out ahead.Behavioral Economics; Cooperation; Prisoner's Dilemma; Social Evolution; Heterogeneous Social Preferences; Indirect Evolutionary Approach