Numerical analysis of a reinforcement learning model with the dynamic aspiration level in the iterated Prisoner's Dilemma

Humans and other animals can adapt their social behavior in response to environmental cues including the feedback obtained through experience. Nevertheless, the effects of the experience-based learning of players in evolution and maintenance of cooperation in social dilemma games remain relatively unclear. Some previous literature showed that mutual cooperation of learning players is difficult or requires a sophisticated learning model. In the context of the iterated Prisoner's Dilemma, we numerically examine the performance of a reinforcement learning model. Our model modifies those of Karandikar et al. (1998), Posch et al. (1999), and Macy and Flache (2002) in which players satisfice if the obtained payoff is larger than a dynamic threshold. We show that players obeying the modified learning mutually cooperate with high probability if the dynamics of threshold is not too fast and the association between the reinforcement signal and the action in the next round is sufficiently strong. The learning players also perform efficiently against the reactive strategy. In evolutionary dynamics, they can invade a population of players adopting simpler but competitive strategies. Our version of the reinforcement learning model does not complicate the previous model and is sufficiently simple yet flexible. It may serve to explore the relationships between learning and evolution in social dilemma situations.Comment: 7 figure

A Minimal Developmental Model Can Increase Evolvability in Soft Robots

Different subsystems of organisms adapt over many time scales, such as rapid changes in the nervous system (learning), slower morphological and neurological change over the lifetime of the organism (postnatal development), and change over many generations (evolution). Much work has focused on instantiating learning or evolution in robots, but relatively little on development. Although many theories have been forwarded as to how development can aid evolution, it is difficult to isolate each such proposed mechanism. Thus, here we introduce a minimal yet embodied model of development: the body of the robot changes over its lifetime, yet growth is not influenced by the environment. We show that even this simple developmental model confers evolvability because it allows evolution to sweep over a larger range of body plans than an equivalent non-developmental system, and subsequent heterochronic mutations 'lock in' this body plan in more morphologically-static descendants. Future work will involve gradually complexifying the developmental model to determine when and how such added complexity increases evolvability

Distributed mentoring: peer interaction and collaborative learning in P2PU

This paper explores how learning design and peer behaviour develops and evolves in a free, open online learning community, the Peer-to-peer university (P2PU). Drawing on ideas relating to 'participatory learning' (Seely-Brown and Adler, 2008), it begins with a theoretical discussion of the ways in which the infrastructural and the social dimensions of peer learning are expressed in terms of the design of three courses, and in relation to mentoring and peer interaction. Evident from the textual interface and social organization of the three courses is that the role of the instructor or course organizer adheres a cooperative model (Burge, 1994), reflected in the aggregation and filtering of materials and the evolution of pedagogical modeling. While the models of participation and engagement vary, depending on socio-technical factors, evident is that the governance model allows both for light models of involvement and the evolution of inquiry towards what we would like to call 'distributed' mentoring. We conclude with an evaluation of the ways in which the courses under study promote a participatory infrastructure, that not only can make the process of learning transparent, but also represent a relationship between teaching and learning in an open fashion

Technical change in Developing Countries: A dynamic model of adoption, learning and industry evolution

This paper develops and analyses a dynamic model, which combines both the adoption and the industry evolution theories. We model the decision of adoption, learning entry and exit of firms. These decisions depend on the interaction of technology characteristics ((effectiveness, machinery and information costs…) and other economic indicators (firm’s size, technology capability, competition concentration, returns of scale,…). We use the model’s theoretical results to analyze simultaneously the effects on the structure and the average efficiency of the industry and to develop a framework for understanding the effect of competitive policy reform and public policy action necessary to enhance adoption and average productivity. The model we suggest also analyses effects on industry evolution and social welfare.Adoption, learning, efficiency, entry , exit, industrial dynamics, evolution, developing countries

Costing the lifecycle of networked learning: Documenting the costs from conception to evaluation

This paper reports the development of a course development lifecycle model which is intended to inform the identification of ‘hidden’ costs associated with network‐based learning. The development of this model formed part of a six‐month research study funded by the Joint Information Systems Committee of the UK Funding Councils. The study aimed to produce a planning document and financial schema through which the full costs of networked learning could be documented A discussion is given of the initial five‐stage model, the testing and development of this model and the evolution of a final three‐phase model. Hypothetical examples are given of the use of the three‐phase model

Learning and innovative elements of strategy adoption rules expand cooperative network topologies

Cooperation plays a key role in the evolution of complex systems. However, the level of cooperation extensively varies with the topology of agent networks in the widely used models of repeated games. Here we show that cooperation remains rather stable by applying the reinforcement learning strategy adoption rule, Q-learning on a variety of random, regular, small-word, scale-free and modular network models in repeated, multi-agent Prisoners Dilemma and Hawk-Dove games. Furthermore, we found that using the above model systems other long-term learning strategy adoption rules also promote cooperation, while introducing a low level of noise (as a model of innovation) to the strategy adoption rules makes the level of cooperation less dependent on the actual network topology. Our results demonstrate that long-term learning and random elements in the strategy adoption rules, when acting together, extend the range of network topologies enabling the development of cooperation at a wider range of costs and temptations. These results suggest that a balanced duo of learning and innovation may help to preserve cooperation during the re-organization of real-world networks, and may play a prominent role in the evolution of self-organizing, complex systems.Comment: 14 pages, 3 Figures + a Supplementary Material with 25 pages, 3 Tables, 12 Figures and 116 reference