Deep Reinforcement Learning for Swarm Systems

Recently, deep reinforcement learning (RL) methods have been applied successfully to multi-agent scenarios. Typically, these methods rely on a concatenation of agent states to represent the information content required for decentralized decision making. However, concatenation scales poorly to swarm systems with a large number of homogeneous agents as it does not exploit the fundamental properties inherent to these systems: (i) the agents in the swarm are interchangeable and (ii) the exact number of agents in the swarm is irrelevant. Therefore, we propose a new state representation for deep multi-agent RL based on mean embeddings of distributions. We treat the agents as samples of a distribution and use the empirical mean embedding as input for a decentralized policy. We define different feature spaces of the mean embedding using histograms, radial basis functions and a neural network learned end-to-end. We evaluate the representation on two well known problems from the swarm literature (rendezvous and pursuit evasion), in a globally and locally observable setup. For the local setup we furthermore introduce simple communication protocols. Of all approaches, the mean embedding representation using neural network features enables the richest information exchange between neighboring agents facilitating the development of more complex collective strategies.Comment: 31 pages, 12 figures, version 3 (published in JMLR Volume 20

Fostering shared knowledge with active graphical representation in different collaboration scenarios

This study investigated how two types of graphical representation tools influence the way in which learners use shared and unshared knowledge resources in two different collaboration scenarios, and how learners represent and transfer shared knowledge under these different conditions. Moreover, the relation between the use of knowledge resources, representation, and the transfer of shared knowledge was analyzed. The type of graphical representation (content-specific vs. content-unspecific) and the collaboration scenario (video conferencing vs. face-to-face) were varied. 64 university students participated. Results show that the learning partners converged in their profiles of resource use. With the content-specific graphical representation, learners used more appropriate knowledge resources. Learners in the computer-mediated scenarios showed a greater bandwidth in their profiles of resource use. A relation between discourse and outcomes could be shown for the transfer but not for the knowledge representation aspectIn dieser Studie werden die Wirkungen von verschiedenen Arten graphischer Repräsentation auf die Nutzung geteilter und ungeteilter Wissensressourcen in zwei verschiedenen Kooperationsszenarien untersucht. Des Weiteren wird analysiert, wie Lernende geteiltes und ungeteiltes Wissen unter diesen verschiedenen Bedingungen repräsentieren und transferieren. Schließlich wird die Beziehung zwischen der Nutzung von Wissensressourcen auf der einen Seite sowie der Repräsentation und dem Transfer geteilten Wissens auf der anderen Seite geprüft. Mit der Art der graphischen Repräsentation (inhaltsspezifisch vs. inhaltsunspezifisch) und dem Kooperationsszenario (Videokonferenz vs. face-to-face) werden zwei Faktoren experimentell variiert. 64 Studierende nahmen an der Studie teil. Ergebnisse zeigen, dass die Lernpartner in ihren Profilen der Ressourcennutzung konvergierten. Lernende, die durch die inhaltsspezifische graphische Repräsentation unterstützt wurden, verwendeten angemessenere Wissensressourcen. Lernende in den computervermittelten Szenarien weisen eine größere Bandbreite in ihren Profilen der Ressourcennutzung auf. Eine direkte Wirkung vom Diskurs der Lernenden auf die Entwicklung geteilten Wissens konnte für den Transfer, aber nicht für die Wissensrepräsentation gezeigt werde

Learning and innovative elements of strategy adoption rules expand cooperative network topologies

Cooperation plays a key role in the evolution of complex systems. However, the level of cooperation extensively varies with the topology of agent networks in the widely used models of repeated games. Here we show that cooperation remains rather stable by applying the reinforcement learning strategy adoption rule, Q-learning on a variety of random, regular, small-word, scale-free and modular network models in repeated, multi-agent Prisoners Dilemma and Hawk-Dove games. Furthermore, we found that using the above model systems other long-term learning strategy adoption rules also promote cooperation, while introducing a low level of noise (as a model of innovation) to the strategy adoption rules makes the level of cooperation less dependent on the actual network topology. Our results demonstrate that long-term learning and random elements in the strategy adoption rules, when acting together, extend the range of network topologies enabling the development of cooperation at a wider range of costs and temptations. These results suggest that a balanced duo of learning and innovation may help to preserve cooperation during the re-organization of real-world networks, and may play a prominent role in the evolution of self-organizing, complex systems.Comment: 14 pages, 3 Figures + a Supplementary Material with 25 pages, 3 Tables, 12 Figures and 116 reference

Epistemic and social scripts in computer-supported collaborative learning

Collaborative learning in computer-supported learning environments typically means that learners work on tasks together, discussing their individual perspectives via text-based media or videoconferencing, and consequently acquire knowledge. Collaborative learning, however, is often sub-optimal with respect to how learners work on the concepts that are supposed to be learned and how learners interact with each other. One possibility to improve collaborative learning environments is to conceptualize epistemic scripts, which specify how learners work on a given task, and social scripts, which structure how learners interact with each other. In this contribution, two studies will be reported that investigated the effects of epistemic and social scripts in a text-based computer-supported learning environment and in a videoconferencing learning environment in order to foster the individual acquisition of knowledge. In each study the factors ‘epistemic script’ and ‘social script’ have been independently varied in a 2×2-factorial design. 182 university students of Educational Science participated in these two studies. Results of both studies show that social scripts can be substantially beneficial with respect to the individual acquisition of knowledge, whereas epistemic scripts apparently do not to lead to the expected effects

Adopting appropriate teaching models to develop knowledge and skills to academic standards in the accounting discipline

Cooperative learning models of teaching are the most suitable teaching models for the development of professional accounting competencies in the accounting discipline. Currently, the role of accountants has changed from being a technical job to more client-oriented job. The teaching and learning of accounting has been changing to match the challenges of this new accounting role. Universities are searching for a number of strategies to teach the professional accounting competencies that are required. The Australian accounting teaching and learning standards provide a thorough set of criteria for determining what is necessary in accounting education. Joyce, Weil, and Calhoun categorised a wide variety of teaching models into four families including: information processing, behavioural, personal, and social models. This paper applied the Australian accounting teaching and learning standards criteria to the models of teaching by Joyce, Weil and Calhoun to evaluate which teaching and learning model would be most appropriate to teach future accountants. The findings indicate that the social interdependence theory and the cooperative learning model are the most appropriate to test for teaching accounting in the accounting discipline