Balancing Selection Pressures, Multiple Objectives, and Neural Modularity to Coevolve Cooperative Agent Behavior

Previous research using evolutionary computation in Multi-Agent Systems indicates that assigning fitness based on team vs.\ individual behavior has a strong impact on the ability of evolved teams of artificial agents to exhibit teamwork in challenging tasks. However, such research only made use of single-objective evolution. In contrast, when a multiobjective evolutionary algorithm is used, populations can be subject to individual-level objectives, team-level objectives, or combinations of the two. This paper explores the performance of cooperatively coevolved teams of agents controlled by artificial neural networks subject to these types of objectives. Specifically, predator agents are evolved to capture scripted prey agents in a torus-shaped grid world. Because of the tension between individual and team behaviors, multiple modes of behavior can be useful, and thus the effect of modular neural networks is also explored. Results demonstrate that fitness rewarding individual behavior is superior to fitness rewarding team behavior, despite being applied to a cooperative task. However, the use of networks with multiple modules allows predators to discover intelligent behavior, regardless of which type of objectives are used

Neuroevolution in Games: State of the Art and Open Challenges

This paper surveys research on applying neuroevolution (NE) to games. In neuroevolution, artificial neural networks are trained through evolutionary algorithms, taking inspiration from the way biological brains evolved. We analyse the application of NE in games along five different axes, which are the role NE is chosen to play in a game, the different types of neural networks used, the way these networks are evolved, how the fitness is determined and what type of input the network receives. The article also highlights important open research challenges in the field.Comment: - Added more references - Corrected typos - Added an overview table (Table 1

Ms Pac-Man versus Ghost Team CEC 2011 competition

Games provide an ideal test bed for computational intelligence and significant progress has been made in recent years, most notably in games such as Go, where the level of play is now competitive with expert human play on smaller boards. Recently, a significantly more complex class of games has received increasing attention: real-time video games. These games pose many new challenges, including strict time constraints, simultaneous moves and open-endedness. Unlike in traditional board games, computational play is generally unable to compete with human players. One driving force in improving the overall performance of artificial intelligence players are game competitions where practitioners may evaluate and compare their methods against those submitted by others and possibly human players as well. In this paper we introduce a new competition based on the popular arcade video game Ms Pac-Man: Ms Pac-Man versus Ghost Team. The competition, to be held at the Congress on Evolutionary Computation 2011 for the first time, allows participants to develop controllers for either the Ms Pac-Man agent or for the Ghost Team and unlike previous Ms Pac-Man competitions that relied on screen capture, the players now interface directly with the game engine. In this paper we introduce the competition, including a review of previous work as well as a discussion of several aspects regarding the setting up of the game competition itself. © 2011 IEEE

Novelty-driven cooperative coevolution

Cooperative coevolutionary algorithms (CCEAs) rely on multiple coevolving populations for the evolution of solutions composed of coadapted components. CCEAs enable, for instance, the evolution of cooperative multiagent systems composed of heterogeneous agents, where each agent is modelled as a component of the solution. Previous works have, however, shown that CCEAs are biased toward stability: the evolutionary process tends to converge prematurely to stable states instead of (near-)optimal solutions. In this study, we show how novelty search can be used to avoid the counterproductive attraction to stable states in coevolution. Novelty search is an evolutionary technique that drives evolution toward behavioural novelty and diversity rather than exclusively pursuing a static objective. We evaluate three novelty-based approaches that rely on, respectively (1) the novelty of the team as a whole, (2) the novelty of the agents’ individual behaviour, and (3) the combination of the two. We compare the proposed approaches with traditional fitness-driven cooperative coevolution in three simulated multirobot tasks. Our results show that team-level novelty scoring is the most effective approach, significantly outperforming fitness-driven coevolution at multiple levels. Novelty-driven cooperative coevolution can substantially increase the potential of CCEAs while maintaining a computational complexity that scales well with the number of populations.info:eu-repo/semantics/publishedVersio

Динамика генотипа в нейроэволюции агентов в моделях искусственной жизни

Кооперативна поведінка є однією з найбільш часто використовуваних та поширених рис для багатоагентних систем. У деяких випадках поява такої поведінки пов’язана із поділом населення на співіснуючі субпопуляції [1, 2]. Групова взаємодія може набувати не лише форми антагоністичного конфлікту, але й зумовлюватися генетичним дрейфом, який приводить до конкуренції поведінкових стратегій та можливої асиміляції [3]. Продемонстровано різні види залежностей між групами агентів та їх поведінковими стратегіями. Використано методологію спостереження за динамікою агентного генотипу [2], відповідно до якої популяція у просторі генотипів може мати вигляд хмари точок, кожна точка якої відповідає одній особині. Розглянуто динаміку центроїда населення — центра хмари генотипу. Аналіз таких траєкторій може сприяти дослідженню різних режимів існування популяції та їх зародження.Cooperation behavior is one of the most used and spread Multi-agent system feature. In some cases emergence of this behaviour can be characterized by division of population on co-evolving subpopulations [1], [2]. Group interaction can take not only antagonistic conflict form but also genetic drift that results with strategies competition and assimilation [3]. In this work we demonstrate different relation between agent grouping and they behavior strategies. We use approach proposed in work [2] methodology of agent genotype dynamic tracking, due to this approach the evolving population can be presented in genotype space as a cloud of points where each point corresponds to one individual. In current work consider the movement of population centroid – the center of the genotype cloud. Analysis of such trajectories can shad the light on the regimes of population existence and genesis.Кооперативное поведение является одной из наиболее часто используемых и распространенных черт для многоагентных систем. В некоторых случаях появление такого поведения связано с разделением населения на сосуществующие субпопуляции [1, 2]. Групповое взаимодействие может принимать не только форму антагонистического конфликта, но и обуслoвливаться генетическим дрейфом, приводящим к конкуренции поведенческих стратегий и возможной ассимиляции [3]. Продемонстрированы различные виды зависимостей между группами агентов и их поведенческими стратегиями. Использована методология наблюдения за динамикой агентного генотипа [2], согласно которой популяция может быть представлена в пространстве генотипов в виде облака точек, где каждая точка соответствует одной особи. Рассмотрена динамика центроида популяции — центр облака генотипа. Анализ таких траекторий может помочь исследованию различных режимов существования популяции и их зарождения

Coevolution of Generative Adversarial Networks

Generative adversarial networks (GAN) became a hot topic, presenting impressive results in the field of computer vision. However, there are still open problems with the GAN model, such as the training stability and the hand-design of architectures. Neuroevolution is a technique that can be used to provide the automatic design of network architectures even in large search spaces as in deep neural networks. Therefore, this project proposes COEGAN, a model that combines neuroevolution and coevolution in the coordination of the GAN training algorithm. The proposal uses the adversarial characteristic between the generator and discriminator components to design an algorithm using coevolution techniques. Our proposal was evaluated in the MNIST dataset. The results suggest the improvement of the training stability and the automatic discovery of efficient network architectures for GANs. Our model also partially solves the mode collapse problem.Comment: Published in EvoApplications 201