Balancing Selection Pressures, Multiple Objectives, and Neural Modularity to Coevolve Cooperative Agent Behavior

Previous research using evolutionary computation in Multi-Agent Systems indicates that assigning fitness based on team vs.\ individual behavior has a strong impact on the ability of evolved teams of artificial agents to exhibit teamwork in challenging tasks. However, such research only made use of single-objective evolution. In contrast, when a multiobjective evolutionary algorithm is used, populations can be subject to individual-level objectives, team-level objectives, or combinations of the two. This paper explores the performance of cooperatively coevolved teams of agents controlled by artificial neural networks subject to these types of objectives. Specifically, predator agents are evolved to capture scripted prey agents in a torus-shaped grid world. Because of the tension between individual and team behaviors, multiple modes of behavior can be useful, and thus the effect of modular neural networks is also explored. Results demonstrate that fitness rewarding individual behavior is superior to fitness rewarding team behavior, despite being applied to a cooperative task. However, the use of networks with multiple modules allows predators to discover intelligent behavior, regardless of which type of objectives are used

An artificial life approach to evolutionary computation: from mobile cellular algorithms to artificial ecosystems

This thesis presents a new class of evolutionary algorithms called mobile cellular evolutionary algorithms (mcEAs). These algorithms are characterized by individuals moving around on a spatial population structure. As a primary objective, this thesis aims to show that by controlling the population density and mobility in mcEAs, it is possible to achieve much better control over the rate of convergence than what is already possible in existing cellular EAs. Using the observations and results from this investigation into selection pressure in mcEAs, a general architecture for developing agent-based evolutionary algorithms called Artificial Ecosystems (AES) is presented. A simple agent-based EA is developed within the scope of AES is presented with two individual-based bottom-up schemes to achieve dynamic population sizing. Experiments with a test suite of optimization problems show that both mcEAs and the agent-based EA produced results comparable to the best solutions found by cellular EAs --Abstract, page iii

On Agent Communication in Large Groups

The problem is fundamental and natural, yet deep - to simulate the simplest possible form of communication that can occur within a large multi-agent system. It would be prohibitive to try and survey all of the research on communication in general so we must restrict our focus. We will devote our efforts to synthetic communication occurring within large groups. In particular, we would like to discover a model for communication that will serve as an abstract model, a prototype, for simulating communication within large groups of biological organisms

A framework for teaching biology using StarLogo TNG : from DNA to evolution

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2009.Includes bibliographical references (p. 65-66).This thesis outlines a 10-unit biology curriculum implemented in StarLogo TNG. The curriculum moves through units on ecology, the DNA-protein relationship, and evolution. By combining the three topics, it aims to highlight the similarities among different scales and the relationships between them. In particular, through the curriculum, students can see how small-scale changes in molecular processes can create large-scale changes in entire populations. In addition, the curriculum encourages students to engage in problembased learning, by which they are trained to approach questions creatively and independently.by Yaa-Lirng Tu.M.Eng

Novelty-driven cooperative coevolution

Cooperative coevolutionary algorithms (CCEAs) rely on multiple coevolving populations for the evolution of solutions composed of coadapted components. CCEAs enable, for instance, the evolution of cooperative multiagent systems composed of heterogeneous agents, where each agent is modelled as a component of the solution. Previous works have, however, shown that CCEAs are biased toward stability: the evolutionary process tends to converge prematurely to stable states instead of (near-)optimal solutions. In this study, we show how novelty search can be used to avoid the counterproductive attraction to stable states in coevolution. Novelty search is an evolutionary technique that drives evolution toward behavioural novelty and diversity rather than exclusively pursuing a static objective. We evaluate three novelty-based approaches that rely on, respectively (1) the novelty of the team as a whole, (2) the novelty of the agents’ individual behaviour, and (3) the combination of the two. We compare the proposed approaches with traditional fitness-driven cooperative coevolution in three simulated multirobot tasks. Our results show that team-level novelty scoring is the most effective approach, significantly outperforming fitness-driven coevolution at multiple levels. Novelty-driven cooperative coevolution can substantially increase the potential of CCEAs while maintaining a computational complexity that scales well with the number of populations.info:eu-repo/semantics/publishedVersio

Emergent Behavior Development and Control in Multi-Agent Systems

Emergence in natural systems is the development of complex behaviors that result from the aggregation of simple agent-to-agent and agent-to-environment interactions. Emergence research intersects with many disciplines such as physics, biology, and ecology and provides a theoretical framework for investigating how order appears to spontaneously arise in complex adaptive systems. In biological systems, emergent behaviors allow simple agents to collectively accomplish multiple tasks in highly dynamic environments; ensuring system survival. These systems all display similar properties: self-organized hierarchies, robustness, adaptability, and decentralized task execution. However, current algorithmic approaches merely present theoretical models without showing how these models actually create hierarchical, emergent systems. To fill this research gap, this dissertation presents an algorithm based on entropy and speciation - defined as morphological or physiological differences in a population - that results in hierarchical emergent phenomena in multi-agent systems. Results show that speciation creates system hierarchies composed of goal-aligned entities, i.e. niches. As niche actions aggregate into more complex behaviors, more levels emerge within the system hierarchy, eventually resulting in a system that can meet multiple tasks and is robust to environmental changes. Speciation provides a powerful tool for creating goal-aligned, decentralized systems that are inherently robust and adaptable, meeting the scalability demands of current, multi-agent system design. Results in base defense, k-n assignment, division of labor and resource competition experiments, show that speciated populations create hierarchical self-organized systems, meet multiple tasks and are more robust to environmental change than non-speciated populations

A METHODOLOGY FOR TECHNOLOGY-TUNED DECISION BEHAVIOR ALGORITHMS FOR TACTICS EXPLORATION

In 2016, the USAF found that current development and acquisition methods may be inadequate to achieve air superiority in 2030. The airspace is expected to be highly contested by 2030 due to the Anti-Access/Area Denial strategies being employed by adversaries. Capability gaps must be addressed in order to maintain air superiority. The USAF identified new development and acquisition paradigms as the number one non-material capability development area. The idea of a new development and acquisition paradigm is not new. Such a paradigm shift occurred during the transition from threat-based acquisition during the cold war to capability-based acquisition during the war on terror. Investigation into current US development and acquisition methods found several notional methodologies. Effectiveness-Based Design and Technology Identification, Evaluation, and Selection for Systems-of-Systems have been proposed as notional solutions. Both methodologies seek to evaluate the means – the technologies used to perform a mission – and the ways – the tactics used to complete a mission – of the technology design space. Proper evaluation of the ways would provide critical information to the decision-maker during technology selection. These findings suggest that a new paradigm focused on effectiveness-based acquisition is needed to improve current development and acquisition methods. To evaluate the ways design space, current methods must move away from a fixed or constrained mission model to one that is minimally defined and capable of exploring tactics for each unique technology. The proposed Technology-tuned Decision Behavior Algorithms for Tactics Exploration (Tech-DEBATE) methodology enables the exploration of the ways, or more formally, the mission action design space. The methodology enables further exploration of the technology design space by improving the quantification of mission effectiveness through deep reinforcement learning in a minimally defined mission environment. The data's foundation is based on traceable tactical alternatives that increase the confidence in the measures of effectiveness for each technology-tactic alternative. The methodology enables more informed decisions for technology investment, thereby reducing risks in the development and acquisition of new technologies. The reduction in risk inherently reduces the costs and development time associated with investment in new technologies. The Tech-DEBATE methodology provides a new methodology for technology evaluation through its emphasis on quantifying mission effectiveness in a minimally defined mission to inform technology investment decisions.Ph.D

2011 Conference Abstracts: Annual Undergraduate Research Conference at the Interface of Biology and Mathematics

Abstract book for the Third Annual Undergraduate Research Conference at the Interface of Biology and Mathematics Date: October 21-22, 2011Plenary speaker: J. Carl Panetta, Department of Pharmaceutical Sciences, St. Jude Children\u27s Research HospitalFeatured Speaker: John Jungck, Mead Chair of the Sciences and Professor of Biology, Beloit Colleg

Making friends on the fly : advances in ad hoc teamwork

textGiven the continuing improvements in design and manufacturing processes in addition to improvements in artificial intelligence, robots are being deployed in an increasing variety of environments for longer periods of time. As the number of robots grows, it is expected that they will encounter and interact with other robots. Additionally, the number of companies and research laboratories producing these robots is increasing, leading to the situation where these robots may not share a common communication or coordination protocol. While standards for coordination and communication may be created, we expect that any standards will lag behind the state-of-the-art protocols and robots will need to additionally reason intelligently about their teammates with limited information. This problem motivates the area of ad hoc teamwork in which an agent may potentially cooperate with a variety of teammates in order to achieve a shared goal. We argue that agents that effectively reason about ad hoc teamwork need to exhibit three capabilities: 1) robustness to teammate variety, 2) robustness to diverse tasks, and 3) fast adaptation. This thesis focuses on addressing all three of these challenges. In particular, this thesis introduces algorithms for quickly adapting to unknown teammates that enable agents to react to new teammates without extensive observations. The majority of existing multiagent algorithms focus on scenarios where all agents share coordination and communication protocols. While previous research on ad hoc teamwork considers some of these three challenges, this thesis introduces a new algorithm, PLASTIC, that is the first to address all three challenges in a single algorithm. PLASTIC adapts quickly to unknown teammates by reusing knowledge it learns about previous teammates and exploiting any expert knowledge available. Given this knowledge, PLASTIC selects which previous teammates are most similar to the current ones online and uses this information to adapt to their behaviors. This thesis introduces two instantiations of PLASTIC. The first is a model-based approach, PLASTIC-Model, that builds models of previous teammates' behaviors and plans online to determine the best course of action. The second uses a policy-based approach, PLASTIC-Policy, in which it learns policies for cooperating with past teammates and selects from among these policies online. Furthermore, we introduce a new transfer learning algorithm, TwoStageTransfer, that allows transferring knowledge from many past teammates while considering how similar each teammate is to the current ones. We theoretically analyze the computational tractability of PLASTIC-Model in a number of scenarios with unknown teammates. Additionally, we empirically evaluate PLASTIC in three domains that cover a spread of possible settings. Our evaluations show that PLASTIC can learn to communicate with unknown teammates using a limited set of messages, coordinate with externally-created teammates that do not reason about ad hoc teams, and act intelligently in domains with continuous states and actions. Furthermore, these evaluations show that TwoStageTransfer outperforms existing transfer learning algorithms and enables PLASTIC to adapt even better to new teammates. We also identify three dimensions that we argue best describe ad hoc teamwork scenarios. We hypothesize that these dimensions are useful for analyzing similarities among domains and determining which can be tackled by similar algorithms in addition to identifying avenues for future research. The work presented in this thesis represents an important step towards enabling agents to adapt to unknown teammates in the real world. PLASTIC significantly broadens the robustness of robots to their teammates and allows them to quickly adapt to new teammates by reusing previously learned knowledge.Computer Science

Revive, Restore, Revitalize: An Eco-economic Methodology for Maasai Mara

The Maasai Mara in Kenya, renowned for its biodiversity, is witnessing ecosystem degradation and species endangerment due to intensified human activities. Addressing this, we introduce a dynamic system harmonizing ecological and human priorities. Our agent-based model replicates the Maasai Mara savanna ecosystem, incorporating 71 animal species, 10 human classifications, and 2 natural resource types. The model employs the metabolic rate-mass relationship for animal energy dynamics, logistic curves for animal growth, individual interactions for food web simulation, and human intervention impacts. Algorithms like fitness proportional selection and particle swarm mimic organism preferences for resources. To guide preservation activities, we formulated 21 management strategies encompassing tourism, transportation, taxation, environmental conservation, research, diplomacy, and poaching, employing a game-theoretic framework. Using the TOPSIS method, we prioritized four key developmental indicators: environmental health, research advancement, economic growth, and security. The interplay of 16 factors determines these indicators, each influenced by our policies to varying degrees. By evaluating the policies' repercussions, we aim to mitigate adverse animal-human interactions and equitably address human concerns. We classified the policy impacts into three categories: Environmental Preservation, Economic Prosperity, and Holistic Development. By applying these policy groupings to our ecosystem model, we tracked the effects on the intricate animal-human-resource dynamics. Utilizing the entropy weight method, we assessed the efficacy of these policy clusters over a decade, identifying the optimal blend emphasizing both environmental conservation and economic progression.Comment: 25 pages, 16 figure