Colearning in Coevolutionary Algorithms

Kartézské genetické programování je druh genetického programování, ve kterém jsou kandidátní programy reprezentovány jako orientované acyklické grafy. Bylo ukázáno, že je možné evoluci kartézských programů urychlit použitím koevoluce, kde se ve druhé populaci vyvíjí prediktory fitness. Prediktory fitness slouží k přibližnému určení kvality kandidátních řešení. Nevýhodou koevolučního přístupu je nutnost provést mnoho časově náročných experimentů pro určení nejvýhodnější velikosti prediktoru pro daný problém. V této práci je představena nová reprezentace prediktorů fitness s plastickým fenotypem, založená na principech souběžného učení v evolučních algoritmech. Plasticita fenotypu umožňuje odvodit různé fenotypy ze stejného genotypu. Díky tomu je možné adaptovat velikost prediktoru na současný průběh evoluce a obtížnost řešeného problému. Navržený algoritmus byl implementován v jazyce C a optimalizován pomocí vektorových instrukcí SSE2 a AVX2. Z experimentů vyplývá, že použitím plastického fenotypu lze dosáhnout srovnatelné kvalitních obrazových filtrů jako u standardního CGP při kratší době běhu programu (průměrné zrychlení je 8,6násobné) a zároveň odpadá nutnost hledání nejvýhodnější velikosti prediktoru jako u koevoluce s prediktory s fixní velikostí.Cartesian genetic programming (CGP) is a form of genetic programming where candidate programs are represented in the form of directed acyclic graphs. It was shown that CGP can be accelerated using coevolution with a population of fitness predictors which are used to estimate the quality of candidate solutions. The major disadvantage of the coevolutionary approach is the necessity of performing many time-consuming experiments to determine the best size of the fitness predictor for the particular task. This project introduces a new fitness predictor representation with phenotype plasticity, based on the principles of colearning in evolutionary algorithms. Phenotype plasticity allows to derive various phenotypes from the same genotype. This allows to adapt the size of the predictors to the current state of the evolution and difficulty of the solved problem. The proposed algorithm was implemented in the C language and optimized using SSE2 and AVX2 vector instructions. The experimental results show that the resulting image filters are comparable with standard CGP in terms of filtering quality. The average speedup is 8.6 compared to standard CGP. The speed is comparable to standard coevolutionary CGP but it is not necessary to experimentally determine the best size of the fitness predictor while applying coevolution to a new, unknown task.

Sustainable Cooperative Coevolution with a Multi-Armed Bandit

This paper proposes a self-adaptation mechanism to manage the resources allocated to the different species comprising a cooperative coevolutionary algorithm. The proposed approach relies on a dynamic extension to the well-known multi-armed bandit framework. At each iteration, the dynamic multi-armed bandit makes a decision on which species to evolve for a generation, using the history of progress made by the different species to guide the decisions. We show experimentally, on a benchmark and a real-world problem, that evolving the different populations at different paces allows not only to identify solutions more rapidly, but also improves the capacity of cooperative coevolution to solve more complex problems.Comment: Accepted at GECCO 201

Coevolutionary Algorithms and Classification

Cílem této práce je automatizovaný návrh programu pro detekci projevů dyskineze z pohybových dat pacientů. K návrhu programu je využito kartézské genetické programování, které bylo z důvodu urychlení procesu návrhu doplněno o koevoluci prediktorů fitness s proměnlivou velikostí, která umožňuje vyhodnocení kvality kandidátních řešení na pouhé části trénovacích dat. Vzniklé řešení dosahuje srovnatelné schopnosti rozlišení mezi třídami (AUC) s existujícím řešením při dosažení v průměru trojnásobného zrychlení procesu návrhu oproti variantě bez prediktorů fitness. Experimenty s metodami křížení prediktorů neukázaly významný rozdíl mezi zvolenými metodami. Zajímavých výsledků však bylo dosaženo při experimentech s celočíselnými datovými typy vhodnými pro implementaci v hardwaru, kdy u datového typu o osmi bitech bez znaménka (uint8_t) bylo dosaženo nejenom srovnatelné schopnosti rozlišení mezi třídami (pro významné projevy dyskineze AUC = 0,93 shodně jako pro existující řešení) a zlepšení rozlišovací schopností u chodících pacientů (AUC = 0,80 oproti AUC = 0,73 u existujícího řešení), ale navíc v průměru téměř devítinásobného zrychlení návrhu oproti variantě bez prediktorů fitness využívající datový typ float.The aim of this work is to automatically design a program that is able to detect dyskinetic movement features in the measured patient's movement data. The program will be developed using Cartesian genetic programming equipped with coevolution of fitness predictors. This type of coevolution allows to speed up a design performed by Cartesian genetic programming by evaluating a quality of candidate solutions using only a part of training data. Evolved classifier achieves a performance (in terms of AUC) that is comparable with the existing solution while achieving threefold acceleration of the learning process compared to the variant without the fitness predictors, in average. Experiments with crossover methods for fitness predictors haven't shown a significant difference between investigated methods. However, interesting results were obtained while investigating integer data types that are more suitable for implementation in hardware. Using an unsigned eight-bit data type (uint8_t) we've achieved not only comparable classification performance (for significant dyskinesia AUC = 0.93 the same as for the existing solutions), with improved AUC for walking patient's data (AUC = 0.80, while existing solutions AUC = 0.73), but also nine times speedup of the design process compared to the approach without fitness predictors employing the float data type, in average.

Novelty-driven cooperative coevolution

Cooperative coevolutionary algorithms (CCEAs) rely on multiple coevolving populations for the evolution of solutions composed of coadapted components. CCEAs enable, for instance, the evolution of cooperative multiagent systems composed of heterogeneous agents, where each agent is modelled as a component of the solution. Previous works have, however, shown that CCEAs are biased toward stability: the evolutionary process tends to converge prematurely to stable states instead of (near-)optimal solutions. In this study, we show how novelty search can be used to avoid the counterproductive attraction to stable states in coevolution. Novelty search is an evolutionary technique that drives evolution toward behavioural novelty and diversity rather than exclusively pursuing a static objective. We evaluate three novelty-based approaches that rely on, respectively (1) the novelty of the team as a whole, (2) the novelty of the agents’ individual behaviour, and (3) the combination of the two. We compare the proposed approaches with traditional fitness-driven cooperative coevolution in three simulated multirobot tasks. Our results show that team-level novelty scoring is the most effective approach, significantly outperforming fitness-driven coevolution at multiple levels. Novelty-driven cooperative coevolution can substantially increase the potential of CCEAs while maintaining a computational complexity that scales well with the number of populations.info:eu-repo/semantics/publishedVersio

Approximating n-player behavioural strategy nash equilibria using coevolution

Coevolutionary algorithms are plagued with a set of problems related to intransitivity that make it questionable what the end product of a coevolutionary run can achieve. With the introduction of solution concepts into coevolution, part of the issue was alleviated, however efficiently representing and achieving game theoretic solution concepts is still not a trivial task. In this paper we propose a coevolutionary algorithm that approximates behavioural strategy Nash equilibria in n-player zero sum games, by exploiting the minimax solution concept. In order to support our case we provide a set of experiments in both games of known and unknown equilibria. In the case of known equilibria, we can confirm our algorithm converges to the known solution, while in the case of unknown equilibria we can see a steady progress towards Nash. Copyright 2011 ACM

Cooperative coevolution of morphologically heterogeneous robots

Morphologically heterogeneous multirobot teams have shown significant potential in many applications. While cooperative coevolutionary algorithms can be used for synthesising controllers for heterogeneous multirobot systems, they have been almost exclusively applied to morphologically homogeneous systems. In this paper, we investigate if and how cooperative coevolutionary algorithms can be used to evolve behavioural control for a morphologically heterogeneous multirobot system. Our experiments rely on a simulated task, where a ground robot with a simple sensor-actuator configuration must cooperate tightly with a more complex aerial robot to find and collect items in the environment. We first show how differences in the number and complexity of skills each robot has to learn can impair the effectiveness of cooperative coevolution. We then show how coevolution’s effectiveness can be improved using incremental evolution or novelty-driven coevolution. Despite its limitations, we show that coevolution is a viable approach for synthesising control for morphologically heterogeneous systems.info:eu-repo/semantics/publishedVersio

A visual demonstration of convergence properties of cooperative coevolution

We introduce a model for cooperative coevolutionary algorithms (CCEAs) using partial mixing, which allows us to compute the expected long-run convergence of such algorithms when individuals ’ fitness is based on the maximum payoff of some N evaluations with partners chosen at random from the other population. Using this model, we devise novel visualization mechanisms to attempt to qualitatively explain a difficult-to-conceptualize pathology in CCEAs: the tendency for them to converge to suboptimal Nash equilibria. We further demonstrate visually how increasing the size of N, or biasing the fitness to include an ideal-collaboration factor, both improve the likelihood of optimal convergence, and under which initial population configurations they are not much help

Leverage AI to Learn, Optimize, and Wargame (LAILOW) for Strategic Laydown and Dispersal (SLD) of the USN Operating Forces

NPS NRP Technical ReportThe SECNAV disperses Navy forces in a deliberate manner to support DoD guidance, policy and budget. The current SLD process is labor intensive, takes too long, and needs AI. The research questions are: - How does the Navy weight competing demands for naval forces between the CCMDs to determine an optimal dispersal of operating forces? - How does the Navy optimize force laydown to maximize force development (Fd) and force generation (Fg) efficiency? We propose LAILOW to address the questions. LAILOW was derived from the ONR funded project and focuses on deep analytics of machine learning, optimization, and wargame. Learn: When there are data, data mining, machine learning, and predictive algorithms are used to analyze data. Historical Phased Force Deployment Data (TPFDDs) and SLD Report Cards data among others, one can learn patterns of what decisions were made and how they are executed with in the past. Optimize: Patterns from learn are used to optimize future SLD plans. A SLD plan may include how many homeports, home bases, hubs, and shore posture locations (Fd) and staffs (Fg). The optimization can be overwhelming. LAILOW uses integrated Soar reinforcement learning (Soar-RL) and coevolutionary algorithms. Soar-RL maps a total SLD plan to individual ones used in excursion modeling and what if analysis. Wargame: There might be no or rare data for new warfighting requirements and capabilities. This motivates wargame simulations. A SLD plan can include state variables or problems (e.g., future global and theater posture, threat characteristics), which is only observed, sensed, and cannot be changed. Control variables are solutions (e.g., a SLD plan). LAILOW sets up a wargame between state and control variables. Problems and solutions coevolve based on evolutionary principles of selection, mutation, and crossover.N3/N5 - Plans & StrategyThis research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.