Novelty-assisted Interactive Evolution Of Control Behaviors

The field of evolutionary computation is inspired by the achievements of natural evolution, in which there is no final objective. Yet the pursuit of objectives is ubiquitous in simulated evolution because evolutionary algorithms that can consistently achieve established benchmarks are lauded as successful, thus reinforcing this paradigm. A significant problem is that such objective approaches assume that intermediate stepping stones will increasingly resemble the final objective when in fact they often do not. The consequence is that while solutions may exist, searching for such objectives may not discover them. This problem with objectives is demonstrated through an experiment in this dissertation that compares how images discovered serendipitously during interactive evolution in an online system called Picbreeder cannot be rediscovered when they become the final objective of the very same algorithm that originally evolved them. This negative result demonstrates that pursuing an objective limits evolution by selecting offspring only based on the final objective. Furthermore, even when high fitness is achieved, the experimental results suggest that the resulting solutions are typically brittle, piecewise representations that only perform well by exploiting idiosyncratic features in the target. In response to this problem, the dissertation next highlights the importance of leveraging human insight during search as an alternative to articulating explicit objectives. In particular, a new approach called novelty-assisted interactive evolutionary computation (NA-IEC) combines human intuition with a method called novelty search for the first time to facilitate the serendipitous discovery of agent behaviors. iii In this approach, the human user directs evolution by selecting what is interesting from the on-screen population of behaviors. However, unlike in typical IEC, the user can then request that the next generation be filled with novel descendants, as opposed to only the direct descendants of typical IEC. The result of such an approach, unconstrained by a priori objectives, is that it traverses key stepping stones that ultimately accumulate meaningful domain knowledge. To establishes this new evolutionary approach based on the serendipitous discovery of key stepping stones during evolution, this dissertation consists of four key contributions: (1) The first contribution establishes the deleterious effects of a priori objectives on evolution. The second (2) introduces the NA-IEC approach as an alternative to traditional objective-based approaches. The third (3) is a proof-of-concept that demonstrates how combining human insight with novelty search finds solutions significantly faster and at lower genomic complexities than fully-automated processes, including pure novelty search, suggesting an important role for human users in the search for solutions. Finally, (4) the NA-IEC approach is applied in a challenge domain wherein leveraging human intuition and domain knowledge accelerates the evolution of solutions for the nontrivial octopus-arm control task. The culmination of these contributions demonstrates the importance of incorporating human insights into simulated evolution as a means to discovering better solutions more rapidly than traditional approaches

Making hurricane track data accessible

Our interactive tool allows the exploration, validation and presentation of hundreds of years of dynamically simulated storm tracks. The tracks were generated as part of a research project to improve the risk assessment of tropical storm damage by the insurance industry. The main impact of the tool is that exploratory interactive visualisation is now being used by the storm track modellers to (a) validate and improve model outputs, (b) discuss outputs with their peers (c) obtain a better understanding of the formation and development of tropical storms and (d) present examples of the behaviour of storms under different conditions to the insurance industry and others. Insights into tropical storm behaviour have been obtained and these insights are being articulated

The relative roles of CO2 and palaeogeography in determining Late Miocene climate: results from a terrestrial model-data comparison

The Late Miocene (∼11.6–5.3 Ma) palaeorecord provides evidence for a warmer and wetter climate than that of today and there is uncertainty in the palaeo-CO2 record of at least 150 ppmv. We present results from fully coupled atmosphere-ocean-vegetation simulations for the Late Miocene that examine the relative roles of palaeogeography (topography and ice sheet geometry) and CO2 concentration in the determination of Late Miocene climate through comprehensive terrestrial model-data comparisons. Assuming that the data accurately reflects the Late Miocene climate, and that the Late Miocene palaeogeographic reconstruction used in the model is robust, then results indicate that the proxy-derived precipitation differences between the Late Miocene and modern can be largely accounted for by the palaeogeographic changes alone. However, the proxy-derived temperatures differences between the Late Miocene and modern can only begin to be accounted for if we assume a palaeo-CO2 concentration towards the higher end of the range of estimates

On singular values decomposition and patterns for human motion analysis and simulation

We are interested in human motion characterization and automatic motion simulation. The apparent redun- dancy of the humanoid w.r.t its explicit tasks lead to the problem of choosing a plausible movement in the framework of redun- dant kinematics. This work explores the intrinsic relationships between singular value decomposition at kinematic level and optimization principles at task level and joint level. Two task- based schemes devoted to simulation of human motion are then proposed and analyzed. These results are illustrated by motion captures, analyses and task-based simulations. Pattern of singular values serve as a basis for a discussion concerning the similarity of simulated and real motions

The effect of network topology on optimal exploration strategies and the evolution of cooperation in a mobile population

We model a mobile population interacting over an underlying spatial structure using a Markov movement model. Interactions take the form of public goods games, and can feature an arbitrary group size. Individuals choose strategically to remain at their current location or to move to a neighbouring location, depending upon their exploration strategy and the current composition of their group. This builds upon previous work where the underlying structure was a complete graph (i.e. there was effectively no structure). Here, we consider alternative network structures and a wider variety of, mainly larger, populations. Previously, we had found when cooperation could evolve, depending upon the values of a range of population parameters. In our current work, we see that the complete graph considered before promotes stability, with populations of cooperators or defectors being relatively hard to replace. By contrast, the star graph promotes instability, and often neither type of population can resist replacement. We discuss potential reasons for this in terms of network topology

Variable time scales, agent-based models, and role-playing games: The PIEPLUE river basin management game

This article presents a specific association of a role-playing game (RPG) and an agent-based model (ABM) aimed at dealing with a large range of time scales. Applications to the field of natural resource management lead one to consider the short time scale of resource use in practice at the same time as the longer ones related to resource dynamics or actors' investments. In their daily practice, stakeholders are translating their long-term strategies, a translation that is contextualized and combined with some cooccurring events. Long-term thinking is required for sustainable use of natural resources, but it should take into account its necessary adaptation on a short time scale. This raises the necessity for tools able to tackle jointly these various time scales. The similarity of architecture between computerized ABMs and RPGs makes them easy to associate in a hybrid tool, targeted at meeting this requirement. The proposition of this article is to allocate the representation of short time scales to computerized ABMs and the long ones to RPGs, while keeping the same static structural conceptual model, shared as a common root by both. This synergy is illustrated with PIEPLUE, an interactive setting tackling water-sharing issues.GESTION DE L'EAU;BASSIN VERSANT;RESSOURCE NATURELLE;MODELE;JEU DE ROLE;SYSTEME MULTIAGENTS;AGENT-BASED MODEL;CASE STUDY OF WATER SHARING;CONCEPTUAL MODEL;HYBRID TOOL;INVESTMENTS;LONG-TERM ISSUES;NATURAL RESOURCE MANAGEMENT;PIEPLUE;PARTICIPATORY MODELING;RESOURCE DYNAMICS;RESOURCE USE;ROLE-PLAYING GAME;STAKEHOLDERS;SUSTAINABLE USE OF NATURAL RESOURCES;TIME-SCALE DIVERSITY;VARIABLE TIME SCALES;WATER MANAGEMENT

Modelling the Eddy Pattern in the harbour of Zeebrugge

Hydrodynamic

Two brains in action: joint-action coding in the primate frontal cortex

Daily life often requires the coordination of our actions with those of another partner. After sixty years (1968-2018) of behavioral neurophysiology of motor control, the neural mechanisms which allow such coordination in primates are unknown. We studied this issue by recording cell activity simultaneously from dorsal premotor cortex (PMd) of two male interacting monkeys trained to coordinate their hand forces to achieve a common goal. We found a population of 'joint-action cells' that discharged preferentially when monkeys cooperated in the task. This modulation was predictive in nature, since in most cells neural activity led in time the changes of the "own" and of the "other" behavior. These neurons encoded the joint-performance more accurately than 'canonical action-related cells', activated by the action per se, regardless of the individual vs. interactive context. A decoding of joint-action was obtained by combining the two brains activities, using cells with directional properties distinguished from those associated to the 'solo' behaviors. Action observation-related activity studied when one monkey observed the consequences of the partner's behavior, i.e. the cursor's motion on the screen, did not sharpen the accuracy of 'joint-action cells' representation, suggesting that it plays no major role in encoding joint-action. When monkeys performed with a non-interactive partner, such as a computer, 'joint-action cells' representation of the "other" (non-cooperative) behavior was significantly degraded. These findings provide evidence of how premotor neurons integrate the time-varying representation of the self-action with that of a co-actor, thus offering a neural substrate for successful visuo-motor coordination between individuals.SIGNIFICANT STATEMENTThe neural bases of inter-subject motor coordination were studied by recording cell activity simultaneously from the frontal cortex of two interacting monkeys, trained to coordinate their hand forces to achieve a common goal. We found a new class of cells, preferentially active when the monkeys cooperated, rather than when the same action was performed individually. These 'joint-action neurons' offered a neural representation of joint-behaviors by far more accurate than that provided by the canonical action-related cells, modulated by the action per se regardless of the individual/interactive context. A neural representation of joint-performance was obtained by combining the activity recorded from the two brains. Our findings offer the first evidence concerning neural mechanisms subtending interactive visuo-motor coordination between co-acting agents