Natural Selection, Adaptive Evolution and Diversity in Computational Ecosystems

The central goal of this thesis is to provide additional criteria towards implementing open-ended evolution in an artificial system. Methods inspired by biological evolution are frequently applied to generate autonomous agents too complex to design by hand. Despite substantial progress in the area of evolutionary computation, additional efforts are needed to identify a coherent set of requirements for a system capable of exhibiting open-ended evolutionary dynamics. The thesis provides an extensive discussion of existing models and of the major considerations for designing a computational model of evolution by natural selection. Thus, the work in this thesis constitutes a further step towards determining the requirements for such a system and introduces a concrete implementation of an artificial evolution system to evaluate the developed suggestions. The proposed system improves upon existing models with respect to easy interpretability of agent behaviour, high structural freedom, and a low-level sensor and effector model to allow numerous long-term evolutionary gradients. In a series of experiments, the evolutionary dynamics of the system are examined against the set objectives and, where appropriate, compared with existing systems. Typical agent behaviours are introduced to convey a general overview of the system dynamics. These behaviours are related to properties of the respective agent populations and their evolved morphologies. It is shown that an intuitive classification of observed behaviours coincides with a more formal classification based on morphology. The evolutionary dynamics of the system are evaluated and shown to be unbounded according to the classification provided by Bedau and Packard’s measures of evolutionary activity. Further, it is analysed how observed behavioural complexity relates to the complexity of the agent-side mechanisms subserving these behaviours. It is shown that for the concrete definition of complexity applied, the average complexity continually increases for extended periods of evolutionary time. In combination, these two findings show how the observed behaviours are the result of an ongoing and lasting adaptive evolutionary process as opposed to being artifacts of the seeding process. Finally, the effect of variation in the system on the diversity of evolved behaviour is investigated. It is shown that coupling individual survival and reproductive success can restrict the available evolutionary trajectories in more than the trivial sense of removing another dimension, and conversely, decoupling individual survival from reproductive success can increase the number of evolutionary trajectories. The effect of different reproductive mechanisms is contrasted with that of variation in environmental conditions. The diversity of evolved strategies turns out to be sensitive to the reproductive mechanism while being remarkably robust to the variation of environmental conditions. These findings emphasize the importance of being explicit about the abstractions and assumptions underlying an artificial evolution system, particularly if the system is intended to model aspects of biological evolution

An approach to evolve and exploit repertoires of general robot behaviours

Recent works in evolutionary robotics have shown the viability of evolution driven by behavioural novelty and diversity. These evolutionary approaches have been successfully used to generate repertoires of diverse and high-quality behaviours, instead of driving evolution towards a single, task-specific solution. Having repertoires of behaviours can enable new forms of robotic control, in which high-level controllers continually decide which behaviour to execute. To date, however, only the use of repertoires of open-loop locomotion primitives has been studied. We propose EvoRBC-II, an approach that enables the evolution of repertoires composed of general closed-loop behaviours, that can respond to the robot's sensory inputs. The evolved repertoire is then used as a basis to evolve a transparent higher-level controller that decides when and which behaviours of the repertoire to execute. Relying on experiments in a simulated domain, we show that the evolved repertoires are composed of highly diverse and useful behaviours. The same repertoire contains sufficiently diverse behaviours to solve a wide range of tasks, and the EvoRBC-II approach can yield a performance that is comparable to the standard tabula-rasa evolution. EvoRBC-II enables automatic generation of hierarchical control through a two-step evolutionary process, thus opening doors for the further exploration of the advantages that can be brought by hierarchical control.info:eu-repo/semantics/acceptedVersio

Evolving a Behavioral Repertoire for a Walking Robot

Numerous algorithms have been proposed to allow legged robots to learn to walk. However, the vast majority of these algorithms is devised to learn to walk in a straight line, which is not sufficient to accomplish any real-world mission. Here we introduce the Transferability-based Behavioral Repertoire Evolution algorithm (TBR-Evolution), a novel evolutionary algorithm that simultaneously discovers several hundreds of simple walking controllers, one for each possible direction. By taking advantage of solutions that are usually discarded by evolutionary processes, TBR-Evolution is substantially faster than independently evolving each controller. Our technique relies on two methods: (1) novelty search with local competition, which searches for both high-performing and diverse solutions, and (2) the transferability approach, which com-bines simulations and real tests to evolve controllers for a physical robot. We evaluate this new technique on a hexapod robot. Results show that with only a few dozen short experiments performed on the robot, the algorithm learns a repertoire of con-trollers that allows the robot to reach every point in its reachable space. Overall, TBR-Evolution opens a new kind of learning algorithm that simultaneously optimizes all the achievable behaviors of a robot.Comment: 33 pages; Evolutionary Computation Journal 201

A novel plasticity rule can explain the development of sensorimotor intelligence

Grounding autonomous behavior in the nervous system is a fundamental challenge for neuroscience. In particular, the self-organized behavioral development provides more questions than answers. Are there special functional units for curiosity, motivation, and creativity? This paper argues that these features can be grounded in synaptic plasticity itself, without requiring any higher level constructs. We propose differential extrinsic plasticity (DEP) as a new synaptic rule for self-learning systems and apply it to a number of complex robotic systems as a test case. Without specifying any purpose or goal, seemingly purposeful and adaptive behavior is developed, displaying a certain level of sensorimotor intelligence. These surprising results require no system specific modifications of the DEP rule but arise rather from the underlying mechanism of spontaneous symmetry breaking due to the tight brain-body-environment coupling. The new synaptic rule is biologically plausible and it would be an interesting target for a neurobiolocal investigation. We also argue that this neuronal mechanism may have been a catalyst in natural evolution.Comment: 18 pages, 5 figures, 7 video

Environmental Influence on the Evolution of Morphological Complexity in Machines

Whether, when, how, and why increased complexity evolves in biological populations is a longstanding open question. In this work we combine a recently developed method for evolving virtual organisms with an information-theoretic metric of morphological complexity in order to investigate how the complexity of morphologies, which are evolved for locomotion, varies across different environments. We first demonstrate that selection for locomotion results in the evolution of organisms with morphologies that increase in complexity over evolutionary time beyond what would be expected due to random chance. This provides evidence that the increase in complexity observed is a result of a driven rather than a passive trend. In subsequent experiments we demonstrate that morphologies having greater complexity evolve in complex environments, when compared to a simple environment when a cost of complexity is imposed. This suggests that in some niches, evolution may act to complexify the body plans of organisms while in other niches selection favors simpler body plans

Evolving robots: from simple behaviours to complete systems

Building robots is generally considered difficult, because the designer not only has to predict the interaction between the robot and the environment, but also has to deal with the ensuing problems. This thesis examines the use of the evolutionary approach in designing robots; the explorations range from evolving simple behaviours for real robots, to complex behaviours (also for real robots), and finally to complete robot systems — including controllers and body plans. A framework is presented for evolving robot control systems. It includes two components: a task independent Genetic Programming sub-system and a task dependent controller evaluation sub-system. The performance evaluation of each robot controller is done in a simulator to reduce the evaluation time, and then the evolved controllers are downloaded to a real robot for performance verification. In addition, a special rep¬ resentation is designed for the reactive robot controller. It is succinct and can capture the important characteristics of a reactive control system, so that the evolutionary system can efficiently evolve the controllers of the desired behaviours for the robots. The framework has been successfully used to evolve controllers for real robots to achieve a variety of simple tasks, such as obstacle avoidance, safe exploration and box-pushing. A methodology is then proposed to scale up the system to evolve controllers for more complicated tasks. It involves adopting the architecture of a behaviour-based system, and evolving separate behaviour controllers and arbitrators for coordination. This allows robot controllers for more complex skills to be constructed in an incremental manner. Therefore the whole control system becomes easy to evolve; moreover, the resulting control system can be explicitly distributed, understandable to the system designer, and easy to maintain. The methodology has been used to evolve control systems for more complex tasks with good results. Finally, the evolutionary mechanism of the framework described above is extended to include a Genetic Algorithm sub-system for the co-evolution of robot body plans — structuralparametersofphysicalrobotsencodedaslinearstringsofrealnumbers. An individual in the extended system thus consists of a brain(controller) and a body. Whenever the individual is evaluated, the controller is executed on the corresponding body for a period of time to measure the performance. In such a system the Genetic Programming part evolves the controller; and the Genetic Algorithm part, the robot body. The results show that the complete robot system can be evolved in this manner. i

The Evolution of Complexity in Autonomous Robots

Evolutionary robotics–the use of evolutionary algorithms to automate the production of autonomous robots–has been an active area of research for two decades. However, previous work in this domain has been limited by the simplicity of the evolved robots and the task environments within which they are able to succeed. This dissertation aims to address these challenges by developing techniques for evolving more complex robots. Particular focus is given to methods which evolve not only the control policies of manually-designed robots, but instead evolve both the control policy and physical form of the robot. These techniques are presented along with their application to investigating previously unexplored relationships between the complexity of evolving robots and the task environments within which they evolve

Guest Editorial Active Learning and Intrinsically Motivated Exploration in Robots: Advances and Challenges

International audienceLEARNING techniques are increasingly being used in today's complex robotic systems. Robots are expected to deal with a large variety of tasks using their high-dimensional and complex bodies, to manipulate objects and also, to interact with humans in an intuitive and friendly way. In this new setting, not all relevant information is available at design time, and robots should typically be able to learn, through self-ex- perimentation or through human–robot interaction, how to tune their innate perceptual-motor skills or to learn, cumulatively, novel skills that were not preprogrammed initially. In a word, robots need to have the capacity to develop in an open-ended manner and in an open-ended environment, in a way that is analogous to human development which combines genetic and epigenetic factors. This challenge is at the center of the developmental robotics field. Among the various technical challenges that are raised by these issues, exploration is paramount. Self-experimentation and learning by interacting with the physical and social world is essential to acquire new knowledge and skills