Evolution of Swarm Robotics Systems with Novelty Search

Novelty search is a recent artificial evolution technique that challenges traditional evolutionary approaches. In novelty search, solutions are rewarded based on their novelty, rather than their quality with respect to a predefined objective. The lack of a predefined objective precludes premature convergence caused by a deceptive fitness function. In this paper, we apply novelty search combined with NEAT to the evolution of neural controllers for homogeneous swarms of robots. Our empirical study is conducted in simulation, and we use a common swarm robotics task - aggregation, and a more challenging task - sharing of an energy recharging station. Our results show that novelty search is unaffected by deception, is notably effective in bootstrapping the evolution, can find solutions with lower complexity than fitness-based evolution, and can find a broad diversity of solutions for the same task. Even in non-deceptive setups, novelty search achieves solution qualities similar to those obtained in traditional fitness-based evolution. Our study also encompasses variants of novelty search that work in concert with fitness-based evolution to combine the exploratory character of novelty search with the exploitatory character of objective-based evolution. We show that these variants can further improve the performance of novelty search. Overall, our study shows that novelty search is a promising alternative for the evolution of controllers for robotic swarms.Comment: To appear in Swarm Intelligence (2013), ANTS Special Issue. The final publication will be available at link.springer.co

Novelty Search in Competitive Coevolution

One of the main motivations for the use of competitive coevolution systems is their ability to capitalise on arms races between competing species to evolve increasingly sophisticated solutions. Such arms races can, however, be hard to sustain, and it has been shown that the competing species often converge prematurely to certain classes of behaviours. In this paper, we investigate if and how novelty search, an evolutionary technique driven by behavioural novelty, can overcome convergence in coevolution. We propose three methods for applying novelty search to coevolutionary systems with two species: (i) score both populations according to behavioural novelty; (ii) score one population according to novelty, and the other according to fitness; and (iii) score both populations with a combination of novelty and fitness. We evaluate the methods in a predator-prey pursuit task. Our results show that novelty-based approaches can evolve a significantly more diverse set of solutions, when compared to traditional fitness-based coevolution.Comment: To appear in 13th International Conference on Parallel Problem Solving from Nature (PPSN 2014

Comparing and Combining Lexicase Selection and Novelty Search

Lexicase selection and novelty search, two parent selection methods used in evolutionary computation, emphasize exploring widely in the search space more than traditional methods such as tournament selection. However, lexicase selection is not explicitly driven to select for novelty in the population, and novelty search suffers from lack of direction toward a goal, especially in unconstrained, highly-dimensional spaces. We combine the strengths of lexicase selection and novelty search by creating a novelty score for each test case, and adding those novelty scores to the normal error values used in lexicase selection. We use this new novelty-lexicase selection to solve automatic program synthesis problems, and find it significantly outperforms both novelty search and lexicase selection. Additionally, we find that novelty search has very little success in the problem domain of program synthesis. We explore the effects of each of these methods on population diversity and long-term problem solving performance, and give evidence to support the hypothesis that novelty-lexicase selection resists converging to local optima better than lexicase selection

Enhanced Optimization with Composite Objectives and Novelty Selection

An important benefit of multi-objective search is that it maintains a diverse population of candidates, which helps in deceptive problems in particular. Not all diversity is useful, however: candidates that optimize only one objective while ignoring others are rarely helpful. This paper proposes a solution: The original objectives are replaced by their linear combinations, thus focusing the search on the most useful tradeoffs between objectives. To compensate for the loss of diversity, this transformation is accompanied by a selection mechanism that favors novelty. In the highly deceptive problem of discovering minimal sorting networks, this approach finds better solutions, and finds them faster and more consistently than standard methods. It is therefore a promising approach to solving deceptive problems through multi-objective optimization.Comment: 7 page

Diseño de metaheurísticas paralelas con el paradigma novelty search para la reducción de incertidumbre en la predicción de fenómenos de propagación

Los incendios forestales son un fenómeno ambiental multicausal de gran prevalencia. El impacto de este fenómeno incluye pérdidas humanas, daños ambientales y económicos. Para mitigar estos daños, existen sistemas de simulación computacionales que predicen el comportamiento del fuego en base a un conjunto de parámetros de entrada o escenario (velocidad, dirección del viento; temperatura; etc.). Sin embargo, los resultados de una simulación suelen tener un alto grado de error por la incertidumbre en los valores de algunas variables, por no ser conocidos o porque su medición puede ser imprecisa o errónea. Por este motivo se han desarrollado métodos que combinan resultados de un conjunto de simulaciones sobre distintos escenarios, para detectar tendencias y así reducir dicha incertidumbre. Dos propuestas recientes, ESSIM-EA y ESSIM-DE, utilizan algoritmos evolutivos paralelos para orientar el espacio de escenarios a considerar, logrando mejoras en la calidad predictiva. Estos enfoques están guiados por una función objetivo que recompensa el avance hacia una solución. En problemas complejos, dicha función objetivo no siempre es un indicador directo de la calidad de las soluciones. En trabajos previos se han encontrado limitaciones como convergencia prematura, y se han requerido acciones de calibración y sintonización para incorporar soluciones más diversas al proceso de predicción. Para superar estas limitaciones, en este trabajo proponemos aplicar el paradigma Novelty Search (búsqueda basada en novedad), que reemplaza la función objetivo por una medida de la novedad de las soluciones encontradas, para generar continuamente soluciones con comportamientos diferentes entre sí. Este enfoque logra evitar óptimos locales y permitiría encontrar soluciones útiles que serían difíciles de hallar por otros algoritmos. Al igual que los métodos existentes, esta propuesta también puede aplicarse a otros modelos de propagación (inundaciones, avalanchas o corrimientos de suelo).Red de Universidades con Carreras en Informátic

Neuro-evolution search methodologies for collective self-driving vehicles

Recently there has been an increasing amount of research into autonomous vehicles for real-world driving. Much progress has been made in the past decade with many automotive manufacturers demonstrating real-world prototypes. Current predictions indicate that roads designed exclusively for autonomous vehicles will be constructed and thus this thesis explores the use of methods to automatically produce controllers for autonomous vehicles that must navigate with each other on these roads. Neuro-Evolution, a method that combines evolutionary algorithms with neural networks, has shown to be effective in reinforcement-learning, multi-agent tasks such as maze navigation, biped locomotion, autonomous racing vehicles and fin-less rocket control. Hence, a neuro-evolution method is selected and investigated for the controller evolution of collective autonomous vehicles in homogeneous teams. The impact of objective and non-objective search (and a combination of both, a hybrid method) for controller evolution is comparatively evaluated for robustness on a range of driving tasks and collection sizes. Results indicate that the objective search was able to generalise the best on unseen task environments compared to all other methods and the hybrid approach was able to yield desired task performance on evolution far earlier than both approaches but was unable to generalise as effectively over new environments

Devising effective novelty search algorithms: A comprehensive empirical study

Novelty search is a state-of-the-art evolutionary approach that promotes behavioural novelty instead of pursuing a static objective. Along with a large number of successful applications, many different variants of novelty search have been proposed. It is still unclear, however, how some key parameters and algorithmic components influence the evolutionary dynamics and performance of novelty search. In this paper, we conduct a comprehensive empirical study focused on novelty search’s algorithmic components. We study the k parameter — the number of nearest neighbours used in the computation of novelty scores; the use and function of an archive; how to combine novelty search with fitness-based evolution; and how to configure the mutation rate of the underlying evolutionary algorithm. Our study is conducted in a simulated maze navigation task. Our results show that the configuration of novelty search can have a significant impact on performance and behaviour space exploration. We conclude with a number of guidelines for the implementation and configuration of novelty search, which should help future practitioners to apply novelty search more effectively.info:eu-repo/semantics/acceptedVersio

Novelty search employed into the development of cancer treatment simulations

Conventional optimization methodologies may be hindered when the automated search is stuck into local optima because of a deceptive objective function landscape. Consequently, open ended search methodologies, such as novelty search, have been proposed to tackle this issue. Overlooking the objective, while putting pressure into discovering novel solutions may lead to better solutions in practical problems. Novelty search was employed here to optimize the simulated design of a targeted drug delivery system for tumor treatment under the PhysiCell simulator. A hybrid objective equation was used containing both the actual objective of an effective tumor treatment and the novelty measure of the possible solutions. Different weights of the two components of the hybrid equation were investigated to unveil the significance of each one