Sustainable Cooperative Coevolution with a Multi-Armed Bandit

This paper proposes a self-adaptation mechanism to manage the resources allocated to the different species comprising a cooperative coevolutionary algorithm. The proposed approach relies on a dynamic extension to the well-known multi-armed bandit framework. At each iteration, the dynamic multi-armed bandit makes a decision on which species to evolve for a generation, using the history of progress made by the different species to guide the decisions. We show experimentally, on a benchmark and a real-world problem, that evolving the different populations at different paces allows not only to identify solutions more rapidly, but also improves the capacity of cooperative coevolution to solve more complex problems.Comment: Accepted at GECCO 201

Bandit-based cooperative coevolution for tackling contribution imbalance in large-scale optimization problems

This paper addresses the issue of computational resource allocation within the context of cooperative coevolution. Cooperative coevolution typically works by breaking a problem down into smaller subproblems (or components) and coevolving them in a round-robin fashion, resulting in a uniform resource allocation among its components. Despite its success on a wide range of problems, cooperative coevolution struggles to perform efficiently when its components do not contribute equally to the overall objective value. This is of crucial importance on large-scale optimization problems where such difference are further magnified. To resolve this imbalance problem, we extend the standard cooperative coevolution to a new generic framework capable of learning the contribution of each component using multi-armed bandit techniques. The new framework allocates the computational resources to each component proportional to their contributions towards improving the overall objective value. This approach results in a more economical use of the limited computational resources. We study different aspects of the proposed framework in the light of extensive experiments. Our empirical results confirm that even a simple bandit-based credit assignment scheme can significantly improve the performance of cooperative coevolution on large-scale continuous problems, leading to competitive performance as compared to the state-of-the-art algorithms

Cooperative Coevolution for Non-Separable Large-Scale Black-Box Optimization: Convergence Analyses and Distributed Accelerations

Given the ubiquity of non-separable optimization problems in real worlds, in this paper we analyze and extend the large-scale version of the well-known cooperative coevolution (CC), a divide-and-conquer optimization framework, on non-separable functions. First, we reveal empirical reasons of why decomposition-based methods are preferred or not in practice on some non-separable large-scale problems, which have not been clearly pointed out in many previous CC papers. Then, we formalize CC to a continuous game model via simplification, but without losing its essential property. Different from previous evolutionary game theory for CC, our new model provides a much simpler but useful viewpoint to analyze its convergence, since only the pure Nash equilibrium concept is needed and more general fitness landscapes can be explicitly considered. Based on convergence analyses, we propose a hierarchical decomposition strategy for better generalization, as for any decomposition there is a risk of getting trapped into a suboptimal Nash equilibrium. Finally, we use powerful distributed computing to accelerate it under the multi-level learning framework, which combines the fine-tuning ability from decomposition with the invariance property of CMA-ES. Experiments on a set of high-dimensional functions validate both its search performance and scalability (w.r.t. CPU cores) on a clustering computing platform with 400 CPU cores

A review of population-based metaheuristics for large-scale black-box global optimization: Part B

This paper is the second part of a two-part survey series on large-scale global optimization. The first part covered two major algorithmic approaches to large-scale optimization, namely decomposition methods and hybridization methods such as memetic algorithms and local search. In this part we focus on sampling and variation operators, approximation and surrogate modeling, initialization methods, and parallelization. We also cover a range of problem areas in relation to large-scale global optimization, such as multi-objective optimization, constraint handling, overlapping components, the component imbalance issue, and benchmarks, and applications. The paper also includes a discussion on pitfalls and challenges of current research and identifies several potential areas of future research

Towards a more efficient use of computational budget in large-scale black-box optimization

Evolutionary algorithms are general purpose optimizers that have been shown effective in solving a variety of challenging optimization problems. In contrast to mathematical programming models, evolutionary algorithms do not require derivative information and are still effective when the algebraic formula of the given problem is unavailable. Nevertheless, the rapid advances in science and technology have witnessed the emergence of more complex optimization problems than ever, which pose significant challenges to traditional optimization methods. The dimensionality of the search space of an optimization problem when the available computational budget is limited is one of the main contributors to its difficulty and complexity. This so-called curse of dimensionality can significantly affect the efficiency and effectiveness of optimization methods including evolutionary algorithms. This research aims to study two topics related to a more efficient use of computational budget in evolutionary algorithms when solving large-scale black-box optimization problems. More specifically, we study the role of population initializers in saving the computational resource, and computational budget allocation in cooperative coevolutionary algorithms. Consequently, this dissertation consists of two major parts, each of which relates to one of these research directions. In the first part, we review several population initialization techniques that have been used in evolutionary algorithms. Then, we categorize them from different perspectives. The contribution of each category to improving evolutionary algorithms in solving large-scale problems is measured. We also study the mutual effect of population size and initialization technique on the performance of evolutionary techniques when dealing with large-scale problems. Finally, assuming uniformity of initial population as a key contributor in saving a significant part of the computational budget, we investigate whether achieving a high-level of uniformity in high-dimensional spaces is feasible given the practical restriction in computational resources. In the second part of the thesis, we study the large-scale imbalanced problems. In many real world applications, a large problem may consist of subproblems with different degrees of difficulty and importance. In addition, the solution to each subproblem may contribute differently to the overall objective value of the final solution. When the computational budget is restricted, which is the case in many practical problems, investing the same portion of resources in optimizing each of these imbalanced subproblems is not the most efficient strategy. Therefore, we examine several ways to learn the contribution of each subproblem, and then, dynamically allocate the limited computational resources in solving each of them according to its contribution to the overall objective value of the final solution. To demonstrate the effectiveness of the proposed framework, we design a new set of 40 large-scale imbalanced problems and study the performance of some possible instances of the framework

Efficient Resource Allocation in Cooperative Co-Evolution for Large-Scale Global Optimization

Cooperative co-evolution (CC) is an explicit means of problem decomposition in multipopulation evolutionary algorithms for solving large-scale optimization problems. For CC, subpopulations representing subcomponents of a large-scale optimization problem co-evolve, and are likely to have different contributions to the improvement of the best overall solution to the problem. Hence, it makes sense that more computational resources should be allocated to the subpopulations with greater contributions. In this paper, we study how to allocate computational resources in this context and subsequently propose a new CC framework named CCFR to efficiently allocate computational resources among the subpopulations according to their dynamic contributions to the improvement of the objective value of the best overall solution. Our experimental results suggest that CCFR can make efficient use of computational resources and is a highly competitive CCFR for solving large-scale optimization problems

Placement interactif de capteurs mobiles dans des environnements tridimensionnels non convexes

La présente thèse propose un système complet de placement de capteurs mobiles dans un environnement pleinement tridimensionnel et préalablement inconnu. Les capteurs mobiles sont des capteurs placés sur des unités robotiques autonomes, soit des véhicules possédant une unité de calcul et pouvant se déplacer dans l’environnement. Le placement de capteur est fondé sur une vue désirée par un utilisateur du système nommé vue virtuelle. La vue virtuelle est contrôlée à distance en changeant les paramètres intrinsèques et extrinsèques du capteur virtuel, soit sa position, sa résolution, son champ de vue, etc. Le capteur virtuel n’est alors soumis à aucune contrainte physique, par exemple il peut être placé à n’importe quelle hauteur dans l’environnement et avoir un champ de vue et une résolution arbitrairement grande. Les capteurs mobiles (réels) ont pour tâche de récupérer toute l’information contenue dans le point de vue virtuel. Ce n’est qu’en combinant leur capacité sensorielle que les capteurs mobiles pourront capter l’information demandée par l’utilisateur. Tout d’abord, cette thèse s’attaque au problème de placement de capteurs en définissant une fonction de visibilité servant à évaluer le positionnement d’un groupe de capteurs dans l’environnement. La fonction de visibilité développée est applicable aux environnements tridimensionnels et se base sur le principe de ligne de vue directe entre un capteur et la cible. De plus, la fonction prend en compte la densité d’échantillonnage des capteurs afin de reproduire la densité désirée indiquée par le capteur virtuel. Ensuite, ce travail propose l’utilisation d’un modèle de l’environnement pleinement tridimensionnel et pouvant être construit de manière incrémentale, rendant son utilisation possible dans un environnement tridimensionnel non convexe préalablement inconnu. Puis, un algorithme d’optimisation coopératif est présenté afin de trouver simultanément le nombre de capteurs et leur positionnement respectif afin d’acquérir l’information contenue dans la vue virtuelle. Finalement, la thèse démontre expérimentalement dans diverses conditions que le système proposé est supérieur à l’état de l’art pour le placement de capteurs dans le but d’observer une scène bidimensionnelle. Il est aussi établi expérimentalement en simulation et en réalité que les performances se transposent à l’observation d’environnements tridimensionnels non convexes préalablement inconnus.This Thesis proposes a novel mobile sensor placement system working in initially unknown three dimensional environment. The mobile sensors are fix sensors placed on autonomous robots, which are ground and aerial vehicles equipped with computing units. The sensor placement is based on a user-defined view, named the virtual view. This view is manipulated through a virtual sensor intrinsic and extrinsic parameters, such as its position, orientation, field of view, resolution, etc. The virtual sensor is not subject to any physical constraint, for example it can be place where no sensor could be or it possess an arbitrary large field of view and resolution. The mobile (real) sensors have to acquire the entire information contained in this virtual view. It is only by combining the sensory capacity of an unknown number of sensors that they can acquire the necessary information. First, this Thesis addresses the sensor placement problem by defining a visibility function to qualify a group of sensor configurations in the environment. This function is applicable to three dimensional environments and is based on direct line of sight principle, where we compute the sensor sampling density in its visibility region. Then, this Thesis proposes the use of an incrementally built model of the environment containing all the information needed by the objective function. Next, a cooperative optimization algorithm is put forward to simultaneously find the number of sensors and their respective position required to capture all the information in the virtual view. Finally, the proposed system is experimentally shown to use less sensor to acquire the scene of interest at a higher resolution than state of the art methods in initially known two dimensional environments. It is also shown in simulation and practice that the performance of the system can be transposed to initially unknown non-convex three dimensional environments

Learning to coordinate in sparse asymmetric multiagent systems

Multiagent learning offers a rich framework to address challenging real-world problems such as remote exploration and healthcare coordination, which require autonomous agents to express elaborate interactions. To be effective in such systems, agents must collectively reason about and pursue high-level, long-term, and possibly nebulous objectives while adapting their strategy to changing environments, inter-agent relationships, and team dynamics. This work introduces six contributions that address this multifaceted problem through the lens of two distinct perspectives: reward structures for high-level objectives that allow agents to consider behaviors before pursuing them, and diversity structures that incentivize asymmetric agents (agents with distinct capabilities and egocentric objectives) to discover complementary specializations required for robust teamwork. The first contribution, Asymmetric D++, distills sparse team feedback into dense informative rewards by encouraging agents to create asymmetric counterfactuals based on their likelihood to cooperate. The second contribution introduces an uncertainty-aware reward approximation that enables the application of Asymmetric D++ for exploration and learning in sparse reward settings. The third contribution, Behavior Refinement, presents a hierarchical framework that shifts focus from optimizing a single behavior to learning a repertoire of diverse behaviors required to complete variegated tasks. Behavior Refinement allows systematic exploration of the policy space via a combination of diversity search and team-objective maximization. The fourth contribution introduces the Island Model, a computational framework that builds on Behavior Refinement for informed behavior space exploration and team balancing for asymmetric agents. The final two contributions, expand upon the Island Model to develop an asynchronous learning framework that allows asymmetric agents to explore diverse environment-agnostic inter-agent relationships to balance multiple potentially conflicting objectives. The amalgamation of this work facilitates asymmetric agents to learn diverse specializations, express complex trade-offs, and discover robust inter-agent relationships required to solve challenging coordination problems. Additionally, the techniques introduced in this work aid in investigating the rich tapestry of agent synergies that evolve in response to changes in the environment and team objectives.Keywords: Multiagent Reinforcement Learning, Multiagent Coordination, Asymmetric Multiagent Systems, Multiagent Evolutionary Learnin

Socio-hydrology from Local to Large Scales: An Agent-based Modeling Approach

For decades, the interaction between water and people has attracted hydrologists’ attention. However, the coevolution of social and natural processes, which occurs across a range of time scales, has not yet been adequately characterized. This research gap has motivated more research in recent years under the umbrella of “socio-hydrology”. The purpose of socio-hydrology is to posit the endogeneity of humans in a hydrological system and then to investigate feedback mechanisms between hydrological and human systems that might lead to emergent phenomena. The current state-of-the-art in socio-hydrology faces several challenges that include (1) a tenuous connection of socio-hydrology to broader research on social, economic, and policy aspects of water resources, (2) the (in)capability of socio-hydrological models to capture human behavior by generic feedback mechanisms that can be extrapolated to other places, and (3) unsatisfying calibration or validation processes in modeling. To address the first gap, a socio-hydrology study needs to connect proper social theories on water-related human decision making with a water resource model based on a given context and scale. Addressing the second gap calls for socio-hydrology research with case studies in different and contrasting regions and at different scales. In fact, such study can shed light on the similarities and differences in socio-hydrological systems in different contexts and scales as initial steps for future research. The third research gap calls for a socio-hydrology study that improves calibration and validation processes. Thus, to address all these gaps in one thesis, two case studies with completely different environments are chosen to investigate various phenomena at different scales. The research presented here contributes to socio-hydrological understanding at two spatial scales. To account for the heterogeneity of human decision making and its interactions with the hydrologic system, an agent-based modeling (ABM) approach is used in this research. The first objective is to explore human adaptation to drought as well as the subsequent expected or unexpected effects on the agricultural sector and to develop a socio-hydrological model to predict agricultural water demand. To do so, an agent-based agricultural water demand model (ABAD) is developed. This model is applied to the Bow River Basin in Alberta, Canada, as a study region, which has recently experienced drought periods. The second objective is to explore conflict-and-cooperation processes in transboundary rivers as socio-hydrological phenomena at a large scale. The Eastern Nile Basin Socio-hydrological (ENSH) model is developed and applied to the Eastern Nile Basin (ENB) in Africa in which conflict-and-cooperation dynamics can be seen among Egypt, Sudan, and Ethiopia. The ENSH model aims to quantify and simulate these countries’ willingness to cooperate in the ENB. ABAD demonstrates (1) how farmers’ attitudes toward profits, risk aversion, environmental protection, social interaction, and irrigation expansion explain the dynamics of the water demand and (2) how the conservation program may paradoxically lead to the rebound phenomenon whereby the water demand may increase after decreasing through modernized irrigation systems. Through the ABAD model analysis, economic factors are found to dominantly control possible rebounds. Based on the insights gained via the model analysis, it is discussed that several strategies, including community participation and water restrictions, can be adopted to avoid the rebound phenomenon in irrigation systems. Fostering farmers’ awareness about the average water use in their community could be a means to avoid the rebound phenomenon through community participation. Also, another strategy to avoid the rebound phenomenon could be to reassign water allocations to reduce farmers’ water rights. The ENSH model showed that (1) socio-political factors (i.e., relative political stability and foreign direct investment) can explain two historical trends (i.e., (a) fluctuations in Ethiopia’s willingness to cooperate between 1983 and 2009 and (b) a decreasing Ethiopia’s willingness to cooperate between 2009 and 2016); (2) the 2008 food crisis (i.e., Sudan’s food gap) may account for Sudan recovering its willingness to cooperate; and (3) Egypt’s political (in)stability plays a role in its willingness to cooperate. The outcomes of this research can provide valuable insights to support policymakers for the long-term sustainability of water planning. This research investigates two main socio-hydrological phenomena at different spatial scales: the agricultural rebound phenomenon at a small geographical scale and the conflict and cooperation phenomena at a large geographical scale. The emergence of these phenomena can be a complex resultant of interaction and feedback mechanisms between the social system at the individual, institutional, and society levels and the hydrological system. Through developing quantitative socio-hydrological models, this research investigates the feedback mechanisms that may lead to the rebound phenomenon at a small scale and the conflict and cooperation phenomenon at a large scale. Finally, the research shows how these socio-hydrological models can be used for sustainable water management to avoid negative long-term consequences

Evolutionary Computation 2020

Intelligent optimization is based on the mechanism of computational intelligence to refine a suitable feature model, design an effective optimization algorithm, and then to obtain an optimal or satisfactory solution to a complex problem. Intelligent algorithms are key tools to ensure global optimization quality, fast optimization efficiency and robust optimization performance. Intelligent optimization algorithms have been studied by many researchers, leading to improvements in the performance of algorithms such as the evolutionary algorithm, whale optimization algorithm, differential evolution algorithm, and particle swarm optimization. Studies in this arena have also resulted in breakthroughs in solving complex problems including the green shop scheduling problem, the severe nonlinear problem in one-dimensional geodesic electromagnetic inversion, error and bug finding problem in software, the 0-1 backpack problem, traveler problem, and logistics distribution center siting problem. The editors are confident that this book can open a new avenue for further improvement and discoveries in the area of intelligent algorithms. The book is a valuable resource for researchers interested in understanding the principles and design of intelligent algorithms