Cooperative Coevolution for Non-Separable Large-Scale Black-Box Optimization: Convergence Analyses and Distributed Accelerations

Given the ubiquity of non-separable optimization problems in real worlds, in this paper we analyze and extend the large-scale version of the well-known cooperative coevolution (CC), a divide-and-conquer optimization framework, on non-separable functions. First, we reveal empirical reasons of why decomposition-based methods are preferred or not in practice on some non-separable large-scale problems, which have not been clearly pointed out in many previous CC papers. Then, we formalize CC to a continuous game model via simplification, but without losing its essential property. Different from previous evolutionary game theory for CC, our new model provides a much simpler but useful viewpoint to analyze its convergence, since only the pure Nash equilibrium concept is needed and more general fitness landscapes can be explicitly considered. Based on convergence analyses, we propose a hierarchical decomposition strategy for better generalization, as for any decomposition there is a risk of getting trapped into a suboptimal Nash equilibrium. Finally, we use powerful distributed computing to accelerate it under the multi-level learning framework, which combines the fine-tuning ability from decomposition with the invariance property of CMA-ES. Experiments on a set of high-dimensional functions validate both its search performance and scalability (w.r.t. CPU cores) on a clustering computing platform with 400 CPU cores

Stochastic make-to-stock inventory deployment problem: an endosymbiotic psychoclonal algorithm based approach

Integrated steel manufacturers (ISMs) have no specific product, they just produce finished product from the ore. This enhances the uncertainty prevailing in the ISM regarding the nature of the finished product and significant demand by customers. At present low cost mini-mills are giving firm competition to ISMs in terms of cost, and this has compelled the ISM industry to target customers who want exotic products and faster reliable deliveries. To meet this objective, ISMs are exploring the option of satisfying part of their demand by converting strategically placed products, this helps in increasing the variability of product produced by the ISM in a short lead time. In this paper the authors have proposed a new hybrid evolutionary algorithm named endosymbiotic-psychoclonal (ESPC) to decide what and how much to stock as a semi-product in inventory. In the proposed theory, the ability of previously proposed psychoclonal algorithms to exploit the search space has been increased by making antibodies and antigen more co-operative interacting species. The efficacy of the proposed algorithm has been tested on randomly generated datasets and the results compared with other evolutionary algorithms such as genetic algorithms (GA) and simulated annealing (SA). The comparison of ESPC with GA and SA proves the superiority of the proposed algorithm both in terms of quality of the solution obtained and convergence time required to reach the optimal/near optimal value of the solution

Study the Effects of Multilevel Selection in Multi-Population Cultural Algorithm

This is a study on the effects of multilevel selection (MLS) theory in optimizing numerical functions. Based on this theory, a new architecture for Multi-Population Cultural Algorithm is proposed which incorporates a new multilevel selection framework (ML-MPCA). The approach used in this paper is based on biological group selection theory that states natural selection acts collectively on all the members of a given group. The effects of cooperation are studied using n-player prisoner’s dilemma. In this game, N individuals are randomly divided into m groups and individuals independently choose to be either cooperator or defector. A two-level selection process is introduced namely within group selection and between group selection. Individuals interact with the other members of the group in an evolutionary game that determines their fitness. The principal idea behind incorporating this multilevel selection model is to avoid premature convergence and to escape from local optima and for better exploration of the search space. We test our algorithm using the CEC 2015 expensive benchmark functions to evaluate its performance. These problems are a set of 15 functions which includes varied function categories. We show that our proposed algorithm improves solution accuracy and consistency. For 10 dimensional problems, the proposed method has 8 out 15 better results and for 30-dimensional problems we have 11 out of 15 better results when compared to the existing algorithms. The proposed model can be extended to more than two levels of selection and can also include migration

Distributed evolutionary algorithms and their models: A survey of the state-of-the-art

The increasing complexity of real-world optimization problems raises new challenges to evolutionary computation. Responding to these challenges, distributed evolutionary computation has received considerable attention over the past decade. This article provides a comprehensive survey of the state-of-the-art distributed evolutionary algorithms and models, which have been classified into two groups according to their task division mechanism. Population-distributed models are presented with master-slave, island, cellular, hierarchical, and pool architectures, which parallelize an evolution task at population, individual, or operation levels. Dimension-distributed models include coevolution and multi-agent models, which focus on dimension reduction. Insights into the models, such as synchronization, homogeneity, communication, topology, speedup, advantages and disadvantages are also presented and discussed. The study of these models helps guide future development of different and/or improved algorithms. Also highlighted are recent hotspots in this area, including the cloud and MapReduce-based implementations, GPU and CUDA-based implementations, distributed evolutionary multiobjective optimization, and real-world applications. Further, a number of future research directions have been discussed, with a conclusion that the development of distributed evolutionary computation will continue to flourish

Novelty-driven cooperative coevolution

Cooperative coevolutionary algorithms (CCEAs) rely on multiple coevolving populations for the evolution of solutions composed of coadapted components. CCEAs enable, for instance, the evolution of cooperative multiagent systems composed of heterogeneous agents, where each agent is modelled as a component of the solution. Previous works have, however, shown that CCEAs are biased toward stability: the evolutionary process tends to converge prematurely to stable states instead of (near-)optimal solutions. In this study, we show how novelty search can be used to avoid the counterproductive attraction to stable states in coevolution. Novelty search is an evolutionary technique that drives evolution toward behavioural novelty and diversity rather than exclusively pursuing a static objective. We evaluate three novelty-based approaches that rely on, respectively (1) the novelty of the team as a whole, (2) the novelty of the agents’ individual behaviour, and (3) the combination of the two. We compare the proposed approaches with traditional fitness-driven cooperative coevolution in three simulated multirobot tasks. Our results show that team-level novelty scoring is the most effective approach, significantly outperforming fitness-driven coevolution at multiple levels. Novelty-driven cooperative coevolution can substantially increase the potential of CCEAs while maintaining a computational complexity that scales well with the number of populations.info:eu-repo/semantics/publishedVersio

Evolutionary Reinforcement Learning: A Survey

Reinforcement learning (RL) is a machine learning approach that trains agents to maximize cumulative rewards through interactions with environments. The integration of RL with deep learning has recently resulted in impressive achievements in a wide range of challenging tasks, including board games, arcade games, and robot control. Despite these successes, there remain several crucial challenges, including brittle convergence properties caused by sensitive hyperparameters, difficulties in temporal credit assignment with long time horizons and sparse rewards, a lack of diverse exploration, especially in continuous search space scenarios, difficulties in credit assignment in multi-agent reinforcement learning, and conflicting objectives for rewards. Evolutionary computation (EC), which maintains a population of learning agents, has demonstrated promising performance in addressing these limitations. This article presents a comprehensive survey of state-of-the-art methods for integrating EC into RL, referred to as evolutionary reinforcement learning (EvoRL). We categorize EvoRL methods according to key research fields in RL, including hyperparameter optimization, policy search, exploration, reward shaping, meta-RL, and multi-objective RL. We then discuss future research directions in terms of efficient methods, benchmarks, and scalable platforms. This survey serves as a resource for researchers and practitioners interested in the field of EvoRL, highlighting the important challenges and opportunities for future research. With the help of this survey, researchers and practitioners can develop more efficient methods and tailored benchmarks for EvoRL, further advancing this promising cross-disciplinary research field

Knowledge management overview of feature selection problem in high-dimensional financial data: Cooperative co-evolution and Map Reduce perspectives

The term big data characterizes the massive amounts of data generation by the advanced technologies in different domains using 4Vs volume, velocity, variety, and veracity-to indicate the amount of data that can only be processed via computationally intensive analysis, the speed of their creation, the different types of data, and their accuracy. High-dimensional financial data, such as time-series and space-Time data, contain a large number of features (variables) while having a small number of samples, which are used to measure various real-Time business situations for financial organizations. Such datasets are normally noisy, and complex correlations may exist between their features, and many domains, including financial, lack the al analytic tools to mine the data for knowledge discovery because of the high-dimensionality. Feature selection is an optimization problem to find a minimal subset of relevant features that maximizes the classification accuracy and reduces the computations. Traditional statistical-based feature selection approaches are not adequate to deal with the curse of dimensionality associated with big data. Cooperative co-evolution, a meta-heuristic algorithm and a divide-And-conquer approach, decomposes high-dimensional problems into smaller sub-problems. Further, MapReduce, a programming model, offers a ready-To-use distributed, scalable, and fault-Tolerant infrastructure for parallelizing the developed algorithm. This article presents a knowledge management overview of evolutionary feature selection approaches, state-of-The-Art cooperative co-evolution and MapReduce-based feature selection techniques, and future research directions

06061 Abstracts Collection -- Theory of Evolutionary Algorithms

From 05.02.06 to 10.02.06, the Dagstuhl Seminar 06061 ``Theory of Evolutionary Algorithms\u27\u27 was held in the International Conference and Research Center (IBFI), Schloss Dagstuhl. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed. Abstracts of the presentations given during the seminar as well as abstracts of seminar results and ideas are put together in this paper. The first section describes the seminar topics and goals in general. Links to extended abstracts or full papers are provided, if available

Knowledge Migration Strategies for Optimization of Multi-Population Cultural Algorithm

Evolutionary Algorithms (EAs) are meta-heuristic algorithms used for optimization of complex problems. Cultural Algorithm (CA) is one of the EA which incorporates knowledge for optimization. CA with multiple population spaces each incorporating culture and genetic evolution to obtain better solutions are known as Multi-Population Cultural Algorithm (MPCA). MPCA allows to introduce a diversity of knowledge in a dynamic and heterogeneous environment. In an MPCA each population represents a solution space. An individual belonging to a given population could migrate from one population to another for the purpose of introducing new knowledge that influences other individuals in the population. In this thesis, we provide different migration strategies which are inspired from game theory model to improve the quality of solutions. Migration among the different population in MPCA can address the problem of knowledge sharing among population spaces. We have introduced five different migration strategies which are related to the field of economics. The principal idea behind incorporating these strategies is to improve the rate of convergence, increase diversity, better exploration of the search space, to avoid premature convergence and to escape from local optima. Strategies are particularly taken from the economics background as it allows the individual and the population to use their knowledge and make a decision whether to cooperate or to defect with other individuals and populations. We have tested the proposed algorithms against CEC 2015 expensive benchmark problems. These problems are a set of 15 functions which includes varied function categories. Results depict that it leads a to better solution when proposed algorithms used for problems with complex nature and higher dimensions. For 10 dimensional problems the proposed strategies have 7 out 15 better results and for 30 dimensional problems we have 12 out of 15 better results when compared to the existing algorithms