Evolutionary Algorithms

Evolutionary algorithms (EAs) are population-based metaheuristics, originally inspired by aspects of natural evolution. Modern varieties incorporate a broad mixture of search mechanisms, and tend to blend inspiration from nature with pragmatic engineering concerns; however, all EAs essentially operate by maintaining a population of potential solutions and in some way artificially 'evolving' that population over time. Particularly well-known categories of EAs include genetic algorithms (GAs), Genetic Programming (GP), and Evolution Strategies (ES). EAs have proven very successful in practical applications, particularly those requiring solutions to combinatorial problems. EAs are highly flexible and can be configured to address any optimization task, without the requirements for reformulation and/or simplification that would be needed for other techniques. However, this flexibility goes hand in hand with a cost: the tailoring of an EA's configuration and parameters, so as to provide robust performance for a given class of tasks, is often a complex and time-consuming process. This tailoring process is one of the many ongoing research areas associated with EAs.Comment: To appear in R. Marti, P. Pardalos, and M. Resende, eds., Handbook of Heuristics, Springe

Cooperative co-evolution for feature selection in big data with random feature grouping

© 2020, The Author(s). A massive amount of data is generated with the evolution of modern technologies. This high-throughput data generation results in Big Data, which consist of many features (attributes). However, irrelevant features may degrade the classification performance of machine learning (ML) algorithms. Feature selection (FS) is a technique used to select a subset of relevant features that represent the dataset. Evolutionary algorithms (EAs) are widely used search strategies in this domain. A variant of EAs, called cooperative co-evolution (CC), which uses a divide-and-conquer approach, is a good choice for optimization problems. The existing solutions have poor performance because of some limitations, such as not considering feature interactions, dealing with only an even number of features, and decomposing the dataset statically. In this paper, a novel random feature grouping (RFG) has been introduced with its three variants to dynamically decompose Big Data datasets and to ensure the probability of grouping interacting features into the same subcomponent. RFG can be used in CC-based FS processes, hence called Cooperative Co-Evolutionary-Based Feature Selection with Random Feature Grouping (CCFSRFG). Experiment analysis was performed using six widely used ML classifiers on seven different datasets from the UCI ML repository and Princeton University Genomics repository with and without FS. The experimental results indicate that in most cases [i.e., with naïve Bayes (NB), support vector machine (SVM), k-Nearest Neighbor (k-NN), J48, and random forest (RF)] the proposed CCFSRFG-1 outperforms an existing solution (a CC-based FS, called CCEAFS) and CCFSRFG-2, and also when using all features in terms of accuracy, sensitivity, and specificity

Learning Bayesian networks using evolutionary computation and its application in classification.

by Lee Shing-yan.Thesis (M.Phil.)--Chinese University of Hong Kong, 2001.Includes bibliographical references (leaves 126-133).Abstracts in English and Chinese.Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Problem Statement --- p.4Chapter 1.2 --- Contributions --- p.4Chapter 1.3 --- Thesis Organization --- p.5Chapter 2 --- Background --- p.7Chapter 2.1 --- Bayesian Networks --- p.7Chapter 2.1.1 --- A Simple Example [42] --- p.8Chapter 2.1.2 --- Formal Description and Notations --- p.9Chapter 2.1.3 --- Learning Bayesian Network from Data --- p.14Chapter 2.1.4 --- Inference on Bayesian Networks --- p.18Chapter 2.1.5 --- Applications of Bayesian Networks --- p.19Chapter 2.2 --- Bayesian Network Classifiers --- p.20Chapter 2.2.1 --- The Classification Problem in General --- p.20Chapter 2.2.2 --- Bayesian Classifiers --- p.21Chapter 2.2.3 --- Bayesian Network Classifiers --- p.22Chapter 2.3 --- Evolutionary Computation --- p.28Chapter 2.3.1 --- Four Kinds of Evolutionary Computation --- p.29Chapter 2.3.2 --- Cooperative Coevolution --- p.31Chapter 3 --- Bayesian Network Learning Algorithms --- p.33Chapter 3.1 --- Related Work --- p.34Chapter 3.1.1 --- Using GA --- p.34Chapter 3.1.2 --- Using EP --- p.36Chapter 3.1.3 --- Criticism of the Previous Approaches --- p.37Chapter 3.2 --- Two New Strategies --- p.38Chapter 3.2.1 --- A Hybrid Framework --- p.38Chapter 3.2.2 --- A New Operator --- p.39Chapter 3.3 --- CCGA --- p.44Chapter 3.3.1 --- The Algorithm --- p.45Chapter 3.3.2 --- CI Test Phase --- p.46Chapter 3.3.3 --- Cooperative Coevolution Search Phase --- p.47Chapter 3.4 --- HEP --- p.52Chapter 3.4.1 --- A Novel Realization of the Hybrid Framework --- p.54Chapter 3.4.2 --- Merging in HEP --- p.55Chapter 3.4.3 --- Prevention of Cycle Formation --- p.55Chapter 3.5 --- Summary --- p.56Chapter 4 --- Evaluation of Proposed Learning Algorithms --- p.57Chapter 4.1 --- Experimental Methodology --- p.57Chapter 4.2 --- Comparing the Learning Algorithms --- p.61Chapter 4.2.1 --- Comparing CCGA with MDLEP --- p.63Chapter 4.2.2 --- Comparing HEP with MDLEP --- p.65Chapter 4.2.3 --- Comparing CCGA with HEP --- p.68Chapter 4.3 --- Performance Analysis of CCGA --- p.70Chapter 4.3.1 --- Effect of Different α --- p.70Chapter 4.3.2 --- Effect of Different Population Sizes --- p.72Chapter 4.3.3 --- Effect of Varying Crossover and Mutation Probabilities --- p.73Chapter 4.3.4 --- Effect of Varying Belief Factor --- p.76Chapter 4.4 --- Performance Analysis of HEP --- p.77Chapter 4.4.1 --- The Hybrid Framework and the Merge Operator --- p.77Chapter 4.4.2 --- Effect of Different Population Sizes --- p.80Chapter 4.4.3 --- Effect of Different --- p.81Chapter 4.4.4 --- Efficiency of the Merge Operator --- p.84Chapter 4.5 --- Summary --- p.85Chapter 5 --- Learning Bayesian Network Classifiers --- p.87Chapter 5.1 --- Issues in Learning Bayesian Network Classifiers --- p.88Chapter 5.2 --- The Multinet Classifier --- p.89Chapter 5.3 --- The Augmented Bayesian Network Classifier --- p.91Chapter 5.4 --- Experimental Methodology --- p.94Chapter 5.5 --- Experimental Results --- p.97Chapter 5.6 --- Discussion --- p.103Chapter 5.7 --- Application in Direct Marketing --- p.106Chapter 5.7.1 --- The Direct Marketing Problem --- p.106Chapter 5.7.2 --- Response Models --- p.108Chapter 5.7.3 --- Experiment --- p.109Chapter 5.8 --- Summary --- p.115Chapter 6 --- Conclusion --- p.116Chapter 6.1 --- Summary --- p.116Chapter 6.2 --- Future Work --- p.118Chapter A --- A Supplementary Parameter Study --- p.120Chapter A.1 --- Study on CCGA --- p.120Chapter A.1.1 --- Effect of Different α --- p.120Chapter A.1.2 --- Effect of Different Population Sizes --- p.121Chapter A.1.3 --- Effect of Varying Crossover and Mutation Probabilities --- p.121Chapter A.1.4 --- Effect of Varying Belief Factor --- p.122Chapter A.2 --- Study on HEP --- p.123Chapter A.2.1 --- The Hybrid Framework and the Merge Operator --- p.123Chapter A.2.2 --- Effect of Different Population Sizes --- p.124Chapter A.2.3 --- Effect of Different Δα --- p.124Chapter A.2.4 --- Efficiency of the Merge Operator --- p.12

Regularized model learning in EDAs for continuous and multi-objective optimization

Probabilistic modeling is the de�ning characteristic of estimation of distribution algorithms (EDAs) which determines their behavior and performance in optimization. Regularization is a well-known statistical technique used for obtaining an improved model by reducing the generalization error of estimation, especially in high-dimensional problems. `1-regularization is a type of this technique with the appealing variable selection property which results in sparse model estimations. In this thesis, we study the use of regularization techniques for model learning in EDAs. Several methods for regularized model estimation in continuous domains based on a Gaussian distribution assumption are presented, and analyzed from di�erent aspects when used for optimization in a high-dimensional setting, where the population size of EDA has a logarithmic scale with respect to the number of variables. The optimization results obtained for a number of continuous problems with an increasing number of variables show that the proposed EDA based on regularized model estimation performs a more robust optimization, and is able to achieve signi�cantly better results for larger dimensions than other Gaussian-based EDAs. We also propose a method for learning a marginally factorized Gaussian Markov random �eld model using regularization techniques and a clustering algorithm. The experimental results show notable optimization performance on continuous additively decomposable problems when using this model estimation method. Our study also covers multi-objective optimization and we propose joint probabilistic modeling of variables and objectives in EDAs based on Bayesian networks, speci�cally models inspired from multi-dimensional Bayesian network classi�ers. It is shown that with this approach to modeling, two new types of relationships are encoded in the estimated models in addition to the variable relationships captured in other EDAs: objectivevariable and objective-objective relationships. An extensive experimental study shows the e�ectiveness of this approach for multi- and many-objective optimization. With the proposed joint variable-objective modeling, in addition to the Pareto set approximation, the algorithm is also able to obtain an estimation of the multi-objective problem structure. Finally, the study of multi-objective optimization based on joint probabilistic modeling is extended to noisy domains, where the noise in objective values is represented by intervals. A new version of the Pareto dominance relation for ordering the solutions in these problems, namely �-degree Pareto dominance, is introduced and its properties are analyzed. We show that the ranking methods based on this dominance relation can result in competitive performance of EDAs with respect to the quality of the approximated Pareto sets. This dominance relation is then used together with a method for joint probabilistic modeling based on `1-regularization for multi-objective feature subset selection in classi�cation, where six di�erent measures of accuracy are considered as objectives with interval values. The individual assessment of the proposed joint probabilistic modeling and solution ranking methods on datasets with small-medium dimensionality, when using two di�erent Bayesian classi�ers, shows that comparable or better Pareto sets of feature subsets are approximated in comparison to standard methods

Machine learning into metaheuristics: A survey and taxonomy of data-driven metaheuristics

During the last years, research in applying machine learning (ML) to design efficient, effective and robust metaheuristics became increasingly popular. Many of those data driven metaheuristics have generated high quality results and represent state-of-the-art optimization algorithms. Although various appproaches have been proposed, there is a lack of a comprehensive survey and taxonomy on this research topic. In this paper we will investigate different opportunities for using ML into metaheuristics. We define uniformly the various ways synergies which might be achieved. A detailed taxonomy is proposed according to the concerned search component: target optimization problem, low-level and high-level components of metaheuristics. Our goal is also to motivate researchers in optimization to include ideas from ML into metaheuristics. We identify some open research issues in this topic which needs further in-depth investigations

The Markov network fitness model

Fitness modelling is an area of research which has recently received much interest among the evolutionary computing community. Fitness models can improve the efficiency of optimisation through direct sampling to generate new solutions, guiding of traditional genetic operators or as surrogates for a noisy or long-running fitness functions. In this chapter we discuss the application of Markov networks to fitness modelling of black-box functions within evolutionary computation, accompanied by discussion on the relationship betweenMarkov networks andWalsh analysis of fitness functions.We review alternative fitness modelling and approximation techniques and draw comparisons with the Markov network approach. We discuss the applicability of Markov networks as fitness surrogates which may be used for constructing guided operators or more general hybrid algorithms.We conclude with some observations and issues which arise from work conducted in this area so far

A Methodology to Evolve Cooperation in Pursuit Domain using Genetic Network Programming

The design of strategies to devise teamwork and cooperation among agents is a central research issue in the field of multi-agent systems (MAS). The complexity of the cooperative strategy design can rise rapidly with increasing number of agents and their behavioral sophistication. The field of cooperative multi-agent learning promises solutions to such problems by attempting to discover agent behaviors as well as suggesting new approaches by applying machine learning techniques. Due to the difficulty in specifying a priori for an effective algorithm for multiple interacting agents, and the inherent adaptability of artificially evolved agents, recently, the use of evolutionary computation as a machining learning technique and a design process has received much attention. In this thesis, we design a methodology using an evolutionary computation technique called Genetic Network Programming (GNP) to automatically evolve teamwork and cooperation among agents in the pursuit domain. Simulation results show that our proposed methodology was effective in evolving teamwork and cooperation among agents. Compared with Genetic Programming approaches, its performance is significantly superior, its computation cost is less and the learning speed is faster. We also provide some analytical results of the proposed approach

Computational intelligence techniques in asset risk analysis

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The problem of asset risk analysis is positioned within the computational intelligence paradigm. We suggest an algorithm for reformulating asset pricing, which involves incorporating imprecise information into the pricing factors through fuzzy variables as well as a calibration procedure for their possibility distributions. Then fuzzy mathematics is used to process the imprecise factors and obtain an asset evaluation. This evaluation is further automated using neural networks with sign restrictions on their weights. While such type of networks has been only used for up to two network inputs and hypothetical data, here we apply thirty-six inputs and empirical data. To achieve successful training, we modify the Levenberg-Marquart backpropagation algorithm. The intermediate result achieved is that the fuzzy asset evaluation inherits features of the factor imprecision and provides the basis for risk analysis. Next, we formulate a risk measure and a risk robustness measure based on the fuzzy asset evaluation under different characteristics of the pricing factors as well as different calibrations. Our database, extracted from DataStream, includes thirty-five companies traded on the London Stock Exchange. For each company, the risk and robustness measures are evaluated and an asset risk analysis is carried out through these values, indicating the implications they have on company performance. A comparative company risk analysis is also provided. Then, we employ both risk measures to formulate a two-step asset ranking method. The assets are initially rated according to the investors' risk preference. In addition, an algorithm is suggested to incorporate the asset robustness information and refine further the ranking benefiting market analysts. The rationale provided by the ranking technique serves as a point of departure in designing an asset risk classifier. We identify the fuzzy neural network structure of the classifier and develop an evolutionary training algorithm. The algorithm starts with suggesting preliminary heuristics in constructing a sufficient training set of assets with various characteristics revealed by the values of the pricing factors and the asset risk values. Then, the training algorithm works at two levels, the inner level targets weight optimization, while the outer level efficiently guides the exploration of the search space. The latter is achieved by automatically decomposing the training set into subsets of decreasing complexity and then incrementing backward the corresponding subpopulations of partially trained networks. The empirical results prove that the developed algorithm is capable of training the identified fuzzy network structure. This is a problem of such complexity that prevents single-level evolution from attaining meaningful results. The final outcome is an automatic asset classifier, based on the investors’ perceptions of acceptable risk. All the steps described above constitute our approach to reformulating asset risk analysis within the approximate reasoning framework through the fusion of various computational intelligence techniques

Evolutionary Computation 2020

Intelligent optimization is based on the mechanism of computational intelligence to refine a suitable feature model, design an effective optimization algorithm, and then to obtain an optimal or satisfactory solution to a complex problem. Intelligent algorithms are key tools to ensure global optimization quality, fast optimization efficiency and robust optimization performance. Intelligent optimization algorithms have been studied by many researchers, leading to improvements in the performance of algorithms such as the evolutionary algorithm, whale optimization algorithm, differential evolution algorithm, and particle swarm optimization. Studies in this arena have also resulted in breakthroughs in solving complex problems including the green shop scheduling problem, the severe nonlinear problem in one-dimensional geodesic electromagnetic inversion, error and bug finding problem in software, the 0-1 backpack problem, traveler problem, and logistics distribution center siting problem. The editors are confident that this book can open a new avenue for further improvement and discoveries in the area of intelligent algorithms. The book is a valuable resource for researchers interested in understanding the principles and design of intelligent algorithms