Adaptive sampling for noisy problems

Abstract. The usual approach to deal with noise present in many realworld optimization problems is to take an arbitrary number of samples of the objective function and use the sample average as an estimate of the true objective value. The number of samples is typically chosen arbitrarily and remains constant for the entire optimization process. This paper studies an adaptive sampling technique that varies the number of samples based on the uncertainty of deciding between two individuals. Experiments demonstrate the effect of adaptive sampling on the final solution quality reached by a genetic algorithm and the computational cost required to find the solution. The results suggest that the adaptive technique can effectively eliminate the need to set the sample size a priori, but in many cases it requires high computational costs.

Pruning Neural Networks with Distribution Estimation Algorithms

Abstract. This paper describes the application of four evolutionary algorithms to the pruning of neural networks used in classification problems. Besides of a simple genetic algorithm (GA), the paper considers three distribution estimation algorithms (DEAs): a compact GA, an extended compact GA, and the Bayesian Optimization Algorithm. The objective is to determine if the DEAs present advantages over the simple GA in terms of accuracy or speed in this problem. The experiments considered a feedforward neural network trained with standard backpropagation and 15 public-domain and artificial data sets. In most cases, the pruned networks seemed to have better or equal accuracy than the original fully-connected networks. We found few differences in the accuracy of the networks pruned by the four EAs, but found large differences in the execution time. The results suggest that a simple GA with a small population might be the best algorithm for pruning networks on the data sets we tested.

A Survey of Parallel Genetic Algorithms

Genetic algorithms (GAs) are powerful search techniques that are used successfully to solve problems in many different disciplines. Parallel GAs are particularly easy to implement and promise substantial gains in performance. As such, there has been extensive research in this field. This survey attempts to collect, organize, and present in a unified way some of the most representative publications on parallel genetic algorithms. To organize the literature, the paper presents a categorization of the techniques used to parallelize GAs, and shows examples of all of them. However, since the majority of the research in this field has concentrated on parallel GAs with multiple populations, the survey focuses on this type of algorithms. Also, the paper describes some of the most significant problems in modeling and designing multi-population parallel GAs and presents some recent advancements

Using Markov Chains to Analyze a Bounding Case of Parallel Genetic Algorithms

This paper uses Markov chains to analyze the search quality of a bounding case of parallel genetic algorithms with multiple populations. In the bounding case considered here, each population exchanges individuals with all the others. First, the migration rate is set to the maximum value possible, and later the analysis is refined to consider lower migration rates. In the algorithm examined by this paper, migration occurs only after each population converges. Then, incoming individuals are incorporated into the populations and the algorithm restarts. The analysis shows how to calculate the probability that each population will eventually converge to the correct solution, and the expected number of migration-restart events until all the populations converge to the same solution

A Markov Chain Analysis of Parallel Genetic Algorithms with Arbitrary Topologies and Migration Rates

Implementations of parallel genetic algorithms (GAs) with multiple populations are common, but they introduce several parameters whose effect on the quality of the search is not well understood. Parameters such as the number of populations, their size, the topology of communications, and the migration rate have to be set carefully to reach adequate solutions. This paper shows how to predict the effects of the parallel GA's parameters on its search quality. The analysis considers any number of populations of arbitrary size, and does not place limitations on the topology of communications or the migration rate. The paper reviews some recent results on the case where each population is connected to all the others and the migration rate is set to the maximum value possible. This bounding case is the simplest to analyze, and it introduces the methodology that is used in the remainder of the paper to analyze arbitrary configurations of parallel GAs. This investigation considers that migratio..

On the Effects of Migration on the Fitness Distribution of Parallel Evolutionary Algorithms

Migration of individuals between populations may increase the selection pressure. This has the desirable consequence of speeding up convergence, but it may result in an excessively rapid loss of variation that may cause the search to fail. This paper describes the effects of migration on the distribution of tness. The calculations consider nite populations, arbitrary migration rates, and topologies with dierent numbers of neighbors. The paper shows that even if dierent algorithms are congured to produce the same selection intensity, they change the composition of the population in dierent ways. The results suggest that migration preserves more diversity as the number of neighbors increases. 1 INTRODUCTION A popular method to parallelize evolutionary algorithms (EAs) is to use multiple populations (also called demes) and allocate each to a dierent processor. In this method, the populations periodically exchange a few individuals in a process analogous to migratio..