Abstract. To reduce the number of expensive fitness function evaluations in evolutionary optimization, several individual-based and generation-based evolution control methods have been suggested. This paper compares four individual-based evolution control frameworks on three widely used test functions. Feedforward neural networks are employed for fitness estimation. Two conclusions can be drawn from our simulation results. First, the pre-selection strategy seems to be the most stable individual-based evolution control method. Second, structure optimization of neural networks mostly improves the performance of all compared algorithms. 

Gräning, L

Jin, Y

Sendhoff, B

Lars Gräning

Yaochu Jin

Bernhard Sendhoff

CiteSeerX

Efficient Evolutionary Optimization Using Individual-based Evolution Control and Neural Networks: A Comparative Study

University of Surrey

Efficient Evolutionary Optimization UsingIndividual-based Evolution Control and NeuralNetworks: A Comparative StudyLars Gräning1 and Yaochu Jin2 and Bernhard Sendhoff21- Department of Neuroinformatics, Technical University of IlmenauP.O.Box 100565, 98684 Ilmenau, Germany2- Honda Research Institute EuropeCarl-Legien-Str. 30, 63073 Offenbach/Main, Germanyyaochu.jin@honda-ri.deAbstract. To reduce the number of expensive fitness function evaluationsin evolutionary optimization, several individual-based and generation-basedevolution control methods have been suggested. This paper compares fourindividual-based evolution control frameworks on three widely used testfunctions. Feedforward neural networks are employed for fitness estima-tion. Two conclusions can be drawn from our simulation results. First, thepre-selection strategy seems to be the most stable individual-based evolu-tion control method. Second, structure optimization of neural networksmostly improves the performance of all compared algorithms.1 IntroductionIt has been shown that evolutionary algorithms are very powerful in solvingmany real-world optimization tasks such as design optimization, see e.g., [1].In order to reduce the number of time-consuming fitness evaluations, one ideais to estimate the fitness using computationally efficient meta-models [2]. Inthis work, artificial neural networks are used due to their strong approximationcapability [3]. One problem to deal with in real-world optimization problems isthat it is difficult to acquire enough training data to achieve sufficiently goodapproximation, which could result in false convergence [4]. Therefore it is notadvisable to use only the neural network as a surrogate for the original fitnessfunction. To avoid false convergence, the neural network model should be used inconjunction with the original fitness function. This is termed evolution control ormodel management [4]. If evolution control is used, new data become available,which can then be used for on-line neural network training during optimization.In this paper, we compare four individual-based evolution control methods whichwill be described in the next section.2 Individual-Based Evolution Control MethodsIn individual-based evolution control methods, the main issue is to determine ineach generation which individuals should be evaluated using the expensive fitnessfunction and which should be estimated. In the following, four individual-basedstrategies are described. As illustrated in Fig. 1, the four methods can be de-scribed in a common framework. First, λ’ offspring individuals are generatedfrom µ parents using recombination and mutation. After that, the individual-based control method determines which λ∗ offspring are evaluated with the orig-inal fitness function. The evaluation results are used to train the neural networkbefore the fitness of the remaining λ’−λ∗ offspring is estimated by the neuralnetwork. Finally, µ parents are selected from the best λ individuals according totheir fitness. This procedure repeats until a termination condition is satisfied.λ−λ’ ∗’µIndividual−basedcontrol strategySelectionOriginal FitnessFunctionλRecombine/MutateNeural NetworkOnlineLearningλ∗λFig. 1: A generic framework for evolutionary optimization using individual-basedcontrol methods.2.1 Best Strategy (BS)In the best strategy [4], λ′ = λ offspring are evaluated with the neural networkand the λ∗ best ones are re-evaluated with the original fitness function. Aftertraining the neural network the remaining λ′−λ∗ individuals are evaluated againwith the neural network. The µ best individuals from the λ individuals becomeparents of the next generation.2.2 Pre-Selection (PreS)In the pre-selection strategy [5], λ′ > λ offspring individuals are generatedthrough recombination and mutation, and the neural network is used to esti-mate the fitness value of the offspring. The λ∗ = λ most promising individualsare pre-selected from the λ’ offspring and re-evaluated using the original fitnessfunction. The main difference to the best strategy is that the µ parents are se-lected only from the λ∗ individuals evaluated using the original fitness function.2.3 Clustering Technique (CT)In [6], a different approach is presented to determine which individuals are tobe evaluated with the original fitness function. Using the k-means clusteringtechnique, the λ′ = λ offspring individuals are grouped into λ∗ clusters. Nowthe λ∗ individuals closest to the cluster center are evaluated using the originalfitness function. The results of the fitness evaluations, as in other methods, areused to train the neural network. The fitness value of the remaining λ′ − λ∗individuals is estimated using the neural network. Finally, µ individuals areselected from all λ′ offspring as parents of the next generation.2.4 Clustering Technique with Best Strategy (CTBS)The clustering technique with best strategy is extended from the clustering tech-nique. The offspring individuals are grouped into λ∗ clusters. After clustering,the neural network is used to estimate the fitness of individuals. Instead of theindividual closest to each cluster center, the best individual in each cluster willbe evaluated with the original fitness function.3 Structure Optimization of the Neural NetworkTo improve the approximation quality of the neural network, the structure ofthe neural network can also be optimized during the design optimization. Agenetic algorithm has been used to optimize both the structure and the weightsof the neural network. In generating offspring, specific mutation operations areemployed. The mutation operations allow to insert or delete a single connectionor neuron, and the weights are mutated by adding a normally distributed randomnumber. After mutation, life-time learning of the weights is performed andthe learned weights are coded back to the individuals, which is known as theLamarckian mechanism. The EP-tournament-selection is adopted to reproducethe individuals representing the neural networks with the lowest mean squarederror on the training data. See [7] for details.4 Simulation ResultsIn the simulations, a (µ, λ) covariance matrix adaptation evolution strategy with-out recombination [8] is adopted, where µ is fixed to 3 and λ is fixed to 12. Thestrategy parameters of the covariance matrix are initialized between σmin = 0.05and σmax = 4. The settings of λ’ and λ∗ listed in Table 1 are based on recom-mendations or findings in [5] and [6]. The maximum number of expensive fitnessevaluations is 1200.PlainES PreS BS CT CTBSλ’ 12 24 12 12 12λ∗ 12 12 6 6 6Table 1: Settings of the parameters λ’ and λ∗.The neural network used in the simulations consists of 10 input nodes, onehidden layer with four hidden neurons and one output. If structure optimizationis used the number of hidden neurons is not fixed. An improved version of theRProp algorithm is used to train the MLP online during optimization. To achievegood local approximation of the original fitness landscape, only data of the mostrecent 50 evaluations are used for training.(a) (b)Sphere126 366 606 846 108610−410−2100Exact EvaluationsFitnessplainESPreSBSCTCTBS126 366 606 846 108610−410−2100Exact EvaluationsFitnessplainESPreSBSCTCTBSRosenbrock126 366 606 846 1086101102103104Exact EvaluationsFitnessplainESPreSBSCTCTBS126 366 606 846 1086101102103104Exact EvaluationsFitnessplainESPreSBSCTCTBSAckley126 366 606 846 108610−310−210−1100101Exact EvaluationsFitnessplainESPreSBSCTCTBS126 366 606 846 108610−210−1100101Exact EvaluationsFitnessplainESPreSBSCTCTBSFig. 2: Results from the three 10-dimensional test functions. (a) Without struc-ture optimization, and (b) with structure optimization.The results from the Sphere, Rosenbrock and Ackley function are presentedin Fig. 2 and Fig. 3. The left column presents the results with and the rightcolumn without structure optimization of the neural networks. In Fig. 2, themedian of the best fitness in each generation over 20 runs are plotted againstthe number of expensive fitness evaluations.To show the statistical significance between the evolution control methodsand the plain evolution strategy, the boxplot of the results are given in Fig. 3.The boxplot illustrates the median and the variance of the fitness value of thebest individual in the final generation over 20 runs. The notches of the boxes inthe plot are the graphical equivalence to the student t-test. If the notch in theboxplot of the two strategies overlap, there is no significant difference betweenthe medians of the strategies at a significance level of p = 0.05.(a) (b)SphereplainES PreS BS CT CTBS10−510−410−310−2FitnessplainES PreS BS CT CTBS10−510−410−3FitnessRosenbrockplainES PreS BS CT CTBS100101102103FitnessplainES PreS BS CT CTBS101102103FitnessAckleyPlainES PreS BS CT CTBS10−210−1100FitnessplainES PreS BS CT CTBS10−210−1100FitnessFig. 3: Boxplot of the best fitness in the final generation over 20 runs after 1200exact fitness evaluations are done. (a) Without structure optimization, and (b)with structure optimization.From Fig. 2 and Fig. 3, we can see that all evolution control methods exceptthe best strategy improve the performance of the plain evolution strategy sig-nificantly on the 10D Sphere function. But there are no statistically significantdifferences between the model-assisted strategies themselves, no matter whetherstructure optimization of the neural networks are conducted or not.It turns out that the clustering technique with best strategy outperformsother algorithms on the 10D Rosenbrock function, when no structure optimiza-tion of the neural network is carried out. However, all algorithms fail to improvethe performance of the plain evolution strategy significantly. This result maybe attributed to the fact that the number of hidden neurons is not sufficientlylarge to approximate the Rosenbrock function. Meanwhile, the result indicatesthat with structure optimization, the neural networks perform locally very wellon the Rosenbrock function.From Fig. 2, it can be seen that the individual-based evolution control meth-ods perform well on the Ackley function. But as we can see in the boxplotsin Fig. 3 the variance of the strategies are very high except the pre-selectionstrategy. The pre-selection strategy outperforms the plain evolution strategy inalmost all of the 20 runs.5 ConclusionIn this paper, we compared four individual-based evolution control strategies onthree test functions. Neural networks with or without structure optimizationare used for estimating the fitness. From our results, we showed that the pre-selection strategy is the most promising one among the compared individual-based control strategies. The clustering based approaches are not as good as in[6] mainly because a single network instead of network ensembles has been usedin this study.The future work will be to adapt the number of individuals to be controlledduring optimization and to use neural network ensembles instead of a single net-work. The pre-selection and the clustering based strategies will be implementedin design optimization of three-dimensional turbine blades.References[1] M. Olhofer, T. Arima, T. Sonoda, and B. Sendhoff. Optimisation of a statorblade used in a transonic compressor cascade with evolution strategies. In AdaptiveComputation in Design and Manufacture, pages 45–54. Springer, 2000.[2] Y. Jin. A comprehensive survey of fitness approximation in evolutionary computa-tion. Soft Computing, 9(1):3–12, 2005.[3] K. Hornik, M. Stinchcombe, and H. White. Multilayer feedforward networks areuniversal approximators. Neural Networks, 2(5):359–366, 1989.[4] Y. Jin, M. Olhofer, and B. Sendhoff. A framework for evolutionary optimizationwith approximate fitness functions. IEEE Transactions on Evolutionary Computa-tion, 6(5):481–494, 2002.[5] H. Ulmer, F. Streichert, and A. Zell. Evolution strategies with controlled modelassistance. In Congress on Evolutionary Computation, pages 1569–1576, 2004.[6] Y. Jin and B. Sendhoff. Reducing fitness evaluations using clustering techniques andneural networks ensembles. In Genetic and Evolutionary Computation Conference,volume 3102 of LNCS, pages 688–699. Springer, 2004.[7] M. Hüsken, Y. Jin, and B. Sendhoff. Structure optimization of neural networks foraerodynamic optimization. Soft Computing Journal, 9(1):21–28, 2005.[8] N. Hansen and A. Ostermeier. Completely derandomized self-adaptation in evolu-tion strategies. Evolutionary Computation, 9(2):159–196, 2001.

English

Efficient evolutionary optimization using individual-based evolution control and neural networks: A comparative study.

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.69.9555

Efficient evolutionary optimization using individual-based evolution control and neural networks: A comparative study.

Abstract

Similar works

Full text

Available Versions

CiteSeerX

CiteSeerX

University of Surrey