The final published version of this article is available at the link below. Copyright @ MIT Press.Genetic Algorithms (GAs) emulate the natural evolution process and maintain a popilation of potential solutions to a given problem.  Through the population, GAs implicitly maintain the statistics about the search space.  This implicit statistics can be used explicitly to enhance GA's performance.  Inspired by this idea, a statistics-based adaptive non-uniform crossover (SANUX) has been proposed.  SANUX uses the statisics information of the alleles in each locus to adaptively caluclate the swapping probability of that locus for crossover operation.  A simple triangular function has been used to calculate the swapping probability.  In this paper new functions, the trapezoid and exponential functions, are proposed for SANUX instead of the triangular function.  Experiment results show that both functions further improve the performance of SANUX

Yang, S

English

Brunel University Research Archive

in Artificial Life VIII, Standish, Abbass, Bedau (eds)(MIT Press) 2002. pp 182–185 1Adaptive Crossover in Genetic Algorithms Using Statistics MechanismShengxiang YangDepartment of Mathematics and Computer Science, University of LeicesterUniversity Road, Leicester LE1 7RH, UK. Email: s.yang@mcs.le.ac.ukAbstractGenetic Algorithms (GAs) emulate the natural evolu-tion process and maintain a population of potentialsolutions to a given problem. Through the popula-tion, GAs implicitly maintain the statistics about thesearch space. This implicit statistics can be used ex-plicitly to enhance GA’s performance. Inspired by thisidea, a statistics-based adaptive non-uniform crossover(SANUX) has been proposed. SANUX uses the statis-tics information of the alleles in each locus to adap-tively calculate the swapping probability of that locusfor crossover operation. A simple triangular function hasbeen used to calculate the swapping probability. In thispaper two new functions, the trapezoid and exponen-tial functions, are proposed for SANUX instead of thetriangular function. Experiment results show that bothfunctions further improve the performance of SANUX.IntroductionGenetic Algorithms (GAs) emulate the natural evolutionprocess and maintain a population of potential solutionsto a given problem, which are evaluated by a problem-specific fitness function. This population is evolved byrandomly selecting relatively fit members and perform-ing genetic operations, especially recombination and mu-tation, on them to generate a new population (Gold-berg 1989). With the progress of the GA, the aver-age fitness of the population increases, hopefully lead-ing to the optimal solution(s) to the problem. Throughthe population GAs implicitly maintain statistics aboutthe search space. That is, useful materials or buildingblocks permeate in the population. GAs use the selec-tion, crossover and mutation operators to explicitly ex-tract the implicit statistics from the population to reachthe next set of points in the search space.In fact, this implicit statistics in the population can beused explicitly to enhance GA’s performance. Inspiredby this idea, a statistics-based adaptive non-uniformcrossover, called SANUX, has been proposed by Yang(2002). SANUX explicitly uses the statistics informa-tion of the alleles (values of a gene) in each locus to adap-tively calculate the swapping probability of that locus forcrossover operation. In Yang (2002) a simple triangularfunction has been used to calculate the swapping prob-ability from the statistics of allele distribution. In thispaper, two new functions, a trapezoid function and anexponential function, are proposed for SANUX insteadof the simple triangular function. These two functionsare proposed with the aim of using the statistics moreefficiently with respect to construction and protection ofuseful building blocks with the progress of the GA.Related Work on Crossover OperatorsTraditionally, GAs have used one-point or two-pointcrossover (De Jong 1975; Holland 1975). Researchershave also carried out experiments with multi-pointcrossover: n-point crossover (Eshelman, Caruana, &Schaffer 1989) and uniform crossover (Syswerda 1989).With the n-point crossover, n cut points are randomlychosen within the strings and the n−1 segments betweenthe n cut points of the two parents are exchanged. Uni-form crossover is the generalization of n-point crossover.It creates offsprings by swapping each bit of two parentswith a probability ps of 0.5. Spears and De Jong (1991)proposed the parameterized uniform crossover where thedecision of whether to swap for each locus is made bya biased coin flipping, i.e., the swapping probability pscould be other than 0.5.Traditional crossover operators are inherently allbased on uniformly randomization mechanism, i.e., gen-erating cut points (n-point crossover) or swapping points(uniform crossover) uniformly randomly across the chro-mosome. This situation is not very true with naturalevolution because it is intrinsically dynamic and adap-tive. Recently, researchers have applied adaptation tech-niques to crossover to enhance GA’s capabilities (Eiben,Hinterding, & Michalewicz 1999). According to Yang(2002), adaptation in crossover happens in three levelsfrom top to bottom. In the top level, crossover oper-ators are themself adapted during a run of the GA. Inthe medium level, the rate or probability of crossover isaltered during a run of the GA. In the bottom level, theposition of crossing or swapping probability in each locusis adapted during a run of the GA.2 in Artificial Life VIII, Standish, Abbass, Bedau (eds) (MIT Press) 2002. pp 182–185Statistics-based Adaptive Non-UniformCrossoverDescription of SANUXFor the convenience of description and analysis, Yang(2002) has introduced the concepts of intrinsic attributeand extrinsic tendency of allele valuing for a gene lo-cus. In the optimal binary-encoded solution(s) of a givenproblem, a gene locus is called 1-intrinsic if its allele is1, 0-intrinsic if its allele is 0, or neutral if its allele canbe either 0 or 1. During the running of a GA, for a genelocus, if the frequency of 1’s in its alleles over the pop-ulation tends to increase (to the limit of 1.0) with time,it is called 1-inclined; if the frequency of 1’s tends todecrease (to the limit of 0.0), it is called 0-inclined; oth-erwise, if there is no tendency of increasing or decreasing,it is called non-inclined.Usually and hopefully as the GA progresses, thosegene loci that are 1-intrinsic (or 0-intrinsic) will appearto be 1-inclined (or 0-inclined), i.e., the frequency of 1’sin the alleles of these loci will eventually converge to 1 (or0). This convergent information is implicit in the popula-tion. Traditional crossover operators make no use of thisimplicit information and consistently generate cut pointsor swapping points randomly but uniformly across thechromosome. Obviously it will be beneficial to link thesepartially or fully converged genes into groups or buildingblocks and co-evolve them during crossover operations.This is realized in SANUX. SANUX explicitly uses theconvergence information as feedback to adaptively ad-just the swapping probability for each locus based onthe statistics of alleles in that locus.We use the frequency of 1’s in the alleles in a locusover the population (equivalently we can also use the fre-quency of 0’s as the argument) to calculate correspond-ing swapping probability of that locus. The frequency of1’s in the alleles of a locus can be looked as the degree ofconvergence to “1” for that locus. Let L be the lengthof binary strings, f1(i, t) (i = 1, . . . , L) denote the fre-quency of 1’s in the alleles in locus i over the populationat time (generation) t and ps(i, t) (i = 1, . . . , L) denotethe swapping probability of locus i at time t. In Yang(2002), a simple triangular function is used to calculateps(i, t) from f1(i, t), defined as follows:ps(i, t) = Pmax − 2|f1(i, t)− 0.5|(Pmax − Pmin) (1)where |.| is an absolute function, Pmax and Pmin are themaximum and minimum allowable swapping probabili-ties for a locus respectively.Now during the evolution of the GA, after a new popu-lation has been created, we first calculate f1(i, t) for eachlocus i over population t and from this obtain ps(i, t) oflocus i. Then we can perform SANUX operations simi-larly as traditional parameterized uniform crossover, asillustrated in Figure 1.1’s Freq. in loci: 0.9 0.2 0.6 0.9 0.4 0.2Calculating: ⇓ ⇓ ⇓ ⇓ ⇓ ⇓Swapping Prob.: 0.1 0.2 0.4 0.1 0.4 0.2Biased Flipping: ⇓ ⇓ ⇓ ⇓ ⇓ ⇓Created Mask: 0 0 1 0 1 0Applying Mask: ⇓ ⇓Parent P1: 1 1 0 1 0 1Parent P2: 1 1 1 1 1 0Swapping: ⇓ ⇓Child C1: 1 1 1 1 1 1Child C2: 1 1 0 1 0 0Figure 1: An example operation of SANUX with triangu-lar calculation function where Pmax = 0.5 and Pmin = 0.From above descriptions, it can be seen that SANUXis very simple. With SANUX, what we add to tradi-tional uniform crossover are spatially only one real vec-tor that records the swapping probability for each lo-cus, and computationally only one statistics per gener-ation that calculates the frequency of ones (hence theswapping probability) for each locus. Another moreimportant point of SANUX is its property of implicitgene linkage. Gene linkage is the property of group-ing interactive genes to evolve them together. For ex-ample, in Figure 1 loci 1 and 4 are more convergentand implicitly linked though they are not adjacent be-cause the probability for them to co-evolve via crossoveris 0.1 ∗ 0.1 + 0.9 ∗ 0.9 = 0.82, which is very high.New Calculating FunctionsFrom above discussions, we can see that there is a closeloop control in SANUX in the sense of constructing andpreserving building blocks. The GA is first used to con-struct useful building blocks. Then after some timewhen some useful building blocks have been built upthe statistics information is used to protect these build-ing blocks from being destroyed by the crossover andhence divert the crossover to search unconverged loci formore unknown useful building blocks because they haverelatively higher swapping probabilities. With this un-derstanding, we can further enhance this close loop byintroducing new functions instead of the triangular func-tion to calculate the swapping probability of each locusfrom its allele distribution statistics.The first function is a trapezoid function, as shown inFigure 2, defined by the following equation.ps(i, t) =Pmax, if |f1(i, t)− 0.5| ≤ ab−|f1(i,t)−0.5|b−a ∗ (Pmax − Pmin) + Pmin,if a < |f1(i, t)− 0.5| < bPmin, if |f1(i, t)− 0.5| ≥ b(2)in Artificial Life VIII, Standish, Abbass, Bedau (eds)(MIT Press) 2002. pp 182–185 3where a and b are parameters that satisfy the relation of0 ≤ a ≤ b ≤ 0.5. As shown in Figure 2, the parametersa and b split the whole range of f1(i, t) into three zones:I) |f1(i, t) − 0.5| ≤ a; II) a < |f1(i, t) − 0.5| < b; III)|f1(i, t) − 0.5| ≥ b. Zone I is called constructive zonewhere useful building blocks are mainly constructed bySANUX. Zone II is called transient zone where buildingblocks are constructed as well as destroyed. And Zone IIIis called protective zone where building blocks found sofar are protected from being destroyed by SANUX whileexploring unknown useful building blocks is still carriedout by SANUX on unconverged loci. From Figure 2 itcan be seen that the larger the value of a, the wider theregion that allows the GA to construct building blocks,and that the smaller the value of b, the stronger theprotection to converged genes or found building blocksfrom being destroyed by SANUX.PminPmax0.0 0-Inclined      0.5      1-Inclined 1.0Swapping ProbabilityFrequency of 1’s in the Alleles in a LocusI IIII IIIIIIab0.0Figure 2: Trapezoid function for calculating the swap-ping probability of a locus.The second function we introduce here is an exponen-tial function, as shown in Figure 3, defined as follows:ps(i, t) = α ∗ exp(−(f1(i, t)− 0.5)2/β) (3)where α is the parameter with the same functionality asPmax in Equations (1) and (2), and β is the parameterthat combines the functionalities of parameters Pmin, aand b in Equation (2) and controls the falling speed ofps(i, t) when f1(i, t) diverts away from 0.5. The smallerthe value of β, the faster the falling speed of ps(i, t),and the stronger the protection to converged genes orbuilding blocks found so far. As shown in Figure 3, thesolid curve shows the function with (α, β) = (0.5, 0.04)while the dashed curve is when (α, β) = (0.5, 0.02).Both the two functions enlarge the region for the GAto construct building blocks and enhance the protectionto found building blocks at the same time. Both sidesare expected to be beneficial for SANUX’s performance.Experimental StudyThe Test ProblemsThe Max Ones Problem The Max Ones problem issimply to maximize 1’s in a binary string. The fitness ofPminPmax0.0 0-Inclined      0.5      1-Inclined 1.0Swapping ProbabilityFrequency of 1’s in the Alleles in a Locus(0.5, 0.04)(0.5, 0.02)Figure 3: Exponential function for calculating the swap-ping probability of a locus.a string is the number of 1’s it contains. A string lengthof 100 bits is used for this study.Royal Road Functions The Royal Road functionsR1 and R2 are devised by Forrest and Mitchell (1993)to investigate GA’s performance with respect to schemaprocessing and recombination in an idealized form. Theoptimal fitness for R1 and R2 is 64 and 192 respectively.Experiment ResultsTo test the performance of proposed two calculatingfunctions for SANUX, in our experiment study we com-pared the SANUX with the trapezoid and exponen-tial functions over SANUX with the triangular func-tion and traditional 2-point, 0.5 uniform and 0.2 uni-form crossover on the test problems. The parametersfor different functions of SANUX were set as follows:Pmax = 0.5 and Pmin = 0.0 for triangular and trapezoidfunctions, (a, b) = (0.1, 0.4) for trapezoid function, and(α, β) = (0.5, 0.04) for exponential function.For each experiment of combining different crossoverand test problems, 100 independent runs were executedunder the same 100 different random seeds. In all theexperiments, the GA used the fitness proportionate se-lection with the stochastic universal sampling (SUS) andelitist model, and bit flip mutation. And typically theprobabilities of crossover and mutation were fixed to 0.6and 0.001 respectively and the population size was setto 100 for each run. For each run, we recorded the best-so-far fitness every 100 evaluations. Here, only thosechromosomes changed by crossover and mutation oper-ations were evaluated and counted into the number ofevaluations. Each experiment result was averaged overthe 100 independent runs. The experiment results areshown in Figure 4 and Figure 5 respectively.From Figure 4 and Figure 5 it can be seen thatSANUX with different calculating functions has outper-formed traditional crossover operators on the test prob-lems. On the test problems lowering the swapping prob-ability ps from 0.5 to 0.2 improves the performance ofuniform crossover (0.2 uniform crossover outperforms 2-4 in Artificial Life VIII, Standish, Abbass, Bedau (eds) (MIT Press) 2002. pp 182–18560657075808590951000 50 100 150 200Average Best FitnessEvaluations ( x 100 )2-Point0.5 Uniform0.2 UniformSANUX-TriangleSANUX-TrapezoidSANUX-ExponentFigure 4: Average best-so-far curves for GAs with dif-ferent crossover operators on Max Ones problem.point and 0.5 uniform crossover), however 0.2 uniformcrossover is beaten by SANUX. The relative performanceof different crossover seems quite consistent on the testproblems, approximately from best to worst in the fol-lowing order: SANUX-Exponent ⇒ SANUX-Trapezoid⇒ SANUX-Triangle ⇒ 0.2 uniform ⇒ 0.5 uniform ⇒ 2-point crossover. Within SANUX both the trapezoid andexponential functions have outperformed the triangularfunction. This confirms our prediction.ConclusionsIn this paper, we propose two new functions, the trape-zoid and exponential functions, that can be used withinSANUX instead of the triangular function (Yang 2002)to calculate the swapping probability for a locus fromits allele distribution statistics. Both these two functionswiden the region for the GA to construct building blocksand enhance the protection to searched building blocksat the same time, hence are expected to be beneficial forthe performance of GAs.The experiment results show that SANUX with thetrapezoid and exponential functions performs betterthan SANUX with the triangular function and tradi-tional two-point and uniform crossover (parameterizedor not) on the test problems. Our experiment resultsindicate that the exponential function is a good choiceto be used in SANUX for its performance and simplic-ity, and that SANUX with the exponential calculatingfunction is a good candidate crossover operator for GAs.ReferencesDe Jong, K. A. 1975. An Analysis of the Behavior ofa Class of Genetic Adaptive Systems. PhD Thesis,University of Michigan, Ann Abor.Eiben, A. E.; Hinterding, R.; and Michalewicz, Z. 1999.Parameter control in evolutionary algorithms. IEEETrans. on Evolutionary Computation 3(2):124-141.01020304050600 100 200 300 400 500 600Average Best FitnessEvaluations ( x 100 )2-Point0.5 Uniform0.2 UniformSANUX-TriangleSANUX-TrapezoidSANUX-Exponent0204060801001200 100 200 300 400 500 600Average Best FitnessEvaluations ( x 100 )2-Point0.5 Uniform0.2 UniformSANUX-TriangleSANUX-TrapezoidSANUX-ExponentFigure 5: Average best-so-far curves for GAs with differ-ent crossover operators on (Top) R1 and (Bottom) R2.Eshelman, L. J.; Caruana, R. A.; and Schaffer J. D.1989. Biases in the crossover landscape. In Schaffer,D., ed., Proc. of the 3rd Int. Conf. on Genetic Algo-rithms, 10-19. Morgan Kaufmann.Forrest, S., and Mitchell, M. 1993. Relative building-block fitness and the building-block hypothesis. InWhitley, D., ed., Foundations of Genetic Algorithms2. Morgan Kaufmann.Goldberg, D. E. 1989. Genetic Algorithms in Search,Optimization, and Machine Learning. Reading, MA:Addison-Wesley,Holland, J. H. 1975. Adaptation in Natural and Artifi-cial Systems. Ann Arbor, MI: University of MichiganPress.Spears, W. M., and De Jong, K. A. 1991. On the virtuesof parameterized uniform crossover. In Belew, R. K.,and Booker, L. B., eds., Proc. 4th Int. Conf. on Ge-netic Algorithms, 230-236. Morgan Kaufmann.Syswerda, G. 1989. Uniform crossover in genetic al-gorithms. In Schaffer, J. D., ed., Proc. of the 3rd Int.Conf. on Genetic Algorithms, 2-9. Morgan Kaufmann.Yang, S. 2002. Adaptive non-uniform crossover basedon statistics for genetic algorithms. In Langdon, W.B., et al, eds., Proc. of the Genetic and Evolution-ary Computation Conference, GECCO 2002, 650-657.in Artificial Life VIII, Standish, Abbass, Bedau (eds)(MIT Press) 2002. pp 182–185 5Morgan Kaufmann.

An Analysis of the Behavior of a Class of Genetic Adaptive Systems.

Parameter control in evolutionary algorithms.

Adaptive crossover in genetic algorithms using statistics mechanism

process and maintain a population of potential  solutions to a given problem. Through the population,  GAs implicitly maintain the statistics about the  search space. This implicit statistics can be used explicitly  to enhance GA&apos;s performance. Inspired by this  idea, a statistics-based adaptive non-uniform crossover  (SANUX) has been proposed. SANUX uses the statistics  information of the alleles in each locus to adaptively  calculate the swapping probability of that locus  for crossover operation. A simple triangular function has  been used to calculate the swapping probability. In this  paper two new functions, the trapezoid and exponential  functions, are proposed for SANUX instead of the  triangular function. Experiment results show that both  functions further improve the performance of SANUX

Shengxiang Yang

CiteSeerX

Adaptive Crossover in Genetic Algorithms Using Statistics Mechanism

Genetic Algorithms (GAs) emulate the natural evolution process and maintain a population of potential solutions to a given problem. Through the population, GAs implicitly maintain the statistics about the search space. This implicit statistics can be used explicitly to enhance GA's performance. Inspired by this idea, a statistics-based adaptive non-uniform crossover (SANUX) has been proposed. SANUX uses the statistics information of the alleles in each locus to adaptively calculate the swapping probability of that locus for crossover operation. A simple triangular function has been used to calculate the swapping probability. In this paper new functions, the trapezoid and exponential functions, are proposed for SANUX instead of the triangular function. Experiment results show that both functions further improve the performance of SANUX

Yang, Shengxiang

De Montfort University Open Research Archive

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.9.3876

Adaptive crossover in genetic algorithms using statistics mechanism

Abstract

Similar works

Full text

Available Versions

Brunel University Research Archive

CiteSeerX

De Montfort University Open Research Archive