This paper presents a new simulated annealing algorithm to solve
constrained multi-global optimization problems. To compute all
global solutions in a sequential manner, we combine the function
stretching technique with the adaptive simulated annealing variant.
Constraint-handling is carried out through a nondifferentiable
penalty function. To benchmark our penalty stretched simulated
annealing algorithm we solve a set of well-known problems. Our
preliminary numerical results show that the algorithm is promising.This work has been partially supported by the FCT (Fundagao para a Ciencia e a Tecnologia), Portugal

Ana I. Pereira

Ch. Tsitouras

Edite M. G. P. Fernandes

George Psihoyios

Theodore E. Simos

Crossref

Constrained Multi-global Optimization using a Penalty 
Stretched Simulated Annealing Framework 
Ana I. Pereira* and Edite M.G.P. Fernandes^ 
* Department of Mathematics, Polytechnic Institute ofBraganga, Braganga, Portugal 
^ Department of Production and Systems, University of Minho, 4710-057 Braga, Portugal 
Abstract. This paper presents a new simulated annealing algorithm to solve constrained multi-global optimization problems. 
To compute all global solutions in a sequential manner, we combine the function stretching technique with the adaptive 
simulated annealing variant. Constraint-handling is carried out through a nondifferentiable penalty function. To benchmark 
our penalty stretched simulated annealing algorithm we solve a set of well-known problems. Our preliminary numerical results 
show that the algorithm is promising. 
Keywords: multi-global optimization, penalty function, simulated annealing 
PACS: 02.60.Pn 
INTRODUCTION 
The purpose of this paper is to present a penalty framework for solving constrained multi-global optimization (CMGO) 
problems in the following form 
min/(x), subject to ^/(x) < 0, j = 1,...,OT and/;c <x <Ux (1) 
X 
where at least one of the functions / , gj : R" —̂  R is nonlinear, and ^ = {x : /̂  < x < M ,̂ ̂ y (x) < 0, j = 1,.. . , m} is 
the feasible region. Since we do not assume convexity, / may possess many global minima inside .^. Here, we aim to 
find all points x* G ^ such that fix*) < fix) for all x G ^ . We also assume that the problem (1) has a finite number 
of global minimizers. This class of global optimization problems is very important and frequently encountered in 
engineering applications (e.g. [2, 3,4]). Some algorithms for solving this type of problem require substantial gradient 
information and aim to improve the solution in a neighborhood of a given initial approximation. When the problem 
has more than one global solution, the probabihty of convergence to an already detected global solution is very high, 
and depends very closely on the provided initial approximation. 
The most well-known category of methods to handle constraints in nonlinear optimization problems depends on a 
penalty function and a positive penalty parameter. Techniques based on penalty functions transform the constrained 
problem into a sequence of unconstrained problem by penahzing / when constraints are violated and then minimizing 
the penalty function using methods for unconstrained problems. It is expected that the penalty parameter is updated 
along the iterative process, so that convergence to the solution can be accelerated. The choice of the initial penalty and 
its updating formula are issues not well-defined until now. Large values give feasible solutions that have low accuracy, 
since search around the boundary tends to be avoided, while small values generate infeasible with good accuracy 
solutions. 
The remainder of this paper is organized as follows. In the next section, we briefly introduce the penalty function 
that is used to penalize infeasible iterates. Then, we describe the implementation of the function stretching technique 
coupled with the simulated annealing based algorithm so that all global solutions can be computed in a sequential 
manner. The last sections contain the results of aU the numerical experiments and the conclusions. 
CPl 168, Vol. 2, Numerical Analysis and Applied Mathematics, International Conference 2009 
edited by T. E. Simos, G. Psihoyios, and Ch. Tsitouras 
O 2009 American Institute of Physics 978-0-7354-0708-4/09/$25.00 
1354 
Downloaded 15 Sep 2009 to 193.137.16.114. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/proceedings/cpcr.jsp
CONSTRAINT-HANDLING BY A PENALTY TECHNIQUE 
The basic penalty approach defines a fitness for each point x, herein denoted by ^{x\jj.), by adding to the objective 
function value a penalty term that aims to penalize infeasible solutions as follows: 
m 
(^(X;M) = / ( x ) + M £(max{0,^,-W})'^*^'^'"' (2) 
i= i 
where ji is the positive penalty parameter. The power of the constraint violation, 7(.), is a violation dependent constant: 
Y{Z) = 1 if Z < 0.1, and /(z) = 2, otherwise. See, for example, in [7, 8]. We remark that only the inequahty constraints 
are used in the penahzed terms. It follows that the minimizer of the function ^, in (2), that satisfies h <x <Ux for 
a fixed value of the parameter ji, herein denoted by x*{jj.), converges to the solution of the given problem (1) as ji 
increases [1]. An appropriate updating scheme for ji is as follows: 
^k+i = max{TjU,fc,jUmax}, for T > 1 andjUmax > > 1 
for a given Ho > 0, where k represents the iteration counter. 
PENALTY STRETCHED SIMULATED ANNEALING METHOD 
Here, we use the following notation: Â  is the number of global solutions of problem (1), X* = [xJ,X2, • • • ,xiv] is the 
nxN matrix whose columns contain the global solutions. In general, each implementation of a global optimization 
method finds just one global solution. To be able to compute multiple solutions, deflation techniques have to be 
incorporated in the algorithm. Our proposal rehes on a sequential simulated annealing (S A) algorithm in the sense that 
a sequence of global optimization problems is iteratively defined and each problem is solved by the SA algorithm, 
a global point-to-point stochastic algorithm. After the computation of a global solution, the objective function is 
transformed using a function stretching technique. 
>From now on and to simphfy the notation, we use ^(x) instead of ^(x;jU). The function stretching technique was 
initially proposed in [9], in a particle swarm optimization algorithm context, to provide a way to escape from a local 
solution, driving the search to a global one. When a local (non-global) solution, x, is found, the technique stretches the 
function ^ by augmenting (by a certain amount) the objective function at all points x that satisfy ^ (x) > ^ (x), leaving 
^(x) unchanged for all x such that ^(x) < ^(x). The minimization process is then repeated with the new stretched 
objective function to look for the global solution. If another local solution is found, the previously mentioned process 
is repeated. Each global optimization problem of the sequence is solved by ASA, a well-known variant of the simulated 
annealing algorithm [5]. 
In our case, the function stretching technique aims to prevent the convergence of the SA algorithm to a previously 
found solution. Thus, the function stretching technique is apphed only locally, in order to transform ^(x) in a 
neighborhood of X;*, say Ve{xi), e > 0. So, ^(x) is augmented only on the region Ve{xi) leaving all the other minima 
unchanged. The minimum ^ (x;*) disappears but all other minima are left unchanged. Each global optimization problem 
of the sequence is solved by ASA (adaptive simulated annealing), a well-known variant of the simulated annealing 
algorithm [5]. The multi-global procedure terminates when for a predefined set of consecutive iterations no other 
solution is encountered [11, 12]. 
This sequential simulated annealing algorithm solves a sequence of global optimization problems whose objective 
function is obtained by applying a function stretching technique, as outlined in [9, 10], to the objective function of the 
previous problem in the sequence. The mathematical formulation of the algorithm together with the transformations 
that are carried out are the following: 
where ^ (x) is defined as 
and 
mm 0/ (x)= < .) ( , ^ ' (3) 
ix<x<uj, [ YW otherwise 
'^("^ = '^("^+ 2tanh(K(^-(x)-^(x;)) "̂̂  
"̂(x) = (̂ (x) + | | |x-x; | | [sgn((^(x)-(^(xr)) + l] (5) 
1355 
Downloaded 15 Sep 2009 to 193.137.16.114. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/proceedings/cpcr.jsp
with 5i, &2 and K positive constants and "sgn" defines the well-known sign function. Transformations (5) and (4) 
stretch the neighborhood of X;*, with ray e', upwards assigning larger function values to those points to prevent 
the convergence of the global optimization method to that computed solution [12]. The proposed penalty stretched 
simulated annealing (PSSA) algorithm for CMGO is presented below: 
Algorithm 1 (PSSA algorithm) 
Input: Ho, Umax » I, T > I. Setk = 0. 
while outer stopping criteria not met do 
Set do, EQ, Emax, i*̂  = 0 and I = 0. 
while inner stopping criteria not met do 
Set j = 0 and 1 = 1+1 
Compute x*i (pt) = argmin/^<^<„^ $/(x;^t) 
while $/ {x*i (Hk)) - ^max <8oor EJ > e^ax do 
Set j = j+l and EJ = JEQ 
Fori= I,. ..,2m randomly generate Xi G Vej{x*i); find^max = inax{0/(xi),..., 0/(x2m)} 
end while 
Let L*̂  = L 
end while 
1 and e 
l^k+l =max{TjU,fc,jUniax} 
SetX* ^ X*{nj,)andk = k+l 
end while 
In the PSSA algorithm, the conditions to stop the outer iterative process are: \\X*{pk) -X*{pk_i)\\ < £^orfe>feniax-
The inner iterative process terminates if L*̂  does not change for a specified ^ iterations. 
NUMERICAL RESULTS 
The herein proposed multi-global optimization method based on a penalty technique for constraint-handling was 
implemented in the C programming language on a Pentium II, Celeron 466 Mhz with 64Mb of RAM. To evaluate 
the performance of the herein proposed penalty stretched simulated annealing algorithm for constrained multi-global 
optimization problems a set of six benchmark problems, described in full detail in the Appendix of [13], is used. In 
this preliminary study, small dimensional problems (n < 10 and m < 13) with a nonlinear objective function, simple 
bounds and inequality constraints were tested. They are known in the literature as g04, g06, g08, g09, gl2 and gl8. 
The values for the user defined parameters are: T = 10, jUmax = 10*, jJo = 10, 5o = 10^^, eo = 0.15, Emax = 5, 
femax = 10 and Ex = 10^^, and ^ = 5. Each problem was solved by PSSA five times with randomly generated initial 
approximations. 
Details of the selected problems are listed in the Table 1, where P refers to the problem number, "type of / " 
describes the type of objective function, fgiobai is the known global minimum, n is the number of variables, m is 
the number of inequahty constraints, fpssA is our best solution obtained after the five runs, fpssA is the average of 
the obtained solutions over the five runs, NPSSA is the average number of iterations reuired by the penalty stretched 
simulated annealing method, N/̂ va/ is the average number of function evaluations and N/ oc gives the average of the 
obtained global/local solutions. 
TABLE 1. Numerical results 
type of/ J global fi PSSA fpSSA NpssA N feval ^lo 
S04 
g06 
g08 
g09 
Sl2 
gl8 
quadratic 
cubic 
general 
general 
quadratic 
quadratic 
-3.0665E + 04 
-6.9618E + 03 
-9 .5825E-02 
6.8063E + 02 
l.OOOOE + 00 
-8.6603E-01 
5 
2 
2 
7 
3 
9 
6 
2 
2 
4 
1 
13 
-2.5874E + 04 
-6.9419E + 03 
-9 .5825E-02 
6.8870E + 02 
l.OOOOE + 00 
-8.6349E-01 
-2.5874E + 04 
-6.9419E + 03 
-8 .3552E-02 
6.8875E + 02 
9.8562E-01 
-8.5987E-01 
43 
9 
24 
6 
24 
10 
156154 
27550 
79771 
309719 
202219 
945000 
12 
1 
5 
1 
1 
5 
1356 
Downloaded 15 Sep 2009 to 193.137.16.114. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/proceedings/cpcr.jsp
CONCLUSIONS 
We have incorporated a penalty framework into a sequential simulated annealing algorithm that uses a function 
stretching technique, to be able to compute sequentially all the global solutions of inequality constrained multi-modal 
objective based optimization problems. 
The preliminary numerical tests with a set of small problems are encouraging. Further experiments, in particular 
with problems with a large number of local solutions, and with practical engineering applications are required. To 
improve solution accuracy, the use of a local search to refine the obtained solution was mandatory. 
Future challenge is the use of other penalty functions and the extension of this type of multi-global method to 
equality constrained problems. 
ACKNOWLEDGMENTS 
This work has been partially supported by the FCT (Fundagao para a Ciencia e a Tecnologia), Portugal. 
REFERENCES 
1. D. p. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods, Academic Press, 1982. 
2. A. Bonilla-Petriciolet, R. Vdzquez-Romdn, G. A. Iglesias-Silva, K. R. Hall. Performance of Stochastic Global Optimization 
Methods in the Calculation of Phase Analyses for Nonreactive and Reactive Mixtures. Industrial Engineering Chemistry 
Research, 45, 4764-4772 (2006). 
3. C. M. McDonald, C. A. Floudas, Global optimization for the phase stabiUty problem, AIChE. /., 41, 1798-1814 (1995). 
4. C. Floudas, Recent advances in global optimization for process synthesis, design and control: enclosure all solutions. 
Computers and Chemical Engineering 963, 963-973 (1999). 
5. L. Ingber, Very fast simulated re-annealing. Mathematical and Computer Modelling, 12, 967-973 (1989). 
6. T. Le6n, S. Sanmatias and F. Vercher, A multilocal optimization algorithm, TOP 6 (1998), 1-18. 
7. J.-L. Liu and J.-H. Lin, Fvolutionary computation of unconstrained and constrained problems using a novel momentum-type 
particle swarm optimization. Engineering Optimization, 39(3), 287- 305, 2007. 
8. Y.G. Petalas, K.F. Parsopoulos and M.N. Vrahatis, Memetic particle swarm optimization. Annals of Operations Research, 
156,99-127,2007. 
9. K. Parsopoulos, V. Plagianakos, G. Magoulas, and M. Vrahatis, Objective function stretching to alleviate convergence to local 
minima. Nonlinear Analysis 47, 3419-3424 (2001). 
10. K. Parsopoulos, and M. Vrahatis, Recent approaches to global optimization problems through particle swarm optimazition. 
Natural Computing 1, 235-306 (2002). 
11. A. I. P. N. Pereira and F. M. G. P. Fernandes, "On a reduction line search filter method for nonlinear semi-infinite 
programming problems", in Euro Mini Conference "Continuous Optimization and Knowledge-Based Technologies", edited by 
L. Sakalauskas, G.W. Weber, F.K. Zavadskas, ISBN: 978-9955-28-283-9, 2008, pp. 174-179. 
12. A. I. P. N. Pereira and F. M. G. P. Fernandes, A reduction method for semi-infinite programming by means of a global 
stochastic approach. Optimization, 58, 713-726 (2009). 
13. A. F. M. Zavala, A. H. Aguirre, and F. R. V. Diharce. Constrained optimization via particle evolutionary swarm optimization 
algorithm (PFSO). In GECCO'05, 209-216, 2005. 
1357 
Downloaded 15 Sep 2009 to 193.137.16.114. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/proceedings/cpcr.jsp


English

Constrained Multi-global Optimization using a Penalty Stretched Simulated Annealing Framework

Pereira, Ana I. P. N.

Fernandes, Edite Manuela da G. P.

Universidade do Minho: RepositoriUM

Constrained Multi-global Optimization using a Penalty Stretched Simulated Annealing Framework Ana I. Pereira* and Edite M.G.P. Fernandes^ * Department of Mathematics, Polytechnic Institute ofBraganga, Braganga, Portugal ^ Department of Production and Systems, University of Minho, 4710-057 Braga, Portugal Abstract. This paper presents a new simulated annealing algorithm to solve constrained multi-global optimization problems. To compute all global solutions in a sequential manner, we combine the function stretching technique with the adaptive simulated annealing variant. Constraint-handling is carried out through a nondifferentiable penalty function. To benchmark our penalty stretched simulated annealing algorithm we solve a set of well-known problems. Our preliminary numerical results show that the algorithm is promising. Keywords: multi-global optimization, penalty function, simulated annealing PACS: 02.60.Pn INTRODUCTION The purpose of this paper is to present a penalty framework for solving constrained multi-global optimization (CMGO) problems in the following form min/(x), subject to ^/(x) < 0, j = 1,...,OT and/;c <x <Ux (1) X where at least one of the functions / , gj : R" —^  R is nonlinear, and ^ = {x : /^  < x < M ,^ ^ y (x) < 0, j = 1,.. . , m} is the feasible region. Since we do not assume convexity, / may possess many global minima inside .^. Here, we aim to find all points x* G ^ such that fix*) < fix) for all x G ^ . We also assume that the problem (1) has a finite number of global minimizers. This class of global optimization problems is very important and frequently encountered in engineering applications (e.g. [2, 3,4]). Some algorithms for solving this type of problem require substantial gradient information and aim to improve the solution in a neighborhood of a given initial approximation. When the problem has more than one global solution, the probabihty of convergence to an already detected global solution is very high, and depends very closely on the provided initial approximation. The most well-known category of methods to handle constraints in nonlinear optimization problems depends on a penalty function and a positive penalty parameter. Techniques based on penalty functions transform the constrained problem into a sequence of unconstrained problem by penahzing / when constraints are violated and then minimizing the penalty function using methods for unconstrained problems. It is expected that the penalty parameter is updated along the iterative process, so that convergence to the solution can be accelerated. The choice of the initial penalty and its updating formula are issues not well-defined until now. Large values give feasible solutions that have low accuracy, since search around the boundary tends to be avoided, while small values generate infeasible with good accuracy solutions. The remainder of this paper is organized as follows. In the next section, we briefly introduce the penalty function that is used to penalize infeasible iterates. Then, we describe the implementation of the function stretching technique coupled with the simulated annealing based algorithm so that all global solutions can be computed in a sequential manner. The last sections contain the results of aU the numerical experiments and the conclusions. CPl 168, Vol. 2, Numerical Analysis and Applied Mathematics, International Conference 2009 edited by T. E. Simos, G. Psihoyios, and Ch. Tsitouras O 2009 American Institute of Physics 978-0-7354-0708-4/09/$25.00 1354 Downloaded 15 Sep 2009 to 193.137.16.114. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/proceedings/cpcr.jspCONSTRAINT-HANDLING BY A PENALTY TECHNIQUE The basic penalty approach defines a fitness for each point x, herein denoted by ^{x\jj.), by adding to the objective function value a penalty term that aims to penalize infeasible solutions as follows: m (^(X;M) = / ( x ) + M £(max{0,^,-W})'^*^'^'"' (2) i= i where ji is the positive penalty parameter. The power of the constraint violation, 7(.), is a violation dependent constant: Y{Z) = 1 if Z < 0.1, and /(z) = 2, otherwise. See, for example, in [7, 8]. We remark that only the inequahty constraints are used in the penahzed terms. It follows that the minimizer of the function ^, in (2), that satisfies h <x <Ux for a fixed value of the parameter ji, herein denoted by x*{jj.), converges to the solution of the given problem (1) as ji increases [1]. An appropriate updating scheme for ji is as follows: ^k+i = max{TjU,fc,jUmax}, for T > 1 andjUmax > > 1 for a given Ho > 0, where k represents the iteration counter. PENALTY STRETCHED SIMULATED ANNEALING METHOD Here, we use the following notation: A^  is the number of global solutions of problem (1), X* = [xJ,X2, • • • ,xiv] is the nxN matrix whose columns contain the global solutions. In general, each implementation of a global optimization method finds just one global solution. To be able to compute multiple solutions, deflation techniques have to be incorporated in the algorithm. Our proposal rehes on a sequential simulated annealing (S A) algorithm in the sense that a sequence of global optimization problems is iteratively defined and each problem is solved by the SA algorithm, a global point-to-point stochastic algorithm. After the computation of a global solution, the objective function is transformed using a function stretching technique. >From now on and to simphfy the notation, we use ^(x) instead of ^(x;jU). The function stretching technique was initially proposed in [9], in a particle swarm optimization algorithm context, to provide a way to escape from a local solution, driving the search to a global one. When a local (non-global) solution, x, is found, the technique stretches the function ^ by augmenting (by a certain amount) the objective function at all points x that satisfy ^ (x) > ^ (x), leaving ^(x) unchanged for all x such that ^(x) < ^(x). The minimization process is then repeated with the new stretched objective function to look for the global solution. If another local solution is found, the previously mentioned process is repeated. Each global optimization problem of the sequence is solved by ASA, a well-known variant of the simulated annealing algorithm [5]. In our case, the function stretching technique aims to prevent the convergence of the SA algorithm to a previously found solution. Thus, the function stretching technique is apphed only locally, in order to transform ^(x) in a neighborhood of X;*, say Ve{xi), e > 0. So, ^(x) is augmented only on the region Ve{xi) leaving all the other minima unchanged. The minimum ^ (x;*) disappears but all other minima are left unchanged. Each global optimization problem of the sequence is solved by ASA (adaptive simulated annealing), a well-known variant of the simulated annealing algorithm [5]. The multi-global procedure terminates when for a predefined set of consecutive iterations no other solution is encountered [11, 12]. This sequential simulated annealing algorithm solves a sequence of global optimization problems whose objective function is obtained by applying a function stretching technique, as outlined in [9, 10], to the objective function of the previous problem in the sequence. The mathematical formulation of the algorithm together with the transformations that are carried out are the following: where ^ (x) is defined as and mm 0/ (x)= < .) ( , ^ ' (3) ix<x<uj, [ YW otherwise '^("^ = '^("^+ 2tanh(K(^-(x)-^(x;)) "^^  "^(x) = (^ (x) + | | |x-x; | | [sgn((^(x)-(^(xr)) + l] (5) 1355 Downloaded 15 Sep 2009 to 193.137.16.114. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/proceedings/cpcr.jspwith 5i, &2 and K positive constants and "sgn" defines the well-known sign function. Transformations (5) and (4) stretch the neighborhood of X;*, with ray e', upwards assigning larger function values to those points to prevent the convergence of the global optimization method to that computed solution [12]. The proposed penalty stretched simulated annealing (PSSA) algorithm for CMGO is presented below: Algorithm 1 (PSSA algorithm) Input: Ho, Umax » I, T > I. Setk = 0. while outer stopping criteria not met do Set do, EQ, Emax, i*^  = 0 and I = 0. while inner stopping criteria not met do Set j = 0 and 1 = 1+1 Compute x*i (pt) = argmin/^<^<„^ $/(x;^t) while $/ {x*i (Hk)) - ^max <8oor EJ > e^ax do Set j = j+l and EJ = JEQ Fori= I,. ..,2m randomly generate Xi G Vej{x*i); find^max = inax{0/(xi),..., 0/(x2m)} end while Let L*^  = L end while 1 and e l^k+l =max{TjU,fc,jUniax} SetX* ^ X*{nj,)andk = k+l end while In the PSSA algorithm, the conditions to stop the outer iterative process are: \\X*{pk) -X*{pk_i)\\ < £^orfe>feniax-The inner iterative process terminates if L*^  does not change for a specified ^ iterations. NUMERICAL RESULTS The herein proposed multi-global optimization method based on a penalty technique for constraint-handling was implemented in the C programming language on a Pentium II, Celeron 466 Mhz with 64Mb of RAM. To evaluate the performance of the herein proposed penalty stretched simulated annealing algorithm for constrained multi-global optimization problems a set of six benchmark problems, described in full detail in the Appendix of [13], is used. In this preliminary study, small dimensional problems (n < 10 and m < 13) with a nonlinear objective function, simple bounds and inequality constraints were tested. They are known in the literature as g04, g06, g08, g09, gl2 and gl8. The values for the user defined parameters are: T = 10, jUmax = 10*, jJo = 10, 5o = 10^^, eo = 0.15, Emax = 5, femax = 10 and Ex = 10^^, and ^ = 5. Each problem was solved by PSSA five times with randomly generated initial approximations. Details of the selected problems are listed in the Table 1, where P refers to the problem number, "type of / " describes the type of objective function, fgiobai is the known global minimum, n is the number of variables, m is the number of inequahty constraints, fpssA is our best solution obtained after the five runs, fpssA is the average of the obtained solutions over the five runs, NPSSA is the average number of iterations reuired by the penalty stretched simulated annealing method, N/^ va/ is the average number of function evaluations and N/ oc gives the average of the obtained global/local solutions. TABLE 1. Numerical results type of/ J global fi PSSA fpSSA NpssA N feval ^lo S04 g06 g08 g09 Sl2 gl8 quadratic cubic general general quadratic quadratic -3.0665E + 04 -6.9618E + 03 -9 .5825E-02 6.8063E + 02 l.OOOOE + 00 -8.6603E-01 5 2 2 7 3 9 6 2 2 4 1 13 -2.5874E + 04 -6.9419E + 03 -9 .5825E-02 6.8870E + 02 l.OOOOE + 00 -8.6349E-01 -2.5874E + 04 -6.9419E + 03 -8 .3552E-02 6.8875E + 02 9.8562E-01 -8.5987E-01 43 9 24 6 24 10 156154 27550 79771 309719 202219 945000 12 1 5 1 1 5 1356 Downloaded 15 Sep 2009 to 193.137.16.114. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/proceedings/cpcr.jspCONCLUSIONS We have incorporated a penalty framework into a sequential simulated annealing algorithm that uses a function stretching technique, to be able to compute sequentially all the global solutions of inequality constrained multi-modal objective based optimization problems. The preliminary numerical tests with a set of small problems are encouraging. Further experiments, in particular with problems with a large number of local solutions, and with practical engineering applications are required. To improve solution accuracy, the use of a local search to refine the obtained solution was mandatory. Future challenge is the use of other penalty functions and the extension of this type of multi-global method to equality constrained problems. ACKNOWLEDGMENTS This work has been partially supported by the FCT (Fundagao para a Ciencia e a Tecnologia), Portugal. REFERENCES 1. D. p. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods, Academic Press, 1982. 2. A. Bonilla-Petriciolet, R. Vdzquez-Romdn, G. A. Iglesias-Silva, K. R. Hall. Performance of Stochastic Global Optimization Methods in the Calculation of Phase Analyses for Nonreactive and Reactive Mixtures. Industrial Engineering Chemistry Research, 45, 4764-4772 (2006). 3. C. M. McDonald, C. A. Floudas, Global optimization for the phase stabiUty problem, AIChE. /., 41, 1798-1814 (1995). 4. C. Floudas, Recent advances in global optimization for process synthesis, design and control: enclosure all solutions. Computers and Chemical Engineering 963, 963-973 (1999). 5. L. Ingber, Very fast simulated re-annealing. Mathematical and Computer Modelling, 12, 967-973 (1989). 6. T. Le6n, S. Sanmatias and F. Vercher, A multilocal optimization algorithm, TOP 6 (1998), 1-18. 7. J.-L. Liu and J.-H. Lin, Fvolutionary computation of unconstrained and constrained problems using a novel momentum-type particle swarm optimization. Engineering Optimization, 39(3), 287- 305, 2007. 8. Y.G. Petalas, K.F. Parsopoulos and M.N. Vrahatis, Memetic particle swarm optimization. Annals of Operations Research, 156,99-127,2007. 9. K. Parsopoulos, V. Plagianakos, G. Magoulas, and M. Vrahatis, Objective function stretching to alleviate convergence to local minima. Nonlinear Analysis 47, 3419-3424 (2001). 10. K. Parsopoulos, and M. Vrahatis, Recent approaches to global optimization problems through particle swarm optimazition. Natural Computing 1, 235-306 (2002). 11. A. I. P. N. Pereira and F. M. G. P. Fernandes, "On a reduction line search filter method for nonlinear semi-infinite programming problems", in Euro Mini Conference "Continuous Optimization and Knowledge-Based Technologies", edited by L. Sakalauskas, G.W. Weber, F.K. Zavadskas, ISBN: 978-9955-28-283-9, 2008, pp. 174-179. 12. A. I. P. N. Pereira and F. M. G. P. Fernandes, A reduction method for semi-infinite programming by means of a global stochastic approach. Optimization, 58, 713-726 (2009). 13. A. F. M. Zavala, A. H. Aguirre, and F. R. V. Diharce. Constrained optimization via particle evolutionary swarm optimization algorithm (PFSO). In GECCO'05, 209-216, 2005. 1357 Downloaded 15 Sep 2009 to 193.137.16.114. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/proceedings/cpcr.jsp

Constrained multi-global optimization using a penalty stretched simulated annealing framework

Constrained Multi-global Optimization using a Penalty Stretched Simulated Annealing Framework Ana I. Pereira* and Edite M.G.P. Fernandes^ * Department of Mathematics, Polytechnic Institute ofBraganga, Braganga, Portugal ^ Department of Production and Systems, University of Minho, 4710-057 Braga, Portugal Abstract. This paper presents a new simulated annealing algorithm to solve constrained multi-global optimization problems. To compute all global solutions in a sequential manner, we combine the function stretching technique with the adaptive simulated annealing variant. Constraint-handling is carried out through a nondifferentiable penalty function. To benchmark our penalty stretched simulated annealing algorithm we solve a set of well-known problems. Our preliminary numerical results show that the algorithm is promising. Keywords: multi-global optimization, penalty function, simulated annealing PACS: 02.60.Pn INTRODUCTION The purpose of this paper is to present a penalty framework for solving constrained multi-global optimization (CMGO) problems in the following form min/(x), subject to ^/(x) < 0, j = 1,...,OT and/;c <x <Ux (1) X where at least one of the functions / , gj : R" —̂  R is nonlinear, and ^ = {x : /̂  < x < M ,̂ ̂ y (x) < 0, j = 1,.. . , m} is the feasible region. Since we do not assume convexity, / may possess many global minima inside .^. Here, we aim to find all points x* G ^ such that fix*) < fix) for all x G ^ . We also assume that the problem (1) has a finite number of global minimizers. This class of global optimization problems is very important and frequently encountered in engineering applications (e.g. [2, 3,4]). Some algorithms for solving this type of problem require substantial gradient information and aim to improve the solution in a neighborhood of a given initial approximation. When the problem has more than one global solution, the probabihty of convergence to an already detected global solution is very high, and depends very closely on the provided initial approximation. The most well-known category of methods to handle constraints in nonlinear optimization problems depends on a penalty function and a positive penalty parameter. Techniques based on penalty functions transform the constrained problem into a sequence of unconstrained problem by penahzing / when constraints are violated and then minimizing the penalty function using methods for unconstrained problems. It is expected that the penalty parameter is updated along the iterative process, so that convergence to the solution can be accelerated. The choice of the initial penalty and its updating formula are issues not well-defined until now. Large values give feasible solutions that have low accuracy, since search around the boundary tends to be avoided, while small values generate infeasible with good accuracy solutions. The remainder of this paper is organized as follows. In the next section, we briefly introduce the penalty function that is used to penalize infeasible iterates. Then, we describe the implementation of the function stretching technique coupled with the simulated annealing based algorithm so that all global solutions can be computed in a sequential manner. The last sections contain the results of aU the numerical experiments and the conclusions. CPl 168, Vol. 2, Numerical Analysis and Applied Mathematics, International Conference 2009 edited by T. E. Simos, G. Psihoyios, and Ch. Tsitouras O 2009 American Institute of Physics 978-0-7354-0708-4/09/$25.00 1354 Downloaded 15 Sep 2009 to 193.137.16.114. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/proceedings/cpcr.jspCONSTRAINT-HANDLING BY A PENALTY TECHNIQUE The basic penalty approach defines a fitness for each point x, herein denoted by ^{x\jj.), by adding to the objective function value a penalty term that aims to penalize infeasible solutions as follows: m (^(X;M) = / ( x ) + M £(max{0,^,-W})'^*^'^'"' (2) i= i where ji is the positive penalty parameter. The power of the constraint violation, 7(.), is a violation dependent constant: Y{Z) = 1 if Z < 0.1, and /(z) = 2, otherwise. See, for example, in [7, 8]. We remark that only the inequahty constraints are used in the penahzed terms. It follows that the minimizer of the function ^, in (2), that satisfies h <x <Ux for a fixed value of the parameter ji, herein denoted by x*{jj.), converges to the solution of the given problem (1) as ji increases [1]. An appropriate updating scheme for ji is as follows: ^k+i = max{TjU,fc,jUmax}, for T > 1 andjUmax > > 1 for a given Ho > 0, where k represents the iteration counter. PENALTY STRETCHED SIMULATED ANNEALING METHOD Here, we use the following notation: Â  is the number of global solutions of problem (1), X* = [xJ,X2, • • • ,xiv] is the nxN matrix whose columns contain the global solutions. In general, each implementation of a global optimization method finds just one global solution. To be able to compute multiple solutions, deflation techniques have to be incorporated in the algorithm. Our proposal rehes on a sequential simulated annealing (S A) algorithm in the sense that a sequence of global optimization problems is iteratively defined and each problem is solved by the SA algorithm, a global point-to-point stochastic algorithm. After the computation of a global solution, the objective function is transformed using a function stretching technique. >From now on and to simphfy the notation, we use ^(x) instead of ^(x;jU). The function stretching technique was initially proposed in [9], in a particle swarm optimization algorithm context, to provide a way to escape from a local solution, driving the search to a global one. When a local (non-global) solution, x, is found, the technique stretches the function ^ by augmenting (by a certain amount) the objective function at all points x that satisfy ^ (x) > ^ (x), leaving ^(x) unchanged for all x such that ^(x) < ^(x). The minimization process is then repeated with the new stretched objective function to look for the global solution. If another local solution is found, the previously mentioned process is repeated. Each global optimization problem of the sequence is solved by ASA, a well-known variant of the simulated annealing algorithm [5]. In our case, the function stretching technique aims to prevent the convergence of the SA algorithm to a previously found solution. Thus, the function stretching technique is apphed only locally, in order to transform ^(x) in a neighborhood of X;*, say Ve{xi), e > 0. So, ^(x) is augmented only on the region Ve{xi) leaving all the other minima unchanged. The minimum ^ (x;*) disappears but all other minima are left unchanged. Each global optimization problem of the sequence is solved by ASA (adaptive simulated annealing), a well-known variant of the simulated annealing algorithm [5]. The multi-global procedure terminates when for a predefined set of consecutive iterations no other solution is encountered [11, 12]. This sequential simulated annealing algorithm solves a sequence of global optimization problems whose objective function is obtained by applying a function stretching technique, as outlined in [9, 10], to the objective function of the previous problem in the sequence. The mathematical formulation of the algorithm together with the transformations that are carried out are the following: where ^ (x) is defined as and mm 0/ (x)= < .) ( , ^ ' (3) ix<x<uj, [ YW otherwise '^("^ = '^("^+ 2tanh(K(^-(x)-^(x;)) "̂̂  "̂(x) = (̂ (x) + | | |x-x; | | [sgn((^(x)-(^(xr)) + l] (5) 1355 Downloaded 15 Sep 2009 to 193.137.16.114. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/proceedings/cpcr.jspwith 5i, &2 and K positive constants and "sgn" defines the well-known sign function. Transformations (5) and (4) stretch the neighborhood of X;*, with ray e', upwards assigning larger function values to those points to prevent the convergence of the global optimization method to that computed solution [12]. The proposed penalty stretched simulated annealing (PSSA) algorithm for CMGO is presented below: Algorithm 1 (PSSA algorithm) Input: Ho, Umax » I, T > I. Setk = 0. while outer stopping criteria not met do Set do, EQ, Emax, i*̂  = 0 and I = 0. while inner stopping criteria not met do Set j = 0 and 1 = 1+1 Compute x*i (pt) = argmin/^<^<„^ $/(x;^t) while $/ {x*i (Hk)) - ^max <8oor EJ > e^ax do Set j = j+l and EJ = JEQ Fori= I,. ..,2m randomly generate Xi G Vej{x*i); find^max = inax{0/(xi),..., 0/(x2m)} end while Let L*̂  = L end while 1 and e l^k+l =max{TjU,fc,jUniax} SetX* ^ X*{nj,)andk = k+l end while In the PSSA algorithm, the conditions to stop the outer iterative process are: \\X*{pk) -X*{pk_i)\\ < £^orfe>feniax-The inner iterative process terminates if L*̂  does not change for a specified ^ iterations. NUMERICAL RESULTS The herein proposed multi-global optimization method based on a penalty technique for constraint-handling was implemented in the C programming language on a Pentium II, Celeron 466 Mhz with 64Mb of RAM. To evaluate the performance of the herein proposed penalty stretched simulated annealing algorithm for constrained multi-global optimization problems a set of six benchmark problems, described in full detail in the Appendix of [13], is used. In this preliminary study, small dimensional problems (n < 10 and m < 13) with a nonlinear objective function, simple bounds and inequality constraints were tested. They are known in the literature as g04, g06, g08, g09, gl2 and gl8. The values for the user defined parameters are: T = 10, jUmax = 10*, jJo = 10, 5o = 10^^, eo = 0.15, Emax = 5, femax = 10 and Ex = 10^^, and ^ = 5. Each problem was solved by PSSA five times with randomly generated initial approximations. Details of the selected problems are listed in the Table 1, where P refers to the problem number, "type of / " describes the type of objective function, fgiobai is the known global minimum, n is the number of variables, m is the number of inequahty constraints, fpssA is our best solution obtained after the five runs, fpssA is the average of the obtained solutions over the five runs, NPSSA is the average number of iterations reuired by the penalty stretched simulated annealing method, N/̂ va/ is the average number of function evaluations and N/ oc gives the average of the obtained global/local solutions. TABLE 1. Numerical results type of/ J global fi PSSA fpSSA NpssA N feval ^lo S04 g06 g08 g09 Sl2 gl8 quadratic cubic general general quadratic quadratic -3.0665E + 04 -6.9618E + 03 -9 .5825E-02 6.8063E + 02 l.OOOOE + 00 -8.6603E-01 5 2 2 7 3 9 6 2 2 4 1 13 -2.5874E + 04 -6.9419E + 03 -9 .5825E-02 6.8870E + 02 l.OOOOE + 00 -8.6349E-01 -2.5874E + 04 -6.9419E + 03 -8 .3552E-02 6.8875E + 02 9.8562E-01 -8.5987E-01 43 9 24 6 24 10 156154 27550 79771 309719 202219 945000 12 1 5 1 1 5 1356 Downloaded 15 Sep 2009 to 193.137.16.114. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/proceedings/cpcr.jspCONCLUSIONS We have incorporated a penalty framework into a sequential simulated annealing algorithm that uses a function stretching technique, to be able to compute sequentially all the global solutions of inequality constrained multi-modal objective based optimization problems. The preliminary numerical tests with a set of small problems are encouraging. Further experiments, in particular with problems with a large number of local solutions, and with practical engineering applications are required. To improve solution accuracy, the use of a local search to refine the obtained solution was mandatory. Future challenge is the use of other penalty functions and the extension of this type of multi-global method to equality constrained problems. ACKNOWLEDGMENTS This work has been partially supported by the FCT (Fundagao para a Ciencia e a Tecnologia), Portugal. REFERENCES 1. D. p. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods, Academic Press, 1982. 2. A. Bonilla-Petriciolet, R. Vdzquez-Romdn, G. A. Iglesias-Silva, K. R. Hall. Performance of Stochastic Global Optimization Methods in the Calculation of Phase Analyses for Nonreactive and Reactive Mixtures. Industrial Engineering Chemistry Research, 45, 4764-4772 (2006). 3. C. M. McDonald, C. A. Floudas, Global optimization for the phase stabiUty problem, AIChE. /., 41, 1798-1814 (1995). 4. C. Floudas, Recent advances in global optimization for process synthesis, design and control: enclosure all solutions. Computers and Chemical Engineering 963, 963-973 (1999). 5. L. Ingber, Very fast simulated re-annealing. Mathematical and Computer Modelling, 12, 967-973 (1989). 6. T. Le6n, S. Sanmatias and F. Vercher, A multilocal optimization algorithm, TOP 6 (1998), 1-18. 7. J.-L. Liu and J.-H. Lin, Fvolutionary computation of unconstrained and constrained problems using a novel momentum-type particle swarm optimization. Engineering Optimization, 39(3), 287- 305, 2007. 8. Y.G. Petalas, K.F. Parsopoulos and M.N. Vrahatis, Memetic particle swarm optimization. Annals of Operations Research, 156,99-127,2007. 9. K. Parsopoulos, V. Plagianakos, G. Magoulas, and M. Vrahatis, Objective function stretching to alleviate convergence to local minima. Nonlinear Analysis 47, 3419-3424 (2001). 10. K. Parsopoulos, and M. Vrahatis, Recent approaches to global optimization problems through particle swarm optimazition. Natural Computing 1, 235-306 (2002). 11. A. I. P. N. Pereira and F. M. G. P. Fernandes, "On a reduction line search filter method for nonlinear semi-infinite programming problems", in Euro Mini Conference "Continuous Optimization and Knowledge-Based Technologies", edited by L. Sakalauskas, G.W. Weber, F.K. Zavadskas, ISBN: 978-9955-28-283-9, 2008, pp. 174-179. 12. A. I. P. N. Pereira and F. M. G. P. Fernandes, A reduction method for semi-infinite programming by means of a global stochastic approach. Optimization, 58, 713-726 (2009). 13. A. F. M. Zavala, A. H. Aguirre, and F. R. V. Diharce. Constrained optimization via particle evolutionary swarm optimization algorithm (PFSO). In GECCO'05, 209-216, 2005. 1357 Downloaded 15 Sep 2009 to 193.137.16.114. Redistribution subject to AIP license or copyright; see http://proceedings.aip.org/proceedings/cpcr.jsp

http://bibliotecadigital.ipb.pt/bitstream/10198/1613/1/ICNAAM_PF_09.pdf

Constrained multi-global optimization using a penalty stretched simulated annealing framework

Abstract

Similar works

Full text

Available Versions

Crossref

Crossref

Universidade do Minho: RepositoriUM

Crossref

Crossref

Crossref

Crossref

Crossref