This paper presents some multidimensional optimization algorithms. By using the "penalty function" method, these algorithms are used to solving an entire class of economic optimization problems. Comparative numerical results of certain new multidimensional optimization algorithms for solving some test problems known on literature are shown.optimization algorithm, multidimensional optimization, penalty function

Iulian MIRCEA

Radu R. SERBAN

Radu SERBAN

English

Research Papers in Economics

Informatica Economică, nr. 1 (41)/2007  63
Multidimensional Optimization Algorithms Numerical Results 
 
Radu SERBAN, Mircea IULIAN, Radu R. SERBAN 
Academy of Economic Studies, Bucharest 
 
This paper presents some multidimensional optimization algorithms. By using the “penalty 
function” method, these algorithms are used to solving an entire class of economic optimiza-
tion problems. Comparative numerical results of certain new multidimensional optimization 
algorithms for solving some test problems known on literature are shown. 
Keywords: optimization algorithm, multidimensional optimization, penalty function. 
 
ntroduction 
Le
mode
t us consider a nonlinear optimization 
l under general form: 
() min ,   with constraints fx ⎧ ⎪
⎨
, : , 1,2,...,
n
j
() 01 2 j g x ,   j , ,...,m ≤= ⎪ ⎩
 (1) 
where  f gj m ℜ→ ℜ =  are 
continuous real functions of   real variables.  n
If the constraints are equalities and functions 
f  and   are derivable, then 
we can use the well – known method of La-
grange multipliers.  
, 1,2,..., j gj m =
For more general cases, the solving of an op-
timization problem with constraints as ine-
qualities, as it is the case of problem (1), can 
be reduced to solving a sequence of optimi-
zation problems of an unconstrained func-
tion. 
An interesting method to solve problem (1), 
subjecting the efficiency function  f  and 
constraints   to certain 
conditions, is the SUMT method. The meth-
ods cover many applications, known also un-
der the name of penalty methods. The first 
results are due to Fiacco and McCormick [2]. 
, 1,2,..., j gj m =
F
The idea of the method consists in transform-
ing the constrained optimization problem (1) 
into an unconstrained optimization problem 
of a function   (“aggregate” function). 
Function F is constructed so that the effi-
ciency function  f  to be penalized (in the op-
timum sense) by the fact that any point 
 does not satisfy the constraints 
 Denoting by 
n x∈ℜ
, 1,2, . gj m = ... j M the fea-
sible solutions domain of the constrained 
problem (1), 
() { } m j x g x M j
n ,...., 2 , 1 , 0 / = ≤ ℜ ∈ =  
The optimization problem (1) can be written: 
(PR) ( ) { } M x x f ∈ / min , called the “con-
strained problem” . The problem (PR) is re-
placed with a sequence of minimization 
problems of an unconstrained function (pen-
alty function), depending on a parameter: 
(PFR)t    () ( ) () x p
t
x f t x F
1
, + = . Let: 
( ) ( ) t t x F α = ,  (2) be the solution to (PFR)t 
problem (obviously, an unconstrained opti-
mization problem, called also multidimen-
sional optimization problem). 
The penalty function “p” is chosen so that the 
solution to problem (2) to converge to solu-
tion of problem (1) (PR), when parameter   
tends to zero, namely: 
t
() ( ) t x F t
t , lim
*
0 =
→ α  
In literature, [2],[5],  several forms of the 
penalty function p are known. The most used 
ones are:  (3) 
or  (4) 
() () ( [
2
1
0 , max ∑
=
=
m
j
j x g x p ) ]
() () () [] ∑
=
− =
m
j
j x g x p
1
ln
By using, for instance (3), the unconstrained 
multidimensional optimization problem,   
(PFR)t becomes:  () ( ) () () []
2
1
0 , max
1
, ∑
=
+ =
m
j
j x g
t
x f t x F  
(5). According to those mentioned above, we 
may state that the multidimensional optimi-
zation (optimization of a real function of n 
real variables) is an efficient tool to solving 
an entire class of nonlinear economic optimi-
zation problems.  
[1],[3],[4],[5] are some of the most known 
algorithms of the class of multidimensional 
optimization algorithms which do not employ 
the information given by gradient: 
•Cyclic optimization algorithm on coordi-
I Informatica Economică, nr. 1 (41)/2007  64 
nate axes (OCA); 
•Hooke and Jeeves’ algorithm (HJ); 
•Nelder and Mead’s algorithm, called also 
simplex algorithm for multidimensional op-
timization (NM);  
•Rosenbrock’s algorithm (RB). 
Two variants of the cyclic optimization algo-
rithm on coordinate axes (OCAV1 and 
OCAV2) are presented in [6]. Two variants 
of the Hooke and Jeeves’ algorithm (HJV1 
and HJV2) and a variant of the Rosenbrock’s 
algorithm (RBV1) are presented in [5] and 
[6]. 
 
Comparative numerical results 
Further on, we shall present some numerical 
results for the mentioned algorithms. They 
have been tested on two test problems known 
in literature [1], problems which set serious 
“traps” to numerical solving, namely: 
Problem 1 (Rosenbrock’s test function) 
. The 
function has the minimum point    
cu 
( 2
1 x 1
2 2
1 x 2 x 100 ) 2 x , 1 ( f min − + ⎟
⎠
⎞ ⎜
⎝
⎛ − =    x )
() 1 , 1 x = ∗
( ) 0 x f = ∗  along parabola . The 
initial point is: 
2
1 x 2 x =
() ( ) 2 . 24 x f 1 , 2 . 1 x 0 0 = − = cu       . 
Problem 2 (Witte and Holst’s test function) 
( ) ( 2
1 x 1
2
1 x 2 x 100 ) 2 x , 1 x ( f min 3 − + − =    ) . The 
function has the minimum point   
cu
() 1 , 1 x = ∗
( ) 0 x f = ∗ . The initial point 
is ( ) ( )
00 1.2, 1  with    749.037 xf x =− = . 
Problem 3 (Beale’s test function) 
3
12 1 2
1
min ( , ) (1 )
i
i
i
f xx c x x
=
⎡ ⎤ =− − ⎣ ⎦ ∑ , where: 
12 3 1.5 ; 2.25 ; 2.625 cc c = ==  
The initial point   and the optimum 
point  . 
0 (0,0) x =
(3, 0.5) x
∗ =
Accuracy   1 , 0 1 = ε   and also   01 , 0 2 = ε  
were taken into consideration for all prob-
lems. 
The indicator NF represents the evaluations 
number of function  . f  
i)  The results obtained for Rosenbrock’s 
function are the following: 
Cyclic optimization algorithm on axes:   
 
Accuracy  Optimization on axes
(OCA) 
Variant 1 
(OCAV1) 
Variant 2 
(OCAV2) 
1 0.1 ε =  
 
1
2
0.981
0.903
x
x
= ⎧
⎨ = ⎩
 
NF = 1183 
1
2
0.973
0948
x
x
= ⎧
⎨ = ⎩
 
NF = 1415 
1
2
0.964
0.988
x
x
= ⎧
⎨ = ⎩
 
NF = 1502 
2 0.01 ε =  
 
NF > 5 000 
1
2
0.990
0.992
x
x
= ⎧
⎨ = ⎩
 
NF = 3725 
1
2
0.994
0.996
x
x
= ⎧
⎨ = ⎩
 
NF = 4253 
 
It was obtained for Hooke and Jeeves’ algorithm and suggested variants: 
 
Accuracy  Hooke and Jeeves  HJV1  HJV2 
1 0.1 ε =  
 
1
2
0.989
0.901
x
x
= ⎧
⎨ = ⎩
 
NF = 114 
1
2
0.977
0.899
x
x
= ⎧
⎨ = ⎩
 
NF = 111 
1
2
0.979
0.908
x
x
= ⎧
⎨ = ⎩
 
NF = 124 
2 0.01 ε =  
 
1
2
1.042
1.099
x
x
= ⎧
⎨ = ⎩
 
NF =139 
1
2
0.999
1.009
x
x
= ⎧
⎨ = ⎩
 
NF = 137 
1
2
0.991
1.018
x
x
= ⎧
⎨ = ⎩
 
NF = 151 
 
It was obtained for Nelder and Mead algorithm and suggested variant: Informatica Economică, nr. 1 (41)/2007  65
 
Accuracy 
Nelder and Mead VNM  (NM) 
1 0.1 ε =  
 
1
2
0.988
0.902
x
x
= ⎧
⎨ = ⎩
 
NF = 131 
1
2
0.991
0.911
x
x
= ⎧
⎨ = ⎩
 
NF = 119 
2 0.01 ε =  
 
1
2
1.012
1.009
x
x
= ⎧
⎨ = ⎩
 
NF =219 
1
2
9.998
1.010
x
x
= ⎧
⎨ = ⎩
 
NF = 200 
 
It was obtained for Rosenbrock’s algorithm and suggested variant: 
 
Accuracy 
Rosenbrock  RBV1  (RB) 
1 0.1 ε =  
 
1
2
0.975
0.913
349
x
x
NF
=
=
=
 
1
2
0.983
0.951
x
x
=
=
 
NF =299 
2 0.01 ε =  
 
1
2
0.987
0.989
x
x
=
=
 
NF =395 
1
2
0.998
0.993
x
x
=
=
 
NF = 355 
 
ii) The results obtained for Witte and Holst’s problem (problem 2) 
Cyclic optimization algorithm on axes 
 
Accuracy 
 
Optimization on axes
(OCA) 
Variant 1 
(OCAV1) 
Variant 2 
(OCAV2) 
 
1
2
0.918
0.902
x
x
= ⎧
⎨ = ⎩
 
NF = 40 
1
2
0.939
0.927
x
x
= ⎧
⎨ = ⎩
 
NF = 58 
1
2
0.949
0.988
x
x
= ⎧
⎨ = ⎩
 
1 0.1 ε =
NF = 76 
2 0.01 ε =  
 
1
2
0.998
0.991
x
x
= ⎧
⎨ = ⎩
 
NF = 140 
1
2
0.991
0.994
x
x
= ⎧
⎨ = ⎩
 
NF = 183 
1
2
0.999
1.009
x
x
= ⎧
⎨ = ⎩
 
NF = 198 
 
Hooke and Jeeves’ algorithm 
 
Accuracy 
 
HJ HJV1  HJV2 
 
1
2
0.901
0.972
x
x
= ⎧
⎨ = ⎩
 
NF = 312 
1
2
0.912
0.968
x
x
= ⎧
⎨ = ⎩
 
NF = 310 
1
2
0.929
0.970
x
x
= ⎧
⎨ = ⎩
 
1 0.1 ε =
NF = 328 
2 0.01 ε =  
 
1
2
0.997
1.007
x
x
= ⎧
⎨ = ⎩
 
NF = 446 
1
2
0.995
0.995
x
x
= ⎧
⎨ = ⎩
 
NF = 426 
1
2
0.991
1.009
x
x
= ⎧
⎨ = ⎩
 
NF = 450 
 
 
 
 Informatica Economică, nr. 1 (41)/2007  66 
Nelder – Mead’s algorithm 
 
Accurracy 
 
NM VNM 
 
1
2
1.099
0.897
x
x
= ⎧
⎨ = ⎩
 
NF = 1545 
1
2
1.094
0.899
x
x
= ⎧
⎨ = ⎩
 
1 0.1 ε =
NF =1600 
2 0.01 ε =  
 
1
2
1.008
0.991
x
x
= ⎧
⎨ = ⎩
 
NF = 2176 
1
2
1.000
0.999
x
x
= ⎧
⎨ = ⎩
 
NF = 2202 
 
 
Accurracy 
 
Rosenbrock 
(RB)  RBV1 
 
1
2
1.089
0.899
x
x
=
=
 
NF = 196 
1
2
1.099
0.911
x
x
=
=
 
1 0.1 ε =
NF =191 
2 0.01 ε =  
 
1
2
1.009
0.992
x
x
=
=
 
NF = 2264 
1
2
1.007
0.998
x
x
=
=
 
NF = 245 
Further on, we present the results obtained using Rosenbrock’s algorithm for Beale’s test 
function. 
 
Accurracy 
 
Rosenbrock 
(RB)  RBV1 
 
1
2
2.900
0.599
x
x
=
=
 
NF = 60 
1
2
2.931
0.580
x
x
=
=
 
1 0.1 ε =
NF = 60 
2 0.01 ε =  
 
1
2
2.989
0.509
x
x
=
=
 
NF = 101 
1
2
2.993
0.506
x
x
=
=
 
NF = 97 
 
From an experimental point of view, we may 
state that the suggested variants are stable 
and generally use a smaller number of func-
tion evaluations to get the required accuracy. 
 
References 
[1]  Dumitru, V., (1975) - Programare 
neliniara, Appendix P., in cooperation with 
Fl. Luban, S. Moga, R. Serban, Ed. Aca-
demiei R.S.R. , Bucharest; 
[2]  Fiacco, A.V., McCormick (1968) - 
Nonlinear Programming: Sequential Uncon-
strained Minimization Technique, John 
Wiley, N.Y; 
[3] Karmanov, V.G. (1975) - Matematisces-
coe programmirovanie, Izd. Nauka, 
Moscova; 
[4] Panteleev, A.V., Lemova (2002) - T.A., 
Metodi optimizatii v primerah i zadaciah, Ed. 
Visaia Scola, Moskova; 
[5] Serban, R. (1999) - Optimizare cu apli-
catii in economie, Ed. MATRIXROM, Bu-
charest; 
[6]  Serban, R.R. (2005) - Optimizarea 
deciziei firmei, Doctor’s degree thesis, ASE 
Bucharest. 

Multidimensional optimization algorithms numerical results

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics