P. Balan

Directory of Open Access Journals

STUDIA UNIV. BABES »{BOLYAI, INFORMATICA, Volume LIV, Number 1, 2009A METHOD WITH RANDOM MODIFICATION OFGRADIENT COMPONENTS FOR CONVEX MODELSPAVEL B¸ ALANAbstract. A stochastic method is proposed and analyzed, that is a prob-abilistic generalization of gradient method, for solving convex models withrestrictions. A random change of "old" partial derivatives with "new" onesis performed from one iteration to another. Convergence aspects of thismethod are analyzed for the case when the step is adjusted programmati-cally. Certain conditions are indicated, that ensure its convergence to theoptimal solution with probability 1.1. IntroductionCurrent method has an iterative approach of \Connection-Disconnection"type. Using two random variables a series of indices is generated for nextiteration. One is meant to be used for target function and the other one -for function that describes the restriction. Distribution laws that can be usedcan be a priori set or can be modi¯ed within iterations. Usage of di®erentdistribution laws may increase convergence speed of the method. Generatedindices suggest what components of movement vector should be modi¯ed.\Old" components are replaced with corresponding partial derivatives withthe same indices from the above mentioned series. The idea of \Connection-Disconnection" mechanism is following: if the restriction inequality is satis¯edReceived by the editors: December 19, 2008.2000 Mathematics Subject Classi¯cation. 65K05, 90C15.1998 CR Categories and Descriptors. G.1.6 [Optimization]: Subtopic { Convex pro-gramming; Subtopic { Nonlinear programming; Subtopic { Stochastic programming.Key words and phrases. stochastic method, di®erentiable optimization, probabilistic gen-eralization of gradient method, convex model, partial derivatives, movement direction, con-vergence, with probability 1, almost sure, connect-disconnect type.2526 PAVEL B¸ ALANat the current iteration, then the vector that determines the movement direc-tion is built using partial derivatives of target function; otherwise this vectorcontains only partial derivatives of the function from restriction.2. A method with random modification of gradient componentsfor convex modelsThe following problem is considered:(1)8> <> :F(x) ¡! min'(x) · 0x 2 Xwhere X represents a compact and convex set in Euclidian space En.Suppose that problem (1) is solvable.Let us de¯ne V (X;") = [x2XV (x;") - vicinity with " radius of X set. ByV (x;") is marked the vicinity with radius " > 0 of the point x 2 En, or,formally:V (x;") = fy 2 En : kx ¡ yk < "gLet's admit for some " > 0 that F(x) and '(x) are convex and di®eren-tiable functions (with continuous gradients) on V (X;"). Therefore, for 8x 2 Xare de¯ned the following vectors:(gF;1(x);:::;gF;n(x)) = gF(x) = gradF(x) =µdF(x)dx1;:::;dF(x)dxn¶(g';1(x);:::;g';n(x)) = g'(x) = grad'(x) =µd'(x)dx1;:::;d'(x)dxn¶Obviously, the norms kgF(x)k =v u utn Xi=1g2F;i(x), kg'(x)k =v u utn Xi=1g2';i(x) arecontinuous functions on X compact and, consequently, a constant C exists, sothat kgF(x)k · C, kg'(x)k · C for 8x. Hence kgF;i(x)k · C, kg';i(x)k · C,8i = 1;n, 8x 2 X.The numerical method that is proposed to solve the problem (1) has aniterative approach. Assuming that we are positioned on kth iteration theschema is following:MODIFICATION OF GRADIENT COMPONENTS FOR CONVEX MODELS 27Step 1. Two random variables »k, Ãk are simulated in series mk ¸ 1, lk ¸ 1 ofindependent probes with discrete distribution laws a priori de¯ned:»k 1 2 ::: nP Pk»;1 Pk»;2 ::: Pk»;nÃk 1 2 ::: nP PkÃ;1 PkÃ;2 ::: PkÃ;nThat is, on every iteration k the sets Ik = fi1;i2;:::;imkg, Jk =fj1;j2;:::;jlkg of elements that are independent realizations of »k, Ãkvariables with distribution laws de¯ned above are generated, where(2)Pk»;i ¸ P» > 0;8i = 1;n;8k = 0;1;:::PkÃ;i ¸ PÃ > 0;8i = 1;n;8k = 0;1;:::Particularly, we can take mk = lk = 1, that is a single simulation isaccomplished for every random variable »k, Ãk on every iteration.Step 2. gkF(xk), gk'(xk) vectors are calculated according to the rule:(3)gkF(xk) =³gkF;1;:::;gkF;i;:::;gkF;n´; gkF;i ="gk¡1F;i ; if i = 2 IkdF(x)dxi ; if i 2 Ikgk'(xk) =³gk';1;:::;gk';i;:::;gk';n´; gk';i ="gk¡1';i ; if i = 2 Jkd'(x)dxi ; if i 2 Jk8i = 1;nStep 3. The element xk+1 is determined according to relation:(4) xk+1 =YX³~ xk+1´; where ~ xk+1 = xk ¡ ½k´kQX¡~ xk+1¢represents the projection of element ~ xk+1 on the set X.Starting point x0 is arbitrary taken from X (it can be indicated undercertain considerations for some concrete situations).Step 4. The numerical sequence©´kªis de¯ned in following way:(5) ´k =½ gkkgkk; if gk 6= ¹ 0; 8k = 1;2;:::¹ 0; for gk = ¹ 0g0 is considered to be an arbitrary, but bounded vector.Necessarily, classical requirements are imposed on sequence f½kg to ensurethe convergence from probabilistic point of view of the iterative process (4)28 PAVEL B¸ ALANwhich have the form:(6) ½k ¸ 0; ½k ¡!k!10;1 Xk=0½k = 1Additionally we will require existence of such a number ¹ " > 0, that for8r 2 (0;¹ "] and 8¿ 2 (0;1) the convergence of the series occurs [1]:(7)1 Xk=0¿L(k;r) < 1; L(k;r) =½0; if ½k ¸ r or k = 0sk; ifk Pl=k¡sk½l < r andk Pl=k¡sk¡1½l ¸ rIn other words sk is the biggest integer number among all numbers j ¸ 0that satis¯es the relationk Pl=k¡j½l.Remark 1. Particularly, it is easy to show that numerical sequence ½k =R(k+1)®, R > 0, ® 2 (0;1] satis¯es the conditions (6)-(7).B.T. Polyak proposes a schema to solve general convex models [4]. Thisis a deterministic "connect-disconnect" schema where the vector gk de¯nesmovement direction and is formulated in following way:gk = gk(xk) =½gradF¡xk¢; if '¡xk¢· 0grad'(xk); if '(xk) > 0But actual method proposes another representation of movement vector gk:(8) gk = gk(xk) =½gkF¡xk¢; if '¡xk¢· ¿kgk' (xk); if '(xk) > ¿k; ¿k > 0Remark 2. The iterative process can be modi¯ed in following way: di®erentdistribution laws can be taken for random variables »k, Ãk, from one iterationto another, with the condition that relation (2) holds. This can favour theincrease of convergence speed of the sequence©xkª.Applicability of described method can be con¯rmed, ¯rst of all, by estab-lishing convergence, in probabilistic terms, of sequence©xkªtowards optimaldomain of solutions X¤. A special interest represents the convergence withprobability 1 (only this type of convergence can be accepted with con¯dencefrom applied point of view).MODIFICATION OF GRADIENT COMPONENTS FOR CONVEX MODELS 29Theorem 1. For ¯xed 8" > 0, all elements of random sequence©xkªk¸0,obtained as a result of application of described method, are localized almostcertain (with probability 1) in vicinity V (X¤;2"), but excepting a ¯nite numberof elements. Formally this can be represented in the following way:P½µ : limk!1minx¤2X¤° °xk ¡ x¤° ° = 0¾= 1,where xk = xk (µ) and µ =¡µ0;µ1;:::;µk;:::¢,µk =¡i0;i1;:::;ik¢2 Bk- ¾- algebra generated byCartesian product ((I0 £ J0) £ (I1 £ J1) £ ::: £ (Ik £ Jk)).Proof. If X ½ V (X¤;2") then the statement is obvious.Let's admit Xn V (X¤;2") 6= ?. A problem of the form:(9)8> > <> > :F (x) ¡! min'(x) · ¿k;¿k > 0; ¿k ! 0;1 Pk=0½k¿k = 1;¿k½k ! 1x 2 Xis associated to initial model (1) on every iteration k.Two stages for proof development will be accentuated.Stage 1. Firstly the existence of a subsequence©xklª½©xkªk¸0, that almostcertain is contained in VX (X¤;") will be demonstrated, i.e.Pn9©xklª½©xkªk¸0 : xkl 2 VX (X¤;")o= 1.Let's suppose the contrary. In this case for some q 2 (0;1) a naturalnumber Kq can be indicated such that the following event is produced(10)A1 =n9Kq : 8k ¸ Kq;°° °xk ¡ x¤°° ° ¸ "; or xk = 2 VX (X¤;"); 8x¤ 2 X¤owith probability P (A1) ¸ q.Let's denote X" = X n V (X¤;").Since F (x), '(x) are convex and di®erentiable, the following in-equalities are valid:F (x¤) ¡ F¡xk¢¸µdF(xk)dx ;x¤ ¡ xk¶;'(x¤) ¡ '¡xk¢¸µd'(xk)dx ;x¤ ¡ xk¶for 8x¤ 2 X¤, 8xk 2 X.30 PAVEL B¸ ALANLet us denote ¢F = minx2X";x¤2X¤ [F (x) ¡ F (x¤)]. Evidently, ¢F > 0,if " > 0.Taking into consideration all properties enumerated above, two con-stants C1 > 0, C2 > 0 may be chosen, such that kx0 ¡ x00k · C1,8x0;x00 2 X and°° °dF(x)dx°° ° · C2, 8x 2 X.If '¡xk¢· ¿k and xk 2 X", then F¡xk¢¡ F (x¤) ¸ ¢F, or(11)ÃdF¡xk¢dx;xk ¡ x¤!¸ ¢FµdF(xk)dx ;xk ¡ x¤¶° ° °°dF(xk)dx° ° °° ¢ kxk ¡ x¤k¸µdF(xk)dx ;xk ¡ x¤¶C2C1¸¢FC1C2Also, if '¡xk¢> ¿k for xk 2 X", then:(12)Ãd'¡xk¢dx;xk ¡ x¤!¸ ¿kµd'(xk)dx ;xk ¡ x¤¶° °° °d'(xk)dx° °° ° ¢ kxk ¡ x¤k¸µd'(xk)dx ;xk ¡ x¤¶C2C1¸¿kC1C2Let's consider some numbers ±F, ±k' from intervals³0; ¢FC1C2´,³0;¿kC1C2´and label ¹ ±k = min©±F;±k'ª. As a result the following inequalities maybe obtained:(13)µdF(xk)dx ;xk ¡ x¤¶¸ 2¹ ±k° ° °°dF(xk)dx° ° °° ¢° °xk ¡ x¤° °, if '¡xk¢· ¿kµd'(xk)dx ;xk ¡ x¤¶¸ 2¹ ±k°° ° °d'(xk)dx°° ° ° ¢° °xk ¡ x¤° °, if '¡xk¢> ¿kParticularly, ±F, ±k' may be chosen as centres of intervals³0; ¢FC1C2´,³0;¿kC1C2´:(14) ±F =¢F2(C1C2);±k' =¿k2(C1C2)The following events are being consideredMODIFICATION OF GRADIENT COMPONENTS FOR CONVEX MODELS 31Ak1 =©¡´k;xk ¡ x¤¢¸ ¹ ±k°°xk ¡ x¤°°;8x¤ 2 X¤ª. Obviously, theopposite event with regards to Ak1 has the following formAk1 =©9x¤ 2 X¤ :¡´k;xk ¡ x¤¢< ¹ ±k° °xk ¡ x¤° °ª.Then D1 =½1[k=K±1\i=kAi1¾, or, in other words, occurs all Ak1(k ¸Kq), without, perhaps, a ¯nite number. It is obvious that D1 = ½1\k=K±1[i=kAi1¾, or, in other words, an in¯nite number of events Ak1are produced.Let us evaluate P (A1). In order to do this let's representP (A1) = P (A1 \ D1) + P¡A1 \ D1¢Both terms from last expression will be estimated.From the realization of event A1 \ D1 follows the existence of sucha natural number K± < 1 that for all k ¸ K± and 8x¤ 2 X¤ followinginequality occurs(15)³´k;xk ¡ x¤´¸ ¹ ±k° ° °xk ¡ x¤° ° °Taking into consideration (15), for k ¸ K± we have the followingsequence of relations: ° °xk+1 ¡ x¤° °2 ·° °xk ¡ ½k´k ¡ x¤° °2 ==°°xk ¡ x¤°°2 ¡ 2½k¡xk ¡ x¤;´k¢+ ½2k°°´k°°2 ··° °xk ¡ x¤° °2 ¡ 2½k¹ ±k° °xk ¡ x¤° ° + ½2k ··°°xk ¡ x¤°°2 ¡ 2½k¹ ±k" + ½2k ==° °xk ¡ x¤° °2 ¡ ½k¡2¹ ±k" ¡ ½k¢Because ½k ¡!k!10, for some K': ±F > ±k' or ¹ ±k = ±k', as soon ask ¸ K'. According to (9), (14) for some K" ¸ K': ½k · ¹ ±k", as soonas k ¸ K". Evidently, for k ¸ ^ k = maxfK±;K"g:° °°xk+1 ¡ x¤° °°2·° °°xk ¡ x¤° °°2¡ ½k¹ ±k";32 PAVEL B¸ ALAN° °°xk ¡ x¤° °°2·° °°xk¡1 ¡ x¤° °°2¡ ½k¡1¹ ±k¡1" ··°° °xk¡2 ¡ x¤°° °2¡ "¡½k¡2¹ ±k¡2 + ½k¡1¹ ±k¡1¢;:::° ° °xk+1 ¡ x¤° ° °2·° ° °x^ k ¡ x¤° ° °2¡ "k Xi=^ k½i¹ ±i or° ° °xk+1 ¡ x¤° ° °2··° ° °x^ k ¡ x¤° ° °2¡ "k Xi=^ k½i±i'Due to imposed conditions on ¿k in (9), based on relation (14), we get:(16)° ° °xk+1 ¡ x¤° ° °2·° ° °x^ k ¡ x¤° ° °2¡"2(C1C2)k Xi=^ k½i¿i ! 1; for k ! 1We obtain a contradiction because the norm of any vector, moreoverits square value, cannot be negative. Therefore, the realization of eventA1\D1 implies realization of an event, that is practically unrealizable,F1 =n°°xk+1 ¡ x¤°°2 < 0;k ¡! 1o. That is P (A1 \ D1) · P (F1) =0. It means that P (A1) = P¡A1 \ D1¢.Let us evaluate P¡A1 \ D1¢. The following event Bk1 is de¯ned:BkF =fat least one time among iterations of the form j = k ¡ sk;kis generated every possible value of the discrete random variable »kg.Bk' =fat least one time among iterations of the form j = k ¡ sk;kis generated every possible value of the discrete random variable Ãkg.Bk1 = BkF \ Bk'.Simulation of variables »k and Ãk is produced in parallel and inde-pendently. Thanks to the fact that BkF, Bk' events are independent,follows that P¡Bk1¢= P¡BkF¢¢ P¡Bk'¢.Let us prove that P¡Bk1¢¡!k!11. Contrary is supposed: P¡Bk1¢·p < 1, for 8k. We have P³BkF´= 1 ¡ P¡BkF¢and P³Bk'´= 1 ¡P¡Bk'¢.MODIFICATION OF GRADIENT COMPONENTS FOR CONVEX MODELS 33It is absolutely clear that P³BkF´·n Pi=1³1 ¡ Pk»;i´sk¡!k!10, P³Bk'´·n Pi=1³1 ¡ PkÃ;i´sk¡!k!10. Indeed, if we label P = minnP»;PÃo, the fol-lowing sequence of relations takes place:(17)P³BkF´·n Pi=1³1 ¡ Pk»;i´sk· n ¢ max1·i·n³1 ¡ Pk»;i´sk··³sk pn³1 ¡ P»´´sk· (sk pn(1 ¡ P))sk ;P³Bk'´·n Pi=1³1 ¡ PkÃ;i´sk· n ¢ max1·i·n³1 ¡ PkÃ;i´sk··³sk pn³1 ¡ PÃ´´sk· (sk pn(1 ¡ P))skBecause k ! 1, then sk ! 1 andsk pn ! 1+0. For an arbitrary, but¯xed value ¿ 2 (1 ¡ P;1): 9K¿ 2 N, so that for k ¸ K¿ takes placeinequalitysk pn(1 ¡ P) · ¿ < 1. Thus, (sk pn(1 ¡ P))sk · ¿sk ! 0.Or P³BkF´· ¿sk and P³Bk'´· ¿sk. It means that P³BkF´! 0 andP³Bk'´! 0, or, P¡BkF¢! 1 and P¡Bk'¢! 1 for k ! 1. Theseconsiderations conclude to the fact that P¡Bk1¢¡!k!11.The realization of event Bk1 means following: "renovation" of allcomponents of the vectors gk¡skF and gk¡sk' is performed during skiterations starting from k ¡ sk till k (inclusively). In other words,movement vector gk contains as its components all partial derivatives,all evaluated after iteration k ¡ sk.The realization of event Bk1 and the fact that partial derivatives offunctions F (x), '(x) are continuous, conclude to realization of follow-ing event:(18)8> > <> > :8e e " > 0 :°° ° °gi ¡@F(xk)@x°° ° ° · e e "; if '¡xk¢· ¿k° °° °gi ¡@'(xk)@x° °° ° · e e "; if '¡xk¢> ¿k;8i = k ¡ sk;k9> > => > ;Taking into consideration (18), continuity of dot product and satis-faction of conditions (10), (13), we can draw a conclusion that eventAk1 is realized starting with some k = b b k = maxnK¿;^ ko.34 PAVEL B¸ ALANThus, Bk1 ½ Ak1. In this case P¡Bk1¢· P¡Ak1¢and, thereforeP³Ak1´· P³Bk1´. But, according to (2), (7) and (17) follows:P³Bk1´· ¿L(k;r); that is1 Xk=0P³Ak1´·1 Xk=0P³Bk1´·1 X¿L(k;r) < 1k=0We are in such situation that the conditions of the Borel-Cantellilemma [3] are satis¯ed. It means that P¡D1¢= 0. Therefore, q ·P (A1) = P¡A1 \ D1¢· P¡D1¢= 0. Thus, q = 0.A contradiction has been obtained, because we have supposed thatq > 0. Thus, it exists a subsequence©xklª½©xkªk¸0 that almostcertainly is contained in VX (X¤;").Stage 2. Further will be proved that all elements of sequence©xkª, without justa ¯nite number, belong to set VX (X¤;2") with probability 1.Following events are de¯ned:(19)A2 =©9©xklª½©xkª:©xklª½ VX (X¤;")ªB2 =©9©zkmª½©xkª:©zkmª* VX (X¤;2")ªNext, P (B2) will be appreciated. We will ¯nd out that P (B2) =P (B2 \ A2). Indeed,P (B2) = P¡(B2 \ A2) [¡B2 \ A2¢¢= P (B2 \ A2)+P¡B2 \ A2¢=P (B2 \ A2), because P¡B2 \ A2¢· P¡A2¢= 0.Further, following event will be considered: D2 = A2\B2. Supposethat P (D2) > 0. Realization of event D2 means that the transfer fromVX (X¤;") to X n VX (X¤;2") and vice versa takes place in¯nitely.Let us denote by:K1 - the ¯rst iteration the event©xK1 2 VX (X¤;")ªis produced,K2 - the ¯rst iteration the event©xK2 2 VX¡X¤; 32"¢ªis produced,K3 - the ¯rst iteration theinequality ½K3 · 2"¹ ±k is satis¯ed,¹ K = maxfK1;K2;K3g.In case that for some k ¸ ¹ K and xk = 2 VX¡X¤; 32"¢is satis¯edinequality that de¯nes event Ak1, then following sequence of inequalitiesoccurs: ° °xk+1 ¡ x¤° °2 ·° °xk ¡ x¤° °2¡½k¡2"¹ ±k ¡ ½k¢<° °xk ¡ x¤° °2 ; because °°xk ¡ x¤°° > ".MODIFICATION OF GRADIENT COMPONENTS FOR CONVEX MODELS 35That is, as soon as k ¸ ¹ K and xk = 2 VX¡X¤; 32"¢means that:(20)° °°xk+1 ¡ x¤° °° <° °°xk ¡ x¤° °°Since ½k ¡!k!10, will appear K¤ ¸ ¹ K with property that xK¤2VX (X¤;2") n VX¡X¤; 32"¢. This will happen certainly. Particularly,for ½k < "2:° ° °xk+1 ¡ xk° ° ° ·° ° °xk ¡ ½k´k ¡ xk° ° ° · ½k <"2Therefore, there exists a value k that satis¯es xk 2 VX (X¤;2") nVX¡X¤; 32"¢.According to (20),° °xK¤+1 ¡ x¤° ° <° °xK¤¡ x¤° °. In the case thatxK¤+1 = 2 VX¡X¤; 32"¢, then we have°°xK¤+2 ¡ x¤°° <°°xK¤+1 ¡ x¤°° <<° °xK¤¡ x¤° °, and so forth, for all j ¸ 0 that satisfy xK¤+j = 2VX¡X¤; 32"¢, takes place(21) minx¤2X¤° ° °xK¤+j ¡ x¤° ° ° < minx¤2X¤° ° °xK¤¡ x¤° ° ° < 2"Let us denotenxklol¸1- sequence of all elements©xkªwith theproperty that kl ¸ K¤;xkl2 VX (X¤;2") n VX¡X¤; 32"¢and xkl¡1 2VX¡X¤; 32"¢. Then for l ¸ 1;kl < j < kl+1 and xj = 2 VX¡X¤; 32"¢thefollowing inequality occurs:(22) minx¤2X¤° °xj ¡ x¤° ° < minx¤2X¤° °°xkl¡ x¤° °° < 2"Thus, in other words, admitting that for some K elements of typexk = 2 VX¡X¤; 32"¢;k < 1;k ¸ K satisfy inequality from event Ak1,then event B2 cannot occur with positive probability. Suppositionthat D2 is realized means that beyond layer VX¡X¤; 32"¢penetration oflayer XnVX (X¤;2") takes place only when in¯nitely is produced eventAk1 considered previously. But P¡D1¢= 0. So, the conclusion thatcan be drawn is that the transfer from layer VX (X¤;2")nVX¡X¤; 32"¢into layer X n VX (X¤;2") occurs only a ¯nite number of times. Thatis, P (D2) = 0, and implies P (B2) = 0.Theorem is proved.36 PAVEL B¸ ALAN3. ConclusionsElaborated method is especially practical for models where modi¯cationof gradients is \relatively slow". Such models are often encountered in eco-nomical, technical problems etc. It represents a signi¯cant generalization ofmethods meant to solve extremum problems. It can be classi¯ed as a di-rect method of optimization and does not use penalty functions or Lagrangefunction { common toolkit used to solve such kind of problems.References[1] Ermolyev Y.M., Gaivoronski A.A., Stochastic Methods for Solving Minimax Problems,Kibernetika, Naukova Dumka, Kiev, 1983[2] Moiseev N.N, Ivanilov Y.P., Stolyarova E.M., Methods of optimization, Nauka, Moskow,1978[3] Shiryaev A.V., Probability, Nauka, Moskow, 1980[4] Polyak B.T., A general method for solving extreme problems, Dokl. Akad. Nauk SSSR.174, No.1, p.33-36, 1967Moldova State University, Chisinau, Republic of MoldovaE-mail address: nalab.levap@gmail.com

A Method with Random Modification of Gradient Components for Convex Models

Abstract

Similar works

Full text

Available Versions

Directory of Open Access Journals