The problem of finding a feasible solution to a linear inequality system arises in numerous contexts. In [12] an algorithm, called extended relaxation method, that solves the feasibility problem, has been proposed by the authors. Convergence of the algorithm has been proven. In this paper, we consider a class of extended relaxation methods depending on a parameter and prove their convergence. Numerical experiments have been provided, as well

González-Gutiérrez, E.

Hernández Rebollar, Lídia Aurora

Todorov, Maxim I.

Universitat Autònoma de Barcelona. Centre de Recerca Matemàtica

Centre de Recerca Matemàtica

Diposit Digital de Documents de la UAB

RELAXATION METHODS FOR SOLVING LINEARINEQUALITY SYSTEMS: CONVERGING RESULTSE. GONZA´LEZ-GUTIE´RREZ, L. HERNANDEZ REBOLLAR, AND MAXIM I. TODOROVAbstract. The problem of finding a feasible solution to a linear inequalitysystem arises in numerous contexts. In [12] an algorithm, called extendedrelaxation method, that solves the feasibility problem, has been proposed bythe authors. Convergence of the algorithm has been proven. In this paper, weconsider a class of extended relaxation methods depending on a parameter andprove their convergence. Numerical experiments have been provided, as well.1. IntroductionIn various numerical problems one is confronted with the task to find at leastof one solution of linear semi-infinite systems (LSIS’s for short) of the form:(1) σ = {a′tx ≥ bt, t ∈ T}where, T is an arbitrary nonempty index set, x ∈ Rn, a : T → Rn and b : T → Rare arbitrary mappings, and its solution set is represented by F . If not statedotherwise, given the LSIS’s, σ, we will suppose that at 6= 0n for all t ∈ T , suchthat each inequality represents a closed half-space, and F is not empty.The LSIS’s arise in many different frames, both in mathematical theory (see[10], [7], [18], [2]), and in the practice (see [13], [4]). The study of the solutionsof linear semi-infinite programming problems, started in 1920’s (see [11], [5],[6], [17], [20], [16]), requires as a first step to solve the feasibility problem, i.e.,to find an initial point, which is a solution of (1). Numerical methods lookingfor a feasible solutions of LSIS’s, with a finite index set T, has been studied in[1] and [19]. Agmon (see [1]) proposes different numerical procedures to solvethe feasibility problem and give several converging results. One of the proposedscheme is the so called over and under projection with a fixed ratio. However,in case that T is a infinite set, no numerical method for solving the feasibilityproblem is reported until Jeroslow proposed a projection method in 1979, only forT = N [15]. Later Hu, in [14], has considered the same projection method. Bothauthors, under different conditions on the nominal data, have proven convergenceof the algorithms. In a resent paper [12], we have extended the idea of theResearch partially supported by CONACyT of MX, Grant 55681 and Ministry of Scienceand Innovation of SP, Grant MTM2008-06695-C03-01.12 E. GONZA´LEZ-GUTIE´RREZ, L. HERNANDEZ REBOLLAR, AND MAXIM I. TODOROVrelaxation method when |T | < ∞. So, an algorithm, called extended relaxationmethod (ERM) has been proposed. The main differences with the two previouspapers, which represents the ERM, are that instead of the projection at eachstep of the algorithm of the current iteration, we consider, as a next point, itsreflection with respect to a certain hyperplane. And second, we have substantialdifference in the relaxation rule, i.e., how to decide when we have to make thenext step. A convergence theorem has also been obtained. In this present work,inspired of the over and under projection with a fixed ratio methods, proposedby Agmon, we consider a class of extended relaxation methods depending on aparameter λ. In Section 2, we give some preliminaries and in Section 3 we presentthe description of the methods and prove the convergence of the algorithms forλ ∈ (0, 2]. Finally, in Section 4, we give three examples and numerical resultsillustrating the behavior of the proposed algorithms.2. PreliminariesThe Euclidean distance from x¯ to the hyperplane, H = {x ∈ Rn|a′x = b}, suchthat a′x¯ < b will be denoted by d(x¯, H) = b−a′x¯‖a‖ . The geometric idea for ordinarysystems is as follows: suppose an arbitrary initial point, x¯ ∈ Rn, such that it isnot a solution of the system (i.e., x¯ /∈ F ); and we try to find the farthest halfspace(among these defined by the inequalities in the system σ), with boundary H, ofx¯; the next point arises from x¯ by going along the projection vector of x¯ onto thehyperplane H, with a distance λd(x¯, H), where λ > 0 is a prefixed parameter.If µ := d(x¯, H) > 0 is the Euclidean distance from x¯ to the hyperplane H, thenext point will be x¯+ µλ a‖a‖ , where a is the vector from the definition of H.It is clear that, for infinite systems and a fixed xr, g(t, xr) := a′txr− bt is a nonlinear function, called slack or marginal function, and the techniques involvingcalculations of global maxima, for example, Matlab routines, only provide ap-proximations. So, finding the farthest hyperplane is not an easy work, and thento avoid this problem, we propose a relaxation method, in which it is not neces-sary to calculate the farthest hyperplane. In fact, if a sequence {xk} is generated,such that, from a current point, xr, the next point is of the form xr+1 = xr+ελ a‖a‖we need only that at each step ε (together with its corresponding vector a) hasto be sufficiently close, in some sense, to µ.In particular, if the parameter λ = 2 (λ = 1), the next point is the reflection ofx¯ from the hyperplane (the projection of x¯ into H, respectively). The case λ = 1,has been widely studied in [14] and [15]. The authors of this paper, deal withsystems where T ⊂ Rm, and assume the restrictive conditions sup{‖ at ‖ |t ∈T} <∞ and inf{bt |t ∈ T} > −∞. The second paper considers only LSIS’s withT = N. In [12], we propose method for λ = 2, where we consider an arbitraryindex set T and only suppose that the feasible set F is of a full dimensionality.We can even drop this restriction, if the Strong Slater condition, i.e., there existRELAXATION METHODS 3x¯ ∈ F and γ > 0 such that a′tx¯ ≥ bt + γ for all t ∈ T, is fulfilled. Under theseweaker conditions, we have shown that our ERM algorithm converges.In fact, the algorithms in three cases, enter in the scheme of over and underprojection with a fixed ratio methods. Agmon names these methods under projec-tion when λ ∈ (0, 1), projection when λ = 1 and over projection when λ ∈ (1, 2].Further in this presentation, we shall consider a class of ERM algorithms for thewhole range of the parameter λ ∈ (0, 2].3. The ERM algorithmsA formal description of the ERM, depending on λ, is as follows:Algorithm 1. (Extended relaxation method)(1) Choose the parameters λ > 0, M > 2 and β > 0; choose an arbitraryvector x0 ∈ Rn. Set the iteration index r = 0.(2) Minimize the slack function g(t, x) at xr, finding ur = inft∈T g(t, xr). Ifur ≥ 0, stop (xr ∈ F ). Otherwise, take the index setTr = {t ∈ T |g(t, xr) < 0} (indexes of violated inequalities by xr).(3) Set βr = β and consider the global optimization problem(2) sup{bt − a′txr‖at‖ , t ∈ Tr}= µr.(4) Furthermore, find a βr approximation, εr, of the solution, µr, of the prob-lem ( 2) (µr − βr < εr ≤ µr) . If βr < εr(M − 1), thenµrM< εr :=btr − a′trxr‖atr‖≤ µr, for some tr ∈ Tr,and choose xr+1 = xr +λεratr‖atr‖ . Replace r by r+ 1 and loop to step 2. Ifnot, set βr = βr/2 and go to the step 4.Remark 1. In [12], we have shown that in the ERM algorithm for all r = 1, 2, . . . ,εr always exists and it takes finite values different from zero.The following convergence theorem holds:Theorem 2. Let us have a system, σ, and dimF = n. Given an initial point,x0 ∈ Rn, ERM with λ ∈ (0, 2] either ends after a finite number of steps, or itgenerates an infinite sequence, {xr}, converging to some element of F .Proof. Let us fix λ ∈ (0, 2] and let us consider the sequence {xr} ⊂ Rn generatedby the algorithm. We will conclude that limr→∞ xr = xˆ ∈ F .If the sequence is finite and the last point belongs to F, we are done. So, weassume that {xr} is an infinite sequence of infeasible points. For each t ∈ T wedenote Ht = {x ∈ Rn | a′tx = bt}. We have µr > 0, for all r ∈ N, i.e., xr /∈ Htr ,then the vector xr+1 is along the vector atr starting from xr and distance betweenthe two points is λεr.4 E. GONZA´LEZ-GUTIE´RREZ, L. HERNANDEZ REBOLLAR, AND MAXIM I. TODOROVBy the hypothesis there exist z ∈ Rn and δ > 0 such that the open ball Bδ(z)of centre z and radius δ satisfiesBδ(z) ⊂ F ⊂ {x ∈ Rn | a′trx ≥ btr}, r = 1, 2, . . .and ρtr :=d(z,Htr) ≥ δ.By construction, the line determined by xr and xr+1 is orthogonal to Htr . Lethr be the distance from z to that line. Consider the affine hull of {xr, xr+1, z}.We choose a coordinate system in this hyperplane, with abscises axis, the linethroughout the points xr and xr+1, directed in that way, and ordinates axis,the perpendicular to the line throughout the points xr and xr+1, directed insuch a way that z belongs to the first orthant. With this oriented system, thecoordinates of the points xr , xr+1 and z are (−εr, 0), ((λ − 1)εr, 0) = (ξεr, 0),where ξ ∈ (−1, 1], and (ρtr , hr), respectively, with hr ≥ 0 (the case when thedimension of the affine hull is 1 and hr = 0 is trivial). Then‖xr − z‖2 − ‖xr+1 − z‖2 = [(ρtr + εr)2 + h2r]− [(ρtr − ξεr)2 + h2r]= (1− ξ2)ε2r + 2(1 + ξ)ρtrεr.Hence, for r ∈ N, we have0 ≤ ‖xr+1 − z‖2 = ‖xr − z‖2 − (1− ξ2)ε2r − 2(1 + ξ)ρtrεr,since −ρtr ≤ −δ, we have0 ≤ ‖xr+1 − z‖2 ≤ ‖xr − z‖2 − (1− ξ2)ε2r − 2(1 + ξ)δεr.So, we can consider the above relation for the first r−1 terms, i.e., for every k =0, . . . , r − 1‖xk+1 − z‖2 ≤ ‖xk − z‖2 − (1− ξ2)ε2r − 2(1 + ξ)δεr.Adding orderly the above inequalities, we haver−1∑k=0‖xk+1 − z‖2 ≤r−1∑k=0(‖xk − z‖2 − (1− ξ2)ε2r − 2(1 + ξ)δεr),then,‖xr − z‖2 +r−1∑k=1‖xk − z‖2 ≤ ‖x0 − z‖2 +r−1∑k=1‖xk − z‖2−−(1− ξ2)r−1∑k=0ε2r − 2(1 + ξ)δr−1∑k=0εk,removing common terms we have,0 ≤ ‖xr − z‖2 ≤ ‖x0 − z‖2 − (1− ξ2)r−1∑k=0ε2r − 2(1 + ξ)δr−1∑k=0εk,RELAXATION METHODS 5whereby,2(1 + ξ)δr−1∑k=0δεk ≤ (1− ξ2)r−1∑k=0ε2r + 2(1 + ξ)δr−1∑k=0εk ≤ ‖x0 − z‖2,so that,r−1∑k=0εk ≤ 12λδ‖x0 − z‖2.If we consider the sequence ηr−1 =∑r−1k=0 εk, and the constant K =12λδ‖x0−z‖2,we have ηr−1 ≥ 0 for all r ∈ N, then 0 ≤ limr ηr ≤ K, i.e., the sequence {ηr}is bonded and increasing, whereby converges. Hence,∑∞r=0 εr converges as well(and limr εr = 0).Since, in Step 4 of the ERM algorithm, we have chosen εr such that 0 <µrM< εr,i.e., 0 < µr < εrM , we get limr µr = 0.In the Step 4 of the ERM we have‖xr − xr+1‖ = λεr,then the series∑∞r=0 ‖xr − xr+1‖ converges, so∑∞r=0(xr − xr+1) is absolutelyconvergent (see Th. 26.7 [3]), and, limr xr = xˆ, for some xˆ ∈ Rn.Finally, we will show that xˆ ∈ F . For any t ∈ T , and for all r ∈ N we havebt − a′txr‖a′t‖≤{µr, t ∈ Tr,0, otherwise.Taking limit in the above relation when r →∞, we get bt−a′txˆ‖a′t‖ ≤ 0, for all t ∈ T ,and this proves that xˆ ∈ F .Remark 2. We would like to mention that the geometrical proof of the previoustheorem strongly requires the full dimensionality of the feasible set (δ > 0).4. Test examplesRelaxation method has been implemented in the Matlab software, all test ex-amples were implemented by making appropriate modifications to fminbnd Mat-lab routine, due to the fact that the relaxation method uses the routine to solvethe global optimization problem appearing in Step 2. All the three examples wereset to stop when inft∈T g(xr, t) ≥ −1× 10−4. In all cases, the initial guess x0 wasrandomly generated and we set β = 12, and M = 1000.In the Tables 1-3, results of implementation of ERM, just discussed here, aregiven. The first and third column of the tables are self-explanatory. A descriptionof the remaining columns is as follows. The second column, ite, indicates thenumber of iterations made before the stopping criterion was satisfied (we establisha limit of 15000 iterations as maximum), the last column, time, is the time ofexecution in seconds on a computer of 2Ghz dual processor, 2GB of RAM and6 E. GONZA´LEZ-GUTIE´RREZ, L. HERNANDEZ REBOLLAR, AND MAXIM I. TODOROVWindows 7 operating system. Finally, Figures 1-3 show the feasible set (regionin blank), for each of the systemsExample 3. Consider the convex set F defined by F = {x ∈ R2 : (x21 + x22)2 ≥κ2(x21 − x22); 0 ≤ x1 ≤ κ} (a quarter of the lemniscate curve, see Figure 1) andconsider its linear representation, σ, given by σ = {a1tx1 + a2tx2 ≥ bt : t ∈ [0, pi2 ]}wherea1t := −κ cos 3t,a2t := −κ sin 3t,bt := −κ2(cos 3t cos t√cos 2t+ sin 3t sin t√cos 2t).The set F depends on the parameter κ, we choose it as κ = 2. For initial guessx0 = (−1564.979244, 2189.253881), the results are presented in Table 1.0 0.5 1 1.5 2 2.5-1-0.500.511.5xyFigure 1. The feasible set F of the Example 3.λ ite xr time0.1 179 (0.127274, 0.113900) 3.7564220.3 54 (0.127840, 0.114339)′ 0.5961940.5 28 (0.128261, 0.114649)′ 0.3658050.7 17 (0.128210, 0.114510)′ 0.6808840.9 10 (0.127417, 0.113514)′ 0.2825981 7 (0.187578, 0.140557)′ 0.1555911.5 23 (0.242725, 0.015244)′ 0.384612 1066 (0.204403, 0.089469)′ 8.209755Table 1. Results of Example 3RELAXATION METHODS 7Example 4. Consider the linear representation of the ellipse, F = {x ∈ R2 :2x21 + x22 + 2x1x2 + 2x1 ≤ 0} (see Figure 2) given byσ = {(−t4 − 2t3 + 3t2 + 2t− 1)x1 − 2t(t2 − 1)x2 ≥ −2t2 : t ∈ [−1, 1]},(see [11]). With initial guess, x0 = (3.423734, 14.120922), the relaxation algo--5 -4 -3 -2 -1 0 1 2-3-2-101234xyFigure 2. The fesible set F of the Example 4.rithm gives results summarized in Table 2. Note: for the value of the relaxationparameter λ = 0.1, the algorithm exceeds the number of iterations permitted andfinalize without finding a feasible solution.λ ite xr time0.1 - - -0.3 110 (−1.828697, 2.388394)′ 0.3736140.5 59 (−1.827002, 2.39201)′ 0.2600620.7 38 (−1.824438, 2.390390)′ 0.2106640.9 20 (−1.794767, 2.401682)′ 0.1707991 9 (−1.866903, 2.365380)′ 0.1293161.5 6 (−1.722008, 2.319358)′ 0.1278412 12 (−1.686474, 1.201378)′ 0.141970Table 2. Resullts of Example 4Example 5. For n = 2, we consider the system σ = {−tetx1 − tx2 ≥ −1 : t ∈[0, 1]},(see [8], [9]). In this example for t = 0, a0 = 02, so the conditions requiredat the beginning of the paper are not fulfilled. The feasible set of σ is shown inFigure 3. We use the initial guess x0 = (12.353328, 17.188846)′. The results arereported in Table 3. Like in Example 4, for some values of the parameter λ, f.i.,λ = 0.1, 0.3, 0.7 the algorithm exceeds the iteration number permitted and finalizewithout finding a feasible solution.8 E. GONZA´LEZ-GUTIE´RREZ, L. HERNANDEZ REBOLLAR, AND MAXIM I. TODOROV-2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5-10123456xyFigure 3. The feasible set F of the Example 5.λ ite xr time0.1 - - -0.3 - - -0.5 60 (−5.269856, 10.330953)′ 0.2565300.7 - - -0.9 21 (−6.046282, 11.435680) 0.1614491 8 (−5.333119, 10.421842)′ 0.1384741.5 2 (−11.819860, 8.384212)′ 0.1128952 2 (−19.877589, 5.449334)′ 0.118692Table 3. Reports for Example 5At the end, we would like to summarize that, apart of the theoretical convergingresult obtained in the paper, more numerical experiments should be done in orderto get better notion of the role of all parameters, especially λ and M, including inthe ERM algorithms. But anyway, the results confirm once again the commentin [1], that the so called over projection methods have better performance.References[1] Agmon, S., The relaxation method for linear inequalities, Canadian Journal of Mathemat-ics, 6:382-392, 1954.[2] Antunes de Oliveira, A., Rojas-Medar, M.A., Proper efficiency in vector infinite program-ming problems, Optimization Letters, Volume (2009) 3, Number 3, 319-328.[3] Bartle, R.G., The elements of real analysis, John Wiley & Sons, 1964.[4] Censor, Y., Zenios, S., Parallel optimization: Theory, Algorithms, and Applications, Ox-ford University Press, New York, 1997.[5] Coope, I.D., Watson, G.A., A projected Lagrangian algorithm for semi-infinite program-ming. Mathematical Programming 32 (1985), 337–356.[6] Fang, S-C., Wu, S-Y., An inexact approach to solving linear semi-infinite programmingproblems. Optimization 28 (1994), 291–299.RELAXATION METHODS 9[7] Fang, S-C., Wu, S-Y., Birbil, S.I., Solving variational inequalities defined on a domain withinfinitely many linear constraints. Computational Optimization and Applications, Volume37 , Issue 1 (May 2007), 67-81.[8] Goberna M. A., Herna´ndez L., Todorov M.I., On Linear inequality systems with smoothcoefficients, Journal of Opt. Theory and Appl., 124 (2005), 363–386.[9] Goberna M.A., Hernandez L., Todorov M.I., Separating the solution sets of analytical andpolynomial systems. TOP, (2005), Vol. 13, No. 2, pp. 321-329.[10] Goberna, M.A., Lo´pez, M.A., Linear semi-infinite optimization theory: an updated survey,European J. Oper. Res. 143 (2002) 390-415.[11] Goberna, M.A., Lo´pez, M.A., Linear Semi-Infinite Optimization, Wiley, Chichester, Eng-land, 1998.[12] Gonza´lez-Gutie´rrez, E., Todorov, M. I., A relaxation method for solving systems withinfinitely many linear inequalities, Optim. Lett., (2010) DOI 10.1007/s11590-010-0244-4.[13] Herman, G.T., Image reconstruction from projections: The fundamentals of computerizedTomography, Academic Press, New York, 1980.[14] Hu, H., A projection method for solving infinite systems of linear inequalities, Advancesin Optimization and Approximation, Kluwer Academic Publishers (1994), 186-194.[15] Jeroslow, R.G., Some relaxation methods for linear inequalities, Cahiers du Cero, Vol 21(1979), 43-53.[16] Leon, T., Sanmatias, S., Vercher, E., Of numerical treatment of linearly constrained semi-infinite problems. European J. Oper. Res. 121 (2000) 79-91.[17] Lin, C-J., Yang, E.K., Fang, S-C., Implementation of an inexact approach to solving linearsemi-infinite programming problems. Journal of Computational and Applied Mathematics61 (1995), 87–103.[18] Maugeri, A., Raciti, F., On general infinite dimensional complementarity problems, Opti-mization Letters, Volume (2008) 2, Number 1, 71-90.[19] Motzkin, T.S., Shoenberg, I.J., The relaxation method for linear inequalities, Reprintedfrom Canadian Journal of Mathematics, 6:393-404, 1954.[20] Watson, G.A., Numerical Experiments with Globally Convergent Methods for Semi-InfiniteProgramming Problems. Lecture notes in Economics and Mathematical Systems no. 215.Springer Verlag, Berlin, 1983, pp. 193–205.E. Gonza´lez-Gutie´rrezFCFM-BUAP, Puebla, MX & UA, Alicante, SPE-mail address: e g gutierrez@yahoo.esL. Hernandez RebollarFCFM-BUAP, Puebla, MXE-mail address: lhernan@fcfm.buap.mxMaxim I. TodorovUDLAP, Puebla, MX. On leave from IMI-BAS, Sofia, BGE-mail address: maxim.todorov@udlap.mx

Relaxation methods for solving linear inequality systems: Converging results

http://ddd.uab.cat/pub/prepub/2010/hdl_2072_169254/Pr990.pdf

Relaxation methods for solving linear inequality systems: Converging results

Abstract

Similar works

Full text

Available Versions

Diposit Digital de Documents de la UAB