A.L. Dontchev

C. Li

F.-A. Potra

G. Perria

J.-P. Aubin

J.W. Schmidt

P. Saint-Pierre

R. Baier

T. Yamamoto

English

Robert Baier

Mirko Hessel-von Molo

Crossref

Newton’s Method and Secant Method for Set-Valued Mappings

Baier, Robert

Hessel-von Molo, Mirko

EPub Bayreuth

Newton’s Method and Secant Methodfor Set-Valued MappingsRobert Baier1 and Mirko Hessel-von Molo21 University of Bayreuth, Chair of Applied Math., 95440 Bayreuth, Germany,robert.baier@uni-bayreuth.de2 University of Paderborn, Chair of Applied Math., 33098 Paderborn, Germany,mirkoh@math.uni-paderborn.deAbstract. For finding zeros or fixed points of set-valued maps, the factthat the space of convex, compact, nonempty sets of Rn is not a vectorspace presents a major disadvantage. Therefore, fixed point iterations orvariants of Newton’s method, in which the derivative is applied only toa smooth single-valued part of the set-valued map, are often applied forcalculations. We will embed the set-valued map with convex, compactimages (i.e. by embedding its images) and shift the problem to the Ba-nach space of directed sets. This Banach space extends the arithmeticoperations of convex sets and allows to consider the Fréchet-derivativeor divided differences of maps that have embedded convex images. Forthe transformed problem, Newton’s method and the secant method inBanach spaces are applied via directed sets. The results can be visualizedas usually nonconvex sets in Rn.Keywords: set-valued Newton’s method, set-valued secant method,Gauß-Newton method, directed sets, embedding of convex compact sets1 IntroductionIn this article we illustrate how Newton’s method and the similar secant methodcan be applied to solve two standard problems for set-valued mappings withconvex, compact images. Our key method is to embed these problems into aBanach space setting, which allows to reformulate them as zero-finding problems.For references on uses of Newton’s method in Banach spaces see e.g. [12,6]. Wewould like to point out that the ideas presented here differ from the pioneeringworks (e.g. [10,1]) based on homotopy methods or mutational equations. Westart this article with a brief introduction to the Banach space of directed sets.In Sections 2 and 3, we apply this to introduce the Fréchet-derivative as well asNewton’s method and the secant method for set-valued maps. We conclude withseveral numerical examples in Section 4.1.1 Directed setsDirected sets provide a way to interpret compact, convex, nonempty sets of Rn,i.e. sets in C(Rn), as elements of a Banach space. They were introduced in [3]2(see also references therein), their visualization is studied in [3, Part II]. ForC,D ∈ C(Rn) and M ∈ Rm×n, we write C +D for the Minkowski sum, λ ·C forthe multiplication with non-negative scalars and M · C for the linear image.Definition 1. Directed intervals resp. sets−→A ∈ D(Rn) are defined recursively.For n = 1, a directed interval is a pair−→A = (a1(l))l=±1 of two real numbers.(We will also use the notation−→A =−−→[c, d], where a1(−1) = −c and a1(1) = d.)In this case, the norm of−→A is given by ‖−→A‖1 = maxl=±1 |a1(l)|.For n ≥ 2, a directed set has two components−→A =(−−−−−→An−1(l), an(l))l∈Sn−1,given by a map−−−→An−1 : Sn−1 → D(Rn−1) that is uniformly bounded with respectto ‖ · ‖n−1, and a continuous real-valued function an(·). The norm is defined by‖−→A‖ := ‖−→A‖n := max{ supl∈Sn−1‖−−−−−→An−1(l)‖n−1, maxl∈Sn−1|an(l)|} .As a space of functions on Sn−1 with values in a linear space, with theusual pointwise definitions of addition and scalar multiplication, D(Rn) is alinear space. With the norm defined above, D(Rn) is a Banach space that can beequipped with a partial ordering that represents inclusion of sets (see [3, Part I]).Sets in C(Rn) can be embedded in this Banach space via Jn : C(Rn)→ D(Rn) asJ1([c, d]) :=−−→[c, d] = (−c, d) ,Jn(C) :=(Jn−1(Πn−1,l · Y (l, C)), δ∗(l, C))l∈Sn−1(n ≥ 2) .Here, δ∗(l, C) denotes the support function of C in direction l ∈ Sn−1, Y (l, C)the corresponding supporting face (i.e. the maximizers of 〈l, ·〉 on the set C), andΠn−1,l is a linear projection which maps the orthogonal complement of span{l}into Rn−1 and l to the origin in Rn−1, see [3, Part I] for details.Proposition 1 ([3]). The embedding Jn : C(Rn)→ D(Rn) is positively linear:Jn(λ · C + µ ·D) = λ · Jn(C) + µ · Jn(D) (C,D ∈ C(Rn), λ, µ ≥ 0)In contrast to formulations using pairs of sets (see [3] for references), a di-rected set (and thus a difference of embedded convex sets) can be visualized asa subset in Rn which is usually nonconvex. The visualization of the directed setis plotted with arrows (outer normals for embedded convex sets, inner ones fortheir inverses) on the boundaries (see [3, Part II] for details).2 Fréchet-Derivative for Set-Valued MapsAs D(Rm) forms a Banach space, we use the usual definition of the Fréchet-derivative of a function−→F : Rn → D(Rm). To lift a set-valued map F : Rn ⇒ Rmwith images in C(Rm) to the space D(Rm), we will use the notation−→F (x) =Jm(F (x)). If−→F : Rn → D(Rm) is Fréchet-differentiable, F is called directeddifferentiable. This notion of the derivative generalizes the one of set-valuedmaps for one variable studied in [8,4].3Lemma 1 (see [2]). Let F : Rn → C(Rm) and w ∈ Rn.(i) If F (x) = U , then−→F ′(x)(w) = Jm({0}).(ii) If F (x) = r(x) · G(x) + h(x) with r : Rn → [0,∞), h : Rn → Rm, G :Rn → C(Rm) which are (directed) Fréchet-differentiable, then−→F ′(x)(w) =r′(x) · w ·−→G(x) + r(x) ·−→G ′(x)(w) + Jm({h′(x) · w}).Especially, if F (x) = r(x) ·U with U ∈ C(Rm), then−→F ′(x)(w) = r′(x) ·w ·−→U .(iii) If−→F (x) = r(x) ·−→G(x) +−→H (x) and r : Rn → R,−→G,−→H : Rn → D(Rm) areFréchet-differentiable, then−→F ′(x)(w) = r′(x) ·w ·−→G(x) + r(x) ·−→G ′(x)(w) +−→H ′(x)(w).Proof. (i) A constant map in a Banach space has 0 as Fréchet-differential.(ii) follows immediately from (iii) with−→G(x) = Jm(G(x)), Jm(r(x) · G(x)) =r(x) · Jm(G(x)) and−→H (x) = Jm({h(x)}). Since −Jm({h(x)}) = Jm({−h(x)})we can directly show from the definition that−→H ′(x)(w) = Jm({h′(x) · w}).(iii) Clearly, the sum and the product of two functions are Fréchet-differentiable,if the operands have this property. utWe now study explicit formulas for the Fréchet-derivative.Proposition 2. Let F : Rn → C(Rm) and w, x ∈ Rn.(i) If m = 1 and F (x) = [g(x), h(x)] with two differentiable functions g, h : Rn →R and g(x) ≤ h(x) for all x ∈ Rn, then−→F is differentiable with−→F ′(x)(w) =−−−−−−−−−−−−−−→[g′(x) · w, h′(x) · w] .(ii) If m ≥ 2, the Fréchet-differential of−→F (x) = (−→F m−1(l;x), fm(l;x))l∈Sm−1 is−→F ′(x)(w) = (−→F ′m−1(l;x)(w), f′m(l;x) · w)l∈Sm−1 .Proof. (i) apply Lemma 1(ii) to F (x) = g(x) + (h(x)− g(x)) · [−1, 1](ii) for a proof see [2, Proposition 3.6] ut3 Newton’s Method and Secant MethodFor the sake of brevity, we refer to the literature for convergence results aboutiterative methods in Banach spaces. For Newton’s method see e.g. [12,5,6], forthe secant method e.g. [9] and for Gauß-Newton method e.g. [7].3.1 Newton’s Method for Directed SetsProblem 1. Let F : Rn → C(Rm) be given. We are looking for a solution x̂ ∈ Rnof 0 ∈ F (x̂). Using the partial order “≤” on D(Rm) and the embedded map−→F ,this problem can be transformed into Jm({0}) ≤−→F (x̂).To apply Newton’s iteration for directed sets to Problem 1 we choose astarting point x0 ∈ Rn and iteratively obtain xk+1 = xk + dk, where dk ∈ Rn4has to solve the linear inequality−−→F (xk) ≤−→F ′(xk)(dk) . (1)Checking this inequality essentially means checking the positivity of a function,which is easily realized algorithmically.Remark 1. In many publications, e.g. in the study of necessary optimality con-ditions in non-smooth optimization or optimal control problems, the set-valuedmap F is given in the form F (x) = g(x) + H(x) with a smooth functiong : Rn → Rm and an u.s.c. map H : Rn ⇒ Rm (representing subdifferentialsor normal cones in the mentioned applications), see e.g. [5,6]. The set-valuedderivative of F is avoided and an inclusion step of the form0 ∈ g(xk) + g′(xk) · dk +H(xk + dk)is studied. While in this case the computation of dk typically requires a nonlinearinclusion to be solved, for this splitting of F (with Fréchet-differentiable−→H )Newton’s method as suggested above leads to an inclusion problem linearly indk, since Newton’s method in (1) demands a solution of−Jm({g(xk)})−−→H (xk) ≤ Jm({g′(xk) · dk}) +−→H ′(xk)(dk) .Problem 2. Let G,H : Rn → C(Rm) be given. We are looking for a solutionx̂ ∈ Rn of G(x̂) = H(x̂). Transformed to D(Rm), this reads−→G(x̂) =−→H (x̂).Using−→F =−→G −−→H , Newton’s iteration with directed sets for this problemproceeds similarly to the one for Problem 1, with the difference that the updatesdk have to solve the linear equation−→F ′(xk)(dk) = −−→F (xk) . (2)3.2 Secant Method Based on Directed SetsTo apply the secant method as studied e.g. in [11,9], we need a notion of divideddifferences. Set-valued divided differences in one variable are studied e.g. in [8,4].Following [11], we introduce multivariate divided differences for directed sets.Definition 2. Let−→F : Rn → D(Rm) and u, v, w ∈ Rn be given such that uj 6= vjfor j = 1, . . . n. We define−→F [u, v]j := 1uj−vj ·(−→F (u1, . . . , uj−1, uj , vj+1, . . . , vn)−−→F (u1, . . . , uj−1, vj , vj+1, . . . , vn))∈ D(Rm) ,−→F [u, v](w) :=∑nj=1 wj ·−→F [u, v]j ∈ D(Rm) .In the limit for u→ v, this definition yields the Fréchet-differential−→F ′(u).5This approach generalizes the divided differences in one variable and satisfiesthe characteristic equation−→F [u, v](u− v) =−→F (u)−−→F (v) (u, v ∈ Rn) . (3)To apply the secant method to Problem 1, one chooses two starting pointsx−1, x0 ∈ Rn and iteratively computes xk+1 = xk + dk, where dk solves−−→F (xk) ≤−→F [xk, xk−1](dk) . (4)To treat Problem 2 with the secant method, we again set−→F =−→G −−→H , chooseinitial points x−1 and x0 and compute xk+1 = xk + dk, where dk solves−→F [xk, xk−1](dk) = −−→F (xk) . (5)An advantage of (5) is that the derivative−→F ′ is avoided and replaced by differ-ences of function values. It is obvious that the iterations (2) and (5) generalizetheir pointwise analogues.4 ExamplesIn this section we present numerical examples for the procedures proposed above.Example 1. Let U1, . . . , UM ∈ C(Rn) be such that the vectors Jn(Ui) are lin-early independent in D(Rn). Consider an unknown convex combination U0 =∑Mi=1 λi · Ui with coefficients λi ≥ 0, i = 1, . . . ,M and∑Mi=1 λi = 1. In or-der to apply Newton’s method for Problem 2 to compute the vector λ ∈ RM ,we define G(x) =∑Mi=1 xi · Ui for x ∈ RM and H(x) = U0, and set−→F (x) =∑Mi=1 xi · Jn(Ui)− Jn(U0). By Proposition 1−→F (x) =M∑i=1(xi − λi) · Jn(Ui) and−→F ′(x)(w) =M∑i=1wi · Jn(Ui) ,so that the Newton step results inM∑i=1dki · Jn(Ui) = −M∑i=1(xki − λi) · Jn(Ui) andM∑i=1(xki − λi + dki ) · Jn(Ui) = 0 .From the assumption of linear independence we obtain dki − λi + xki = 0 fori = 1, . . . ,M , and hence dk = −xk + λ. This means that already the firstiteration step leads to the solution x1 = x̂ = λ. We obtain the same picture forthe secant method. Since−→F [xk, xk−1]i =1xki − xk−1i·(xki · Jn(Ui)− xk−1i · Jn(Ui))= Jn(Ui) ,−→F [xk, xk−1](dk) =M∑i=1dki ·−→F [xk, xk−1]i =M∑i=1dki · Jn(Ui) ,the left-hand side of (5) equals the one of the Newton step (2).From now on, we consider Problem 2.6Example 2. Let α, β ∈ R and consider the functions g(v) = α‖v‖22v+βv, h(v) =‖v‖2v for v ∈ R2. We consider Problem 2 with G,H : R → C(R2), G(x) =g(x ·B1(0)), H(x) = h(x ·B1(0)). A simple calculation shows that g(x ·B1(0)) =(αx3 +βx) ·B1(0) and h(x ·B1(0)) = x2 ·B1(0). The solutions for α = −1, β = 34are x̂ ∈ {0,− 32 ,12}. To eliminate the trival solution x̂ = 0, we divide the leadingterm by x and obtain−→F (x) = (αx2 + β − x) · J2(B1(0)). For this, one has−→F ′(x) = (2αx− 1) ·−→B with−→B = J2(B1(0)) and−→F [u, v] =1u− v·((αu2 − u+ β)− (αv2 − v + β))·−→B = (α(u+ v)− 1) ·−→B .For the Newton step we thus get dk = −α(xk)2−xk+β2αxk−1 , while the iteration forthe secant method results in dk = − α(xk)2−xk+βα(xk+xk−1)−1 . In Fig. 1(a) the iterates−→Xk := xk · J2(B1(0)) of the secant method are depicted for the starting valuesx−1 = 0, x0 = − 34 . Here, the iterates k = 0, 2, 3 are inverses of embedded sets(inner normals), whereas the other iterates are embedded convex sets (outernormals). For k = −1, the embedded origin is visualized in the plot.In Example 3, we solve Problem 2 using the Gauß-Newton method in [7] andthe Gauß-secant method for two-dimensional directed sets. Similar to the approx-imation of convex sets with finitely many supporting hyperplanes, we approxi-mate a set-valued map−→F : Rn → D(R2) by choosing a finite number of unit vec-tors lµ ∈ S1, µ = 1, . . . ,M and evaluating−→F (x) = (f1(x; η; l)η=±1, f2(x; l))l∈S1in these directions, leading to a discretized map−→F M : Rn → R3M defined by(−→F M (x))i=f1(x;−1; li) if i = 1, . . . ,M,f1(x; 1; li−M ) if i = M + 1, . . . , 2M,f2(x; li−2M ) if i = 2M + 1, . . . , 3M.Recalling the ideas behind the Gauß-Newton method, we replace−→F M (x) =0 by the minimization problem minx∈Rn ‖−→F M (x)‖22 which is solved iterativelyby minimizing ‖−→F M (xk) +−→F ′M (xk)(dk)‖22 or ‖−→F M (xk) +−→F M [xk, xk−1](dk)‖22,respectively. This leads to the normal equations−→F ′M (xk)>−→F ′M (xk)(dk) = −−→F ′M (xk)>−→F M (xk) and (6)−→F [xk, xk−1]>−→F [xk, xk−1](dk) = −−→F [xk, xk−1]>−→F M (xk) (7)as iterative steps. The (i, j)-th entry of the matrix of divided differences is givenfor j = 1, 2 by the expressionsf1(xk1 , xk+j−22 ;−1; li)− f1(xk+j−21 , xk−12 ;−1; li)xkj − xk−1jfor i = 1, . . . ,M,f1(xk1 , xk+j−22 ; 1; li)− f1(xk+j−21 , xk−12 ; 1; li)xkj − xk−1jfor i = M + 1, . . . , 2M,7f2(xk1 , xk+j−22 ); li−2M )− f2(xk+j−21 , xk−12 ; li−2M )xkj − xk−1jfor i = 2M + 1, . . . , 3M,which are much simpler to compute than the corresponding partial derivatives.In the next nonlinear test example we are looking for parameters in thedetermining matrix of an ellipsoid centered in the origin which coincides withthe unit ball.Example 3. For x ∈ R2 we consider G(x) = A(x) ·B1(0) with A(x) =(x1 00 x2),H(x) = B1(0) in Problem 2. In this example, it is rather complicated to evaluatethe derivative of−→F (x) =−→G(x) −−→H (x) analytically. Clearly, x̂ = (1, 1)> is onesolution and−→F ′(x) = (−→G ′1(x; l), g2(x; l))l∈S1 withf2(x; l) = g2(x; l)− 1 = δ∗(l, G(x))− 1 =√l21x21 + l22x22 − 1 ,∂f2∂x(x; l) =1√l21x21 + l22x22(l21x1l22x2)>.The derivative of−→F ′1(x; l) within the Gauß-Newton method is complicated, sinceit involves the projection to R, whereas the calculation of−→F [u, v](w) in theGauß-secant method only requires the weighted sum of differences of functionvalues. Table 1 demonstrates that the Gauß-secant method converges ratherrapidly, but slower than the Gauß-Newton method. In Fig. 1(b) the iterates−→Xk := J2(A(xk)·B1(0)) of the secant method are depicted for the starting valuesx−1 = (4, 3)>, x0 = (3, 2)>. Depending on the starting values, convergence toone of the solutions (±1,±1)> can be observed.To sketch the outline of further research, we mention [2]. In this publication,Newton’s method for set-valued maps is used to approximate invariant sets,i.e. to find sets P for which g(P ) = P . This task is similar to Problem 2, if theset is parametrized e.g. as P (x) = x·B1(0) (see Example 2) and G(x) = g(P (x)),H(x) = h(P (x)), h(x) = x, but considerably more complicated, as the map−→Gdepends on a directed set. This leads to a zero-finding problems of the form0 =−→F (−→X ) = Jn(g(Vn(−→X )) −−→X , the solution of which requires much moreadvanced Banach space techniques than in the present paper.References1. Aubin, J.P.: Mutational and Morphological Analysis. Tools for Shape Evolutionand Morphogenesis. Systems & Control: Foundations & Applications, BirkhäuserBoston Inc., Boston, MA (1999)2. Baier, R., Dellnitz, M., Hessel-von Molo, M., Kevrekidis, I.G., Sertl, S.: The com-putation of invariant sets via Newton’s method (May 2010), submitted, 21 pages.3. Baier, R., Farkhi, E.: Differences of Convex Compact Sets in the Space of DirectedSets. Part I: The Space of Directed Sets. Set-Valued Anal. 9(3), 217–245 (2001),Part II: Visualization of Directed Sets. Same volume, 247–252.84. Baier, R., Perria, G.: Set-valued Hermite interpolation. J. Approx. Theory (2011),doi: 10.1016/j.jat.2010.11.004, 24 pages.5. Dontchev, A.L., Hager, W.W., Veliov, V.M.: Uniform convergence and mesh inde-pendence of Newton’s method for discretized variational problems. SIAM J. Con-trol Optim. 39(3), 961–980 (electronic) (2000)6. Dontchev, A.L., Veliov, V.M.: Metric regularity under approximations. ControlCybernet. 38(4B), 1283–1303 (2009)7. Li, C., Zhang, W.H., Jin, X.Q.: Convergence and uniqueness properties of Gauss-Newton’s method. Comput. Math. Appl. 47(6-7), 1057–1067 (2004)8. Perria, G.: Set-valued interpolation. Bayreuth. Math. Schr. 79, 154 pp. (2007)9. Potra, F.A.: An error analysis for the secant method. Numer. Math. 38(3), 427–445(1981/82)10. Saint-Pierre, P.: Newton and other continuation methods for multivalued inclu-sions. Set-Valued Anal. 3(2), 143–156 (1995)11. Schmidt, J.W.: Eine Übertragung der Regula Falsi auf Gleichungen in Ba-nachräumen. II. Nichtlineare Gleichungssysteme. Z. Angew. Math. Mech. 43(3),97–110 (1963)12. Yamamoto, T.: A method for finding sharp error bounds for Newton’s methodunder the Kantorovich assumptions. Numer. Math. 49(2–3), 203–220 (1986)−1 0 1 2 3 4 5 6 7 8−4 0 4−4−2024x1kx2−1012   −3−1.5   0 1.5   3  −3−1.5   0 1.5   3x1kx2(a) Example 2 (b) Example 3Fig. 1. Iterates of the secant method for two examplesk xk1 xk212‖−→F M (xk)‖22 xk1 xk2 12‖−→F M (xk)‖22-1 4.00000 3.00000 149.28479510 3.00000000 2.00000000 68.997425636149 3.00000 2.00000 68.99742561 1.01002370 1.00647268 0.001644629887 0.98748 0.98710 0.00323432 1.00000078 1.00000078 0.000000000012 0.99942 0.99933 0.0000081Table 1. Iterates xk of the Gauß-Newton and the secant method for Example 3

Newton's method and secant method for set-valued mappings

https://epub.uni-bayreuth.de/5636/1/baier_et_al_lssc_2012.pdf

Newton's method and secant method for set-valued mappings

Abstract

Similar works

Full text

Available Versions

Crossref

EPub Bayreuth