A Newton-like method and its application

AbstractIn this paper we prove an existence and uniqueness theorem for solving the operator equation F(x)+G(x)=0, where F is a Gateaux differentiable continuous operator while the operator G satisfies a Lipschitz-condition on an open convex subset of a Banach space. As corollaries, a theorem of Tapia on a weak Newton's method and the classical convergence theorem for modified Newton-iterates are deduced. An existence theorem for a generalized Euler–Lagrange equation in the setting of Sobolev space is obtained as a consequence of the main theorem. We also obtain a class of Gateaux differentiable operators which are nowhere Frechet differentiable. Illustrative examples are also provided

A trust region-type normal map-based semismooth Newton method for nonsmooth nonconvex composite optimization

We propose a novel trust region method for solving a class of nonsmooth and nonconvex composite-type optimization problems. The approach embeds inexact semismooth Newton steps for finding zeros of a normal map-based stationarity measure for the problem in a trust region framework. Based on a new merit function and acceptance mechanism, global convergence and transition to fast local q-superlinear convergence are established under standard conditions. In addition, we verify that the proposed trust region globalization is compatible with the Kurdyka-{\L}ojasiewicz (KL) inequality yielding finer convergence results. We further derive new normal map-based representations of the associated second-order optimality conditions that have direct connections to the local assumptions required for fast convergence. Finally, we study the behavior of our algorithm when the Hessian matrix of the smooth part of the objective function is approximated by BFGS updates. We successfully link the KL theory, properties of the BFGS approximations, and a Dennis-Mor{\'e}-type condition to show superlinear convergence of the quasi-Newton version of our method. Numerical experiments on sparse logistic regression and image compression illustrate the efficiency of the proposed algorithm.Comment: 56 page

Inexact Newton Methods For Solving Nonsmooth Equations

This paper investigates inexact Newton methods for solving systems of nonsmooth equations. We define two inexact Newton methods for locally Lipschitz functions and we prove local (linear and superlinear) convergence results under the assumptions of semismoothness and BD-regularity at the solution. We introduce a globally convergent inexact iteration function based method. We discuss implementations and we give some numerical examples. © 1995.601-2127145Brown, Saad, Hybrid Krylov Methods for Nonlinear Systems of Equations (1990) SIAM Journal on Scientific and Statistical Computing, 11, pp. 450-481Broyden, A class of methods for solving nonlinear simultaneous equations (1965) Mathematics of Computation, 19, pp. 577-593Broyden, Dennis, More, On the local and superlinear convergence of quasi-Newton methods (1973) J. Inst. Math. Appl., 12, pp. 223-246Chandrasekhar, (1960) Radiative Transfer, , Dover, New YorkChen, On the convergence of Broyden-like methods for nonlinear equations with nondifferentiable terms (1990) Ann. Inst. Statist. Math., 42, pp. 387-401Chen, Nashed, Qi, Convergence of Newton's method for singular smooth and nonsmooth equations using adaptive outer inverses (1993) Appl. Math. Preprint 93/4, , Univ. New South Wales, Sydney, AustraliaChen, Qi, A parameterized Newton method and a Broyden-like method for nonsmooth equations (1994) Comput. Optim. Appl., 3, pp. 157-179Chen, Yamamoto, On the convergence of some quasi-Newton methods for nonlinear equations with nondifferentiable operators (1992) Computing, 48, pp. 87-94Clarke, (1983) Optimization and Nonsmooth Analysis, , Wiley, New YorkDembo, Eisenstat, Steihaug, Inexact Newton Methods (1982) SIAM Journal on Numerical Analysis, 14, pp. 400-408Dennis, Moré, A characterization of superlinear convergence and its application to quasi-Newton methods (1974) Mathematics of Computation, 28, pp. 549-560Dennis, Schnabel, (1983) Numerical Methods for Unconstrained Optimization and Nonlinear Equations, , Prentice-Hall, Englewood Cliffs, NJDeuflhard, Global inexact Newton methods for very large scale nonlinear problems (1991) Impact Comput. Sci. and Engrg., 3, pp. 366-393Deuflhard, Freund, Walter, Fast secant methods for the iterative solution of large nonsymmetric linear systems (1990) Impact Comput. Sci. and Engrg., 2, pp. 244-276S.C. Eisenstat and H.F. Walker, Globally convergent inexact Newton methods, Research Report, Dept. Math. Statist., Utah State Univ. USAFischer, A special newton-type optimization method (1992) Optimization, 24, pp. 269-284Gabriel, Pang, An inexact NE/SQP method for solving the nonlinear complementarity problem (1992) Comput. Optim. Appl., 1, pp. 67-92Gabriel, Pang, A trust region method for constrained nonsmooth equations (1994) Large-Scale Optimization: State of the Art, pp. 159-186. , W.W. Hager, D.W. Hearn, P. Pardalos, Kluwer Academic, BostonGomes-Ruggiero, Martínez, Moretti, Comparing Algorithms for Solving Sparse Nonlinear Systems of Equations (1992) SIAM Journal on Scientific and Statistical Computing, 13, pp. 459-483Gomes-Ruggiero, Martínez, Santos, Solving nonsmooth equations by means of quasi-Newton methods with globalization (1994) Preprint, Dept. Appl. Math., IMECC-UNICAMP, , State Univ. Campinas, Campinas, BrazilHan, Pang, Rangaraj, Globally Convergent Newton Methods for Nonsmooth Equations (1992) Mathematics of Operations Research, 17, pp. 586-607Harker, Xiao, Newton's method for the nonlinear complementarity problem: A B-differentiable equation approach (1990) Mathematical Programming, 48, pp. 339-357Ip, Kyparisis, Local convergence of quasi-Newton methods for B-differentiable equations (1992) Math. Programming, 56, pp. 71-90Kanzow, (1993) Global convergence properties of some iterative methods for linear complementarity problems, , Preprint 72, Institut für Angewandte Mathematik der Universität HamburgKojima, Shindo, Extensions of Newton and quasi-Newton methods to systems of PC1 equations (1986) J. Oper. Res. Soc. Japan, 29, pp. 352-374Kummer, Newton's method for non-differentiable functions (1988) Adv. Math. Optim., pp. 114-125. , J. Guddat, B. Bank, H. Hollatz, P. Kall, D. Klatte, B. Kummer, K. Lommatzsch, L. Tammer, M. Vlach, K. Zimmermann, Akademie-Verlag, BerlinMartínez, Local Convergence Theory of Inexact Newton Methods Based on Structured Least Change Updates (1990) Mathematics of Computation, 55, pp. 143-168Martínez, A Theory of Secant Preconditioners (1993) Mathematics of Computation, 60, pp. 681-698Martínez, Zambaldi, Least change update methods for nonlinear systems with nondifferentiable terms∗ (1993) Numerical Functional Analysis and Optimization, 14, pp. 405-415Mifflin, Semismooth and Semiconvex Functions in Constrained Optimization (1977) SIAM Journal on Control and Optimization, 15, pp. 957-972Moré, A collection of nonlinear model problems (1989) Preprint MCS-P60-0289, Math. and Comput. Sci. Div., , Argonne National Laboratory, Argonne, ILMoré, Garbow, Hillstrom, Testing Unconstrained Optimization Software (1981) ACM Transactions on Mathematical Software, 7, pp. 17-41Moré, Trangenstein, On the Global Convergence of Broyden's Method (1976) Mathematics of Computation, 30, pp. 523-540Ortega, Rheinboldt, (1970) Iterative Solution of Nonlinear Equations in Several Variables, , Academic Press, New YorkPang, Newton's Method for B-Differentiable Equations (1990) Mathematics of Operations Research, 15, pp. 311-341Pang, A B-differentiable equation based, globally, and locally quadratically convergent algorithm for nonlinear programs, complementarity and variational inequality problems (1991) Math. Programming, 51, pp. 101-131Pang, A degree-theoretic approach to parametric nonsmooth equations with multivalued perturbed solution sets (1993) Mathematical Programming, 62, pp. 359-383J.S. Pang, Serial and parallel computation of Karush-Kuhn-Tucker points via nonsmooth equation, SIAM J. Optim., forthcomingPang, Gabriel, NE/SQP: A robust algorithm for the nonlinear complementarity problem (1993) Mathematical Programming, 60, pp. 295-337Pang, Qi, Nonsmooth Equations: Motivation and Algorithms (1993) SIAM Journal on Optimization, 3, pp. 443-465J.S. Pang and L. Qi, A globally convergent Newton method for convex SC1 minimization problems. J. Optim. Theory Appl., forthcomingQi, Convergence Analysis of Some Algorithms for Solving Nonsmooth Equations (1993) Mathematics of Operations Research, 18, pp. 227-244Qi, Superlinearly convergent approximate Newton methods for LC1 optimization problems (1994) Mathematical Programming, 64 (3), pp. 2770-2794L. Qi, Trust region algorithms for solving nonsmooth equations, SIAM J. Optim., forthcomingL. Qi and X. Chen, A globally convergent successive approximation method for severely nonsmooth equations, SIAM J. Control Optim., forthcomingQi, Jiang, Karush-Kuhn-Tucker equations and convergence analysis of Newton methods and quasi-Newton methods for solving these equations (1994) Appl. Math. Report 94/5, , Univ. New South Wales, Sydney, AustraliaQi, Sun, A nonsmooth version of Newton's method (1993) Math. Programming, 58, pp. 353-368Ralph, Global Convergence of Damped Newton's Method for Nonsmooth Equations via the Path Search (1994) Mathematics of Operations Research, 19, pp. 352-389Robinson, Newton's method for a class of nonsmooth functions (1988) Industrial Engineering Working Paper, , Univ. Wisconsin, Madison, WISpedicato, Computational experience with quasi-Newton algorithms for minimization problems of moderately large size (1975) Report CISE-N-175, , International Press, Milano, SegrateSun, Qi, An interior point algorithm of O(m|ϵ|) iterations for C1-convex programming (1992) Mathematical Programming, 57, pp. 239-257Yamamoto, Chen, On split nonsmooth equations (1994) Numerical Mathematics, , Z.C. Shi, International Press, Hong KongYpma, Local convergence of inexact Newton methods (1984) SIAM J. Numer. Anal., 21, pp. 583-59