245,521 research outputs found
Newton-MR: Inexact Newton Method With Minimum Residual Sub-problem Solver
We consider a variant of inexact Newton Method, called Newton-MR, in which
the least-squares sub-problems are solved approximately using Minimum Residual
method. By construction, Newton-MR can be readily applied for unconstrained
optimization of a class of non-convex problems known as invex, which subsumes
convexity as a sub-class. For invex optimization, instead of the classical
Lipschitz continuity assumptions on gradient and Hessian, Newton-MR's global
convergence can be guaranteed under a weaker notion of joint regularity of
Hessian and gradient. We also obtain Newton-MR's problem-independent local
convergence to the set of minima. We show that fast local/global convergence
can be guaranteed under a novel inexactness condition, which, to our knowledge,
is much weaker than the prior related works. Numerical results demonstrate the
performance of Newton-MR as compared with several other Newton-type
alternatives on a few machine learning problems.Comment: 35 page
Generalized Newton's Method based on Graphical Derivatives
This paper concerns developing a numerical method of the Newton type to solve
systems of nonlinear equations described by nonsmooth continuous functions. We
propose and justify a new generalized Newton algorithm based on graphical
derivatives, which have never been used to derive a Newton-type method for
solving nonsmooth equations. Based on advanced techniques of variational
analysis and generalized differentiation, we establish the well-posedness of
the algorithm, its local superlinear convergence, and its global convergence of
the Kantorovich type. Our convergence results hold with no semismoothness
assumption, which is illustrated by examples. The algorithm and main results
obtained in the paper are compared with well-recognized semismooth and
-differentiable versions of Newton's method for nonsmooth Lipschitzian
equations
- …