2 research outputs found
A Regularized Semi-Smooth Newton Method With Projection Steps for Composite Convex Programs
The goal of this paper is to study approaches to bridge the gap between
first-order and second-order type methods for composite convex programs. Our
key observations are: i) Many well-known operator splitting methods, such as
forward-backward splitting (FBS) and Douglas-Rachford splitting (DRS), actually
define a fixed-point mapping; ii) The optimal solutions of the composite convex
program and the solutions of a system of nonlinear equations derived from the
fixed-point mapping are equivalent. Solving this kind of system of nonlinear
equations enables us to develop second-order type methods. Although these
nonlinear equations may be non-differentiable, they are often semi-smooth and
their generalized Jacobian matrix is positive semidefinite due to monotonicity.
By combining with a regularization approach and a known hyperplane projection
technique, we propose an adaptive semi-smooth Newton method and establish its
convergence to global optimality. Preliminary numerical results on
-minimization problems demonstrate that our second-order type
algorithms are able to achieve superlinear or quadratic convergence.Comment: 25 pages, 4 figure
On efficiency of nonmonotone Armijo-type line searches
Monotonicity and nonmonotonicity play a key role in studying the global
convergence and the efficiency of iterative schemes employed in the field of
nonlinear optimization, where globally convergent and computationally efficient
schemes are explored. This paper addresses some features of descent schemes and
the motivation behind nonmonotone strategies and investigates the efficiency of
an Armijo-type line search equipped with some popular nonmonotone terms. More
specifically, we propose two novel nonmonotone terms, combine them into
Armijo's rule and establish the global convergence of sequences generated by
these schemes. Furthermore, we report extensive numerical results and
comparisons indicating the performance of the nonmonotone Armijo-type line
searches using the most popular search directions for solving unconstrained
optimization problems. Finally, we exploit the considered nonmonotone schemes
to solve an important inverse problem arising in signal and image processing