8,632 research outputs found
A trust region-type normal map-based semismooth Newton method for nonsmooth nonconvex composite optimization
We propose a novel trust region method for solving a class of nonsmooth and
nonconvex composite-type optimization problems. The approach embeds inexact
semismooth Newton steps for finding zeros of a normal map-based stationarity
measure for the problem in a trust region framework. Based on a new merit
function and acceptance mechanism, global convergence and transition to fast
local q-superlinear convergence are established under standard conditions. In
addition, we verify that the proposed trust region globalization is compatible
with the Kurdyka-{\L}ojasiewicz (KL) inequality yielding finer convergence
results. We further derive new normal map-based representations of the
associated second-order optimality conditions that have direct connections to
the local assumptions required for fast convergence. Finally, we study the
behavior of our algorithm when the Hessian matrix of the smooth part of the
objective function is approximated by BFGS updates. We successfully link the KL
theory, properties of the BFGS approximations, and a Dennis-Mor{\'e}-type
condition to show superlinear convergence of the quasi-Newton version of our
method. Numerical experiments on sparse logistic regression and image
compression illustrate the efficiency of the proposed algorithm.Comment: 56 page
Stochastic Trust Region Methods with Trust Region Radius Depending on Probabilistic Models
We present a stochastic trust-region model-based framework in which its
radius is related to the probabilistic models. Especially, we propose a
specific algorithm, termed STRME, in which the trust-region radius depends
linearly on the latest model gradient. The complexity of STRME method in
non-convex, convex and strongly convex settings has all been analyzed, which
matches the existing algorithms based on probabilistic properties. In addition,
several numerical experiments are carried out to reveal the benefits of the
proposed methods compared to the existing stochastic trust-region methods and
other relevant stochastic gradient methods
- …