1,048 research outputs found
Splitting methods with variable metric for KL functions
We study the convergence of general abstract descent methods applied to a
lower semicontinuous nonconvex function f that satisfies the
Kurdyka-Lojasiewicz inequality in a Hilbert space. We prove that any precompact
sequence converges to a critical point of f and obtain new convergence rates
both for the values and the iterates. The analysis covers alternating versions
of the forward-backward method with variable metric and relative errors. As an
example, a nonsmooth and nonconvex version of the Levenberg-Marquardt algorithm
is detailled
Zero-Convex Functions, Perturbation Resilience, and Subgradient Projections for Feasibility-Seeking Methods
The convex feasibility problem (CFP) is at the core of the modeling of many
problems in various areas of science. Subgradient projection methods are
important tools for solving the CFP because they enable the use of subgradient
calculations instead of orthogonal projections onto the individual sets of the
problem. Working in a real Hilbert space, we show that the sequential
subgradient projection method is perturbation resilient. By this we mean that
under appropriate conditions the sequence generated by the method converges
weakly, and sometimes also strongly, to a point in the intersection of the
given subsets of the feasibility problem, despite certain perturbations which
are allowed in each iterative step. Unlike previous works on solving the convex
feasibility problem, the involved functions, which induce the feasibility
problem's subsets, need not be convex. Instead, we allow them to belong to a
wider and richer class of functions satisfying a weaker condition, that we call
"zero-convexity". This class, which is introduced and discussed here, holds a
promise to solve optimization problems in various areas, especially in
non-smooth and non-convex optimization. The relevance of this study to
approximate minimization and to the recent superiorization methodology for
constrained optimization is explained.Comment: Mathematical Programming Series A, accepted for publicatio
Nonparametric instrumental regression with non-convex constraints
This paper considers the nonparametric regression model with an additive
error that is dependent on the explanatory variables. As is common in empirical
studies in epidemiology and economics, it also supposes that valid instrumental
variables are observed. A classical example in microeconomics considers the
consumer demand function as a function of the price of goods and the income,
both variables often considered as endogenous. In this framework, the economic
theory also imposes shape restrictions on the demand function, like
integrability conditions. Motivated by this illustration in microeconomics, we
study an estimator of a nonparametric constrained regression function using
instrumental variables by means of Tikhonov regularization. We derive rates of
convergence for the regularized model both in a deterministic and stochastic
setting under the assumption that the true regression function satisfies a
projected source condition including, because of the non-convexity of the
imposed constraints, an additional smallness condition
A first-order stochastic primal-dual algorithm with correction step
We investigate the convergence properties of a stochastic primal-dual
splitting algorithm for solving structured monotone inclusions involving the
sum of a cocoercive operator and a composite monotone operator. The proposed
method is the stochastic extension to monotone inclusions of a proximal method
studied in {\em Y. Drori, S. Sabach, and M. Teboulle, A simple algorithm for a
class of nonsmooth convex-concave saddle-point problems, 2015} and {\em I.
Loris and C. Verhoeven, On a generalization of the iterative soft-thresholding
algorithm for the case of non-separable penalty, 2011} for saddle point
problems. It consists in a forward step determined by the stochastic evaluation
of the cocoercive operator, a backward step in the dual variables involving the
resolvent of the monotone operator, and an additional forward step using the
stochastic evaluation of the cocoercive introduced in the first step. We prove
weak almost sure convergence of the iterates by showing that the primal-dual
sequence generated by the method is stochastic quasi Fej\'er-monotone with
respect to the set of zeros of the considered primal and dual inclusions.
Additional results on ergodic convergence in expectation are considered for the
special case of saddle point models
- …