184 research outputs found
Zero-Convex Functions, Perturbation Resilience, and Subgradient Projections for Feasibility-Seeking Methods
The convex feasibility problem (CFP) is at the core of the modeling of many
problems in various areas of science. Subgradient projection methods are
important tools for solving the CFP because they enable the use of subgradient
calculations instead of orthogonal projections onto the individual sets of the
problem. Working in a real Hilbert space, we show that the sequential
subgradient projection method is perturbation resilient. By this we mean that
under appropriate conditions the sequence generated by the method converges
weakly, and sometimes also strongly, to a point in the intersection of the
given subsets of the feasibility problem, despite certain perturbations which
are allowed in each iterative step. Unlike previous works on solving the convex
feasibility problem, the involved functions, which induce the feasibility
problem's subsets, need not be convex. Instead, we allow them to belong to a
wider and richer class of functions satisfying a weaker condition, that we call
"zero-convexity". This class, which is introduced and discussed here, holds a
promise to solve optimization problems in various areas, especially in
non-smooth and non-convex optimization. The relevance of this study to
approximate minimization and to the recent superiorization methodology for
constrained optimization is explained.Comment: Mathematical Programming Series A, accepted for publicatio
A Bregman-Kaczmarz method for nonlinear systems of equations
We propose a new randomized method for solving systems of nonlinear
equations, which can find sparse solutions or solutions under certain simple
constraints. The scheme only takes gradients of component functions and uses
Bregman projections onto the solution space of a Newton equation. In the
special case of euclidean projections, the method is known as nonlinear
Kaczmarz method. Furthermore, if the component functions are nonnegative, we
are in the setting of optimization under the interpolation assumption and the
method reduces to SGD with the recently proposed stochastic Polyak step size.
For general Bregman projections, our method is a stochastic mirror descent with
a novel adaptive step size. We prove that in the convex setting each iteration
of our method results in a smaller Bregman distance to exact solutions as
compared to the standard Polyak step. Our generalization to Bregman projections
comes with the price that a convex one-dimensional optimization problem needs
to be solved in each iteration. This can typically be done with globalized
Newton iterations. Convergence is proved in two classical settings of
nonlinearity: for convex nonnegative functions and locally for functions which
fulfill the tangential cone condition. Finally, we show examples in which the
proposed method outperforms similar methods with the same memory requirements
An Efficient HPR Algorithm for the Wasserstein Barycenter Problem with Computational Complexity
In this paper, we propose and analyze an efficient Halpern-Peaceman-Rachford
(HPR) algorithm for solving the Wasserstein barycenter problem (WBP) with fixed
supports. While the Peaceman-Rachford (PR) splitting method itself may not be
convergent for solving the WBP, the HPR algorithm can achieve an
non-ergodic iteration complexity with respect to the
Karush-Kuhn-Tucker (KKT) residual. More interestingly, we propose an efficient
procedure with linear time computational complexity to solve the linear systems
involved in the subproblems of the HPR algorithm. As a consequence, the HPR
algorithm enjoys an non-ergodic computational
complexity in terms of flops for obtaining an -optimal solution
measured by the KKT residual for the WBP, where is the dimension
of the variable of the WBP. This is better than the best-known complexity bound
for the WBP. Moreover, the extensive numerical results on both the synthetic
and real data sets demonstrate the superior performance of the HPR algorithm
for solving the large-scale WBP
Convergence in Distribution of Randomized Algorithms: The Case of Partially Separable Optimization
We present a Markov-chain analysis of blockwise-stochastic algorithms for
solving partially block-separable optimization problems. Our main contributions
to the extensive literature on these methods are statements about the Markov
operators and distributions behind the iterates of stochastic algorithms, and
in particular the regularity of Markov operators and rates of convergence of
the distributions of the corresponding Markov chains. This provides a detailed
characterization of the moments of the sequences beyond just the expected
behavior. This also serves as a case study of how randomization restores
favorable properties to algorithms that iterations of only partial information
destroys. We demonstrate this on stochastic blockwise implementations of the
forward-backward and Douglas-Rachford algorithms for nonconvex (and, as a
special case, convex), nonsmooth optimization.Comment: 25 pages, 43 reference
Postquantum Br\`{e}gman relative entropies and nonlinear resource theories
We introduce the family of postquantum Br\`{e}gman relative entropies, based
on nonlinear embeddings into reflexive Banach spaces (with examples given by
reflexive noncommutative Orlicz spaces over semi-finite W*-algebras,
nonassociative L spaces over semi-finite JBW-algebras, and noncommutative
L spaces over arbitrary W*-algebras). This allows us to define a class of
geometric categories for nonlinear postquantum inference theory (providing an
extension of Chencov's approach to foundations of statistical inference), with
constrained maximisations of Br\`{e}gman relative entropies as morphisms and
nonlinear images of closed convex sets as objects. Further generalisation to a
framework for nonlinear convex operational theories is developed using a larger
class of morphisms, determined by Br\`{e}gman nonexpansive operations (which
provide a well-behaved family of Mielnik's nonlinear transmitters). As an
application, we derive a range of nonlinear postquantum resource theories
determined in terms of this class of operations.Comment: v2: several corrections and improvements, including an extension to
the postquantum (generally) and JBW-algebraic (specifically) cases, a section
on nonlinear resource theories, and more informative paper's titl
- …