Bregman strongly nonexpansive operators in reflexive Banach spaces

We present a detailed study of right and left Bregman strongly nonexpansive operators in reflexive Banach spaces. We analyze, in particular, compositions and convex combinations of such operators, and prove the convergence of the Picard iterative method for operators of these types. Finally, we use our results to approximate common zeroes of maximal monotone mappings and solutions to convex feasibility problems.Ministerio de Educación y CienciaJunta de AndalucíaIsrael Science Foundatio

Anisotropic Proximal Gradient

This paper studies a novel algorithm for nonconvex composite minimization which can be interpreted in terms of dual space nonlinear preconditioning for the classical proximal gradient method. The proposed scheme can be applied to composite minimization problems whose smooth part exhibits an anisotropic descent inequality relative to a reference function. In the convex case this is a dual characterization of relative strong convexity in the Bregman sense. It is proved that the anisotropic descent property is closed under pointwise average if the dual Bregman distance is jointly convex and, more specifically, closed under pointwise conic combinations for the KL-divergence. We analyze the method's asymptotic convergence and prove its linear convergence under an anisotropic proximal gradient dominance condition. This is implied by anisotropic strong convexity, a recent dual characterization of relative smoothness in the Bregman sense. Applications are discussed including exponentially regularized LPs and logistic regression with nonsmooth regularization. In the LP case the method can be specialized to the Sinkhorn algorithm for regularized optimal transport and a classical parallel update algorithm for AdaBoost. Complementary to their existing primal interpretations in terms of entropic subspace projections this provides a new dual interpretation in terms of forward-backward splitting with entropic preconditioning

A Bregman-Kaczmarz method for nonlinear systems of equations

We propose a new randomized method for solving systems of nonlinear equations, which can find sparse solutions or solutions under certain simple constraints. The scheme only takes gradients of component functions and uses Bregman projections onto the solution space of a Newton equation. In the special case of euclidean projections, the method is known as nonlinear Kaczmarz method. Furthermore, if the component functions are nonnegative, we are in the setting of optimization under the interpolation assumption and the method reduces to SGD with the recently proposed stochastic Polyak step size. For general Bregman projections, our method is a stochastic mirror descent with a novel adaptive step size. We prove that in the convex setting each iteration of our method results in a smaller Bregman distance to exact solutions as compared to the standard Polyak step. Our generalization to Bregman projections comes with the price that a convex one-dimensional optimization problem needs to be solved in each iteration. This can typically be done with globalized Newton iterations. Convergence is proved in two classical settings of nonlinearity: for convex nonnegative functions and locally for functions which fulfill the tangential cone condition. Finally, we show examples in which the proposed method outperforms similar methods with the same memory requirements

Méthodes d'éclatement basées sur les distances de Bregman pour les inclusions monotones composites et l'optimisation

The goal of this thesis is to design splitting methods based on Bregman distances for solving composite monotone inclusions in reflexive real Banach spaces. These results allow us to extend many techniques that were so far limited to Hilbert spaces. Furthermore, even when restricted to Euclidean spaces, they provide new splitting methods that may be more avantageous numerically than the classical methods based on the Euclidean distance. Numerical applications in image processing are proposed.Le but de cette thèse est d'élaborer des méthodes d'éclatement basées sur les distances de Bregman pour la résolution d'inclusions monotones composites dans les espaces de Banach réels réflexifs. Ces résultats nous permettent d'étendre de nombreuses techniques, jusqu'alors limitées aux espaces hilbertiens. De plus, même dans le cadre restreint d'espaces euclidiens, ils donnent lieu à de nouvelles méthodes de décomposition qui peuvent s'avérer plus avantageuses numériquement que les méthodes classiques basées sur la distance euclidienne. Des applications numériques en traitement de l'image sont proposées