180 research outputs found
Non-smooth Non-convex Bregman Minimization: Unification and new Algorithms
We propose a unifying algorithm for non-smooth non-convex optimization. The
algorithm approximates the objective function by a convex model function and
finds an approximate (Bregman) proximal point of the convex model. This
approximate minimizer of the model function yields a descent direction, along
which the next iterate is found. Complemented with an Armijo-like line search
strategy, we obtain a flexible algorithm for which we prove (subsequential)
convergence to a stationary point under weak assumptions on the growth of the
model function error. Special instances of the algorithm with a Euclidean
distance function are, for example, Gradient Descent, Forward--Backward
Splitting, ProxDescent, without the common requirement of a "Lipschitz
continuous gradient". In addition, we consider a broad class of Bregman
distance functions (generated by Legendre functions) replacing the Euclidean
distance. The algorithm has a wide range of applications including many linear
and non-linear inverse problems in signal/image processing and machine
learning
Learning Repairs for Marked Structures
[Abstract not available
On convergence of the maximum block improvement method
Abstract. The MBI (maximum block improvement) method is a greedy approach to solving optimization problems where the decision variables can be grouped into a finite number of blocks. Assuming that optimizing over one block of variables while fixing all others is relatively easy, the MBI method updates the block of variables corresponding to the maximally improving block at each iteration, which is arguably a most natural and simple process to tackle block-structured problems with great potentials for engineering applications. In this paper we establish global and local linear convergence results for this method. The global convergence is established under the Lojasiewicz inequality assumption, while the local analysis invokes second-order assumptions. We study in particular the tensor optimization model with spherical constraints. Conditions for linear convergence of the famous power method for computing the maximum eigenvalue of a matrix follow in this framework as a special case. The condition is interpreted in various other forms for the rank-one tensor optimization model under spherical constraints. Numerical experiments are shown to support the convergence property of the MBI method
- …