1,200 research outputs found
Proximal Convexification Procedures in Combinatorial Optimization
Final version has appeared under the title "On a primal-proximal heuristic in discrete optimization", in Math. Program., Ser. A 104, pp 105-128 (2005), DOI 10.1007/s10107-004-0571-2Lagrangian relaxation is useful to bound the optimal value of a given optimization problem, and also to obtain relaxed solutions. To obtain primal solutions, it is conceivable to use a convexification procedure suggested by D.P. Bertsekas in 1979, based on the proximal algorithm in the primal space. The present paper studies the theory assessing the approach in the framework of combinatorial optimization. Our results indicate that very little can be expected in theory, even though fairly good practical results have been obtained for the unit-commitment problem
A Smoothed Dual Approach for Variational Wasserstein Problems
Variational problems that involve Wasserstein distances have been recently
proposed to summarize and learn from probability measures. Despite being
conceptually simple, such problems are computationally challenging because they
involve minimizing over quantities (Wasserstein distances) that are themselves
hard to compute. We show that the dual formulation of Wasserstein variational
problems introduced recently by Carlier et al. (2014) can be regularized using
an entropic smoothing, which leads to smooth, differentiable, convex
optimization problems that are simpler to implement and numerically more
stable. We illustrate the versatility of this approach by applying it to the
computation of Wasserstein barycenters and gradient flows of spacial
regularization functionals
Primal-Dual Algorithms for Non-negative Matrix Factorization with the Kullback-Leibler Divergence
Non-negative matrix factorization (NMF) approximates a given matrix as a
product of two non-negative matrices. Multiplicative algorithms deliver
reliable results, but they show slow convergence for high-dimensional data and
may be stuck away from local minima. Gradient descent methods have better
behavior, but only apply to smooth losses such as the least-squares loss. In
this article, we propose a first-order primal-dual algorithm for non-negative
decomposition problems (where one factor is fixed) with the KL divergence,
based on the Chambolle-Pock algorithm. All required computations may be
obtained in closed form and we provide an efficient heuristic way to select
step-sizes. By using alternating optimization, our algorithm readily extends to
NMF and, on synthetic examples, face recognition or music source separation
datasets, it is either faster than existing algorithms, or leads to improved
local optima, or both
MAP inference via Block-Coordinate Frank-Wolfe Algorithm
We present a new proximal bundle method for Maximum-A-Posteriori (MAP)
inference in structured energy minimization problems. The method optimizes a
Lagrangean relaxation of the original energy minimization problem using a multi
plane block-coordinate Frank-Wolfe method that takes advantage of the specific
structure of the Lagrangean decomposition. We show empirically that our method
outperforms state-of-the-art Lagrangean decomposition based algorithms on some
challenging Markov Random Field, multi-label discrete tomography and graph
matching problems
Convex optimization problem prototyping for image reconstruction in computed tomography with the Chambolle-Pock algorithm
The primal-dual optimization algorithm developed in Chambolle and Pock (CP),
2011 is applied to various convex optimization problems of interest in computed
tomography (CT) image reconstruction. This algorithm allows for rapid
prototyping of optimization problems for the purpose of designing iterative
image reconstruction algorithms for CT. The primal-dual algorithm is briefly
summarized in the article, and its potential for prototyping is demonstrated by
explicitly deriving CP algorithm instances for many optimization problems
relevant to CT. An example application modeling breast CT with low-intensity
X-ray illumination is presented.Comment: Resubmitted to Physics in Medicine and Biology. Text has been
modified according to referee comments, and typos in the equations have been
correcte
Subspace System Identification via Weighted Nuclear Norm Optimization
We present a subspace system identification method based on weighted nuclear
norm approximation. The weight matrices used in the nuclear norm minimization
are the same weights as used in standard subspace identification methods. We
show that the inclusion of the weights improves the performance in terms of fit
on validation data. As a second benefit, the weights reduce the size of the
optimization problems that need to be solved. Experimental results from
randomly generated examples as well as from the Daisy benchmark collection are
reported. The key to an efficient implementation is the use of the alternating
direction method of multipliers to solve the optimization problem.Comment: Submitted to IEEE Conference on Decision and Contro
Scaling Algorithms for Unbalanced Transport Problems
This article introduces a new class of fast algorithms to approximate
variational problems involving unbalanced optimal transport. While classical
optimal transport considers only normalized probability distributions, it is
important for many applications to be able to compute some sort of relaxed
transportation between arbitrary positive measures. A generic class of such
"unbalanced" optimal transport problems has been recently proposed by several
authors. In this paper, we show how to extend the, now classical, entropic
regularization scheme to these unbalanced problems. This gives rise to fast,
highly parallelizable algorithms that operate by performing only diagonal
scaling (i.e. pointwise multiplications) of the transportation couplings. They
are generalizations of the celebrated Sinkhorn algorithm. We show how these
methods can be used to solve unbalanced transport, unbalanced gradient flows,
and to compute unbalanced barycenters. We showcase applications to 2-D shape
modification, color transfer, and growth models
A Smooth Primal-Dual Optimization Framework for Nonsmooth Composite Convex Minimization
We propose a new first-order primal-dual optimization framework for a convex
optimization template with broad applications. Our optimization algorithms
feature optimal convergence guarantees under a variety of common structure
assumptions on the problem template. Our analysis relies on a novel combination
of three classic ideas applied to the primal-dual gap function: smoothing,
acceleration, and homotopy. The algorithms due to the new approach achieve the
best known convergence rate results, in particular when the template consists
of only non-smooth functions. We also outline a restart strategy for the
acceleration to significantly enhance the practical performance. We demonstrate
relations with the augmented Lagrangian method and show how to exploit the
strongly convex objectives with rigorous convergence rate guarantees. We
provide numerical evidence with two examples and illustrate that the new
methods can outperform the state-of-the-art, including Chambolle-Pock, and the
alternating direction method-of-multipliers algorithms.Comment: 35 pages, accepted for publication on SIAM J. Optimization. Tech.
Report, Oct. 2015 (last update Sept. 2016
- …