206 research outputs found
A proximal approach for optimization problems involving Kullback divergences
International audienceConvex optimization problems involving information measures have been extensively investigated in source and channel coding. These measures can also be successfully used in inverse problems encountered in signal and image processing. The related optimization problems are often challenging due to their large size. In this paper, we derive closed-form expressions of the proximity operators of Kullback-Leibler and Jeffreys-Kullback divergences. Building upon these results, we develop an efïŹcient primal-dual proximal approach. This allows us to address a wide range of convex optimization problems whose objective function expression includes one of these divergences. An image registration application serves as an example for illustrating the good performance of the proposed method
Proximity Operators of Discrete Information Divergences
Information divergences allow one to assess how close two distributions are
from each other. Among the large panel of available measures, a special
attention has been paid to convex -divergences, such as
Kullback-Leibler, Jeffreys-Kullback, Hellinger, Chi-Square, Renyi, and
I divergences. While -divergences have been extensively
studied in convex analysis, their use in optimization problems often remains
challenging. In this regard, one of the main shortcomings of existing methods
is that the minimization of -divergences is usually performed with
respect to one of their arguments, possibly within alternating optimization
techniques. In this paper, we overcome this limitation by deriving new
closed-form expressions for the proximity operator of such two-variable
functions. This makes it possible to employ standard proximal methods for
efficiently solving a wide range of convex optimization problems involving
-divergences. In addition, we show that these proximity operators are
useful to compute the epigraphical projection of several functions of practical
interest. The proposed proximal tools are numerically validated in the context
of optimal query execution within database management systems, where the
problem of selectivity estimation plays a central role. Experiments are carried
out on small to large scale scenarios
Scaling Algorithms for Unbalanced Transport Problems
This article introduces a new class of fast algorithms to approximate
variational problems involving unbalanced optimal transport. While classical
optimal transport considers only normalized probability distributions, it is
important for many applications to be able to compute some sort of relaxed
transportation between arbitrary positive measures. A generic class of such
"unbalanced" optimal transport problems has been recently proposed by several
authors. In this paper, we show how to extend the, now classical, entropic
regularization scheme to these unbalanced problems. This gives rise to fast,
highly parallelizable algorithms that operate by performing only diagonal
scaling (i.e. pointwise multiplications) of the transportation couplings. They
are generalizations of the celebrated Sinkhorn algorithm. We show how these
methods can be used to solve unbalanced transport, unbalanced gradient flows,
and to compute unbalanced barycenters. We showcase applications to 2-D shape
modification, color transfer, and growth models
Entropic Wasserstein Gradient Flows
This article details a novel numerical scheme to approximate gradient flows
for optimal transport (i.e. Wasserstein) metrics. These flows have proved
useful to tackle theoretically and numerically non-linear diffusion equations
that model for instance porous media or crowd evolutions. These gradient flows
define a suitable notion of weak solutions for these evolutions and they can be
approximated in a stable way using discrete flows. These discrete flows are
implicit Euler time stepping according to the Wasserstein metric. A bottleneck
of these approaches is the high computational load induced by the resolution of
each step. Indeed, this corresponds to the resolution of a convex optimization
problem involving a Wasserstein distance to the previous iterate. Following
several recent works on the approximation of Wasserstein distances, we consider
a discrete flow induced by an entropic regularization of the transportation
coupling. This entropic regularization allows one to trade the initial
Wasserstein fidelity term for a Kulback-Leibler divergence, which is easier to
deal with numerically. We show how KL proximal schemes, and in particular
Dykstra's algorithm, can be used to compute each step of the regularized flow.
The resulting algorithm is both fast, parallelizable and versatile, because it
only requires multiplications by a Gibbs kernel. On Euclidean domains
discretized on an uniform grid, this corresponds to a linear filtering (for
instance a Gaussian filtering when is the squared Euclidean distance) which
can be computed in nearly linear time. On more general domains, such as
(possibly non-convex) shapes or on manifolds discretized by a triangular mesh,
following a recently proposed numerical scheme for optimal transport, this
Gibbs kernel multiplication is approximated by a short-time heat diffusion
A Smoothed Dual Approach for Variational Wasserstein Problems
Variational problems that involve Wasserstein distances have been recently
proposed to summarize and learn from probability measures. Despite being
conceptually simple, such problems are computationally challenging because they
involve minimizing over quantities (Wasserstein distances) that are themselves
hard to compute. We show that the dual formulation of Wasserstein variational
problems introduced recently by Carlier et al. (2014) can be regularized using
an entropic smoothing, which leads to smooth, differentiable, convex
optimization problems that are simpler to implement and numerically more
stable. We illustrate the versatility of this approach by applying it to the
computation of Wasserstein barycenters and gradient flows of spacial
regularization functionals
Learning the Proximity Operator in Unfolded ADMM for Phase Retrieval
This paper considers the phase retrieval (PR) problem, which aims to
reconstruct a signal from phaseless measurements such as magnitude or power
spectrograms. PR is generally handled as a minimization problem involving a
quadratic loss. Recent works have considered alternative discrepancy measures,
such as the Bregman divergences, but it is still challenging to tailor the
optimal loss for a given setting. In this paper we propose a novel strategy to
automatically learn the optimal metric for PR. We unfold a recently introduced
ADMM algorithm into a neural network, and we emphasize that the information
about the loss used to formulate the PR problem is conveyed by the proximity
operator involved in the ADMM updates. Therefore, we replace this proximity
operator with trainable activation functions: learning these in a supervised
setting is then equivalent to learning an optimal metric for PR. Experiments
conducted with speech signals show that our approach outperforms the baseline
ADMM, using a light and interpretable neural architecture.Comment: 10 pages, 5 figures, submitted to IEEE SP
- âŠ