3 research outputs found
Proximity Operators of Discrete Information Divergences
Information divergences allow one to assess how close two distributions are
from each other. Among the large panel of available measures, a special
attention has been paid to convex -divergences, such as
Kullback-Leibler, Jeffreys-Kullback, Hellinger, Chi-Square, Renyi, and
I divergences. While -divergences have been extensively
studied in convex analysis, their use in optimization problems often remains
challenging. In this regard, one of the main shortcomings of existing methods
is that the minimization of -divergences is usually performed with
respect to one of their arguments, possibly within alternating optimization
techniques. In this paper, we overcome this limitation by deriving new
closed-form expressions for the proximity operator of such two-variable
functions. This makes it possible to employ standard proximal methods for
efficiently solving a wide range of convex optimization problems involving
-divergences. In addition, we show that these proximity operators are
useful to compute the epigraphical projection of several functions of practical
interest. The proposed proximal tools are numerically validated in the context
of optimal query execution within database management systems, where the
problem of selectivity estimation plays a central role. Experiments are carried
out on small to large scale scenarios
Decentralized Constrained Optimization, Double Averaging and Gradient Projection
We consider a generic decentralized constrained optimization problem over
static, directed communication networks, where each agent has exclusive access
to only one convex, differentiable, local objective term and one convex
constraint set. For this setup, we propose a novel decentralized algorithm,
called DAGP (Double Averaging and Gradient Projection), based on local
gradients, projection onto local constraints, and local averaging. We achieve
global optimality through a novel distributed tracking technique we call
distributed null projection. Further, we show that DAGP can also be used to
solve unconstrained problems with non-differentiable objective terms, by
employing the so-called epigraph projection operators (EPOs). In this regard,
we introduce a new fast algorithm for evaluating EPOs. We study the convergence
of DAGP and establish convergence in terms of
feasibility, consensus, and optimality. For this reason, we forego the
difficulties of selecting Lyapunov functions by proposing a new methodology of
convergence analysis in optimization problems, which we refer to as aggregate
lower-bounding. To demonstrate the generality of this method, we also provide
an alternative convergence proof for the gradient descent algorithm for smooth
functions. Finally, we present numerical results demonstrating the
effectiveness of our proposed method in both constrained and unconstrained
problems