9 research outputs found
Primal-dual accelerated gradient methods with small-dimensional relaxation oracle
In this paper, a new variant of accelerated gradient descent is proposed. The
pro-posed method does not require any information about the objective function,
usesexact line search for the practical accelerations of convergence, converges
accordingto the well-known lower bounds for both convex and non-convex
objective functions,possesses primal-dual properties and can be applied in the
non-euclidian set-up. Asfar as we know this is the rst such method possessing
all of the above properties atthe same time. We also present a universal
version of the method which is applicableto non-smooth problems. We demonstrate
how in practice one can efficiently use thecombination of line-search and
primal-duality by considering a convex optimizationproblem with a simple
structure (for example, linearly constrained)
On the Complexity of Approximating Wasserstein Barycenter
We study the complexity of approximating Wassertein barycenter of
discrete measures, or histograms of size by contrasting two alternative
approaches, both using entropic regularization. The first approach is based on
the Iterative Bregman Projections (IBP) algorithm for which our novel analysis
gives a complexity bound proportional to to
approximate the original non-regularized barycenter. Using an alternative
accelerated-gradient-descent-based approach, we obtain a complexity
proportional to . As a byproduct, we show that
the regularization parameter in both approaches has to be proportional to
, which causes instability of both algorithms when the desired
accuracy is high. To overcome this issue, we propose a novel proximal-IBP
algorithm, which can be seen as a proximal gradient method, which uses IBP on
each iteration to make a proximal step. We also consider the question of
scalability of these algorithms using approaches from distributed optimization
and show that the first algorithm can be implemented in a centralized
distributed setting (master/slave), while the second one is amenable to a more
general decentralized distributed setting with an arbitrary network topology.Comment: Corrected misprints. Added a reference to accelerated Iterative
Bregman Projections introduced in arXiv:1906.0362
On primal and dual approaches for distributed stochastic convex optimization over networks
We introduce a primal-dual stochastic gradient oracle method for distributed convex optimization problems over networks. We show that the proposed method is optimal in terms of communication steps. Additionally, we propose a new analysis method for the rate of convergence in terms of duality gap and probability of large deviations. This analysis is based on a new technique that allows to bound the distance between the iteration sequence and the optimal point. By the proper choice of batch size, we can guarantee that this distance equals (up to a constant) to the distance between the starting point and the solution
Inexact relative smoothness and strong convexity for optimization and variational inequalities by inexact model
In this paper we propose a general algorithmic framework for first-order methods in optimization in a broad sense, including minimization problems, saddle-point problems and variational inequalities. This framework allows to obtain many known methods as a special case, the list including accelerated gradient method, composite optimization methods, level-set methods, Bregman proximal methods. The idea of the framework is based on constructing an inexact model of the main problem component, i.e. objective function in optimization or operator in variational inequalities. Besides reproducing known results, our framework allows to construct new methods, which we illustrate by constructing a universal conditional gradient method and universal method for variational inequalities with composite structure. These method works for smooth and non-smooth problems with optimal complexity without a priori knowledge of the problem smoothness. As a particular case of our general framework, we introduce relative smoothness for operators and propose an algorithm for VIs with such operator. We also generalize our framework for relatively strongly convex objectives and strongly monotone variational inequalities