1,953 research outputs found
Extragradient method with variance reduction for stochastic variational inequalities
We propose an extragradient method with stepsizes bounded away from zero for
stochastic variational inequalities requiring only pseudo-monotonicity. We
provide convergence and complexity analysis, allowing for an unbounded feasible
set, unbounded operator, non-uniform variance of the oracle and, also, we do
not require any regularization. Alongside the stochastic approximation
procedure, we iteratively reduce the variance of the stochastic error. Our
method attains the optimal oracle complexity (up to
a logarithmic term) and a faster rate in terms of the mean
(quadratic) natural residual and the D-gap function, where is the number of
iterations required for a given tolerance . Such convergence rate
represents an acceleration with respect to the stochastic error. The generated
sequence also enjoys a new feature: the sequence is bounded in if the
stochastic error has finite -moment. Explicit estimates for the convergence
rate, the oracle complexity and the -moments are given depending on problem
parameters and distance of the initial iterate to the solution set. Moreover,
sharper constants are possible if the variance is uniform over the solution set
or the feasible set. Our results provide new classes of stochastic variational
inequalities for which a convergence rate of holds in terms
of the mean-squared distance to the solution set. Our analysis includes the
distributed solution of pseudo-monotone Cartesian variational inequalities
under partial coordination of parameters between users of a network.Comment: 39 pages. To appear in SIAM Journal on Optimization (submitted July
2015, accepted December 2016). Uploaded in IMPA's preprint server at
http://preprint.impa.br/visualizar?id=688
Galerkin Methods for Complementarity Problems and Variational Inequalities
Complementarity problems and variational inequalities arise in a wide variety
of areas, including machine learning, planning, game theory, and physical
simulation. In all of these areas, to handle large-scale problem instances, we
need fast approximate solution methods. One promising idea is Galerkin
approximation, in which we search for the best answer within the span of a
given set of basis functions. Bertsekas proposed one possible Galerkin method
for variational inequalities. However, this method can exhibit two problems in
practice: its approximation error is worse than might be expected based on the
ability of the basis to represent the desired solution, and each iteration
requires a projection step that is not always easy to implement efficiently.
So, in this paper, we present a new Galerkin method with improved behavior: our
new error bounds depend directly on the distance from the true solution to the
subspace spanned by our basis, and the only projections we require are onto the
feasible region or onto the span of our basis
First-order Convergence Theory for Weakly-Convex-Weakly-Concave Min-max Problems
In this paper, we consider first-order convergence theory and algorithms for
solving a class of non-convex non-concave min-max saddle-point problems, whose
objective function is weakly convex in the variables of minimization and weakly
concave in the variables of maximization. It has many important applications in
machine learning including training Generative Adversarial Nets (GANs). We
propose an algorithmic framework motivated by the inexact proximal point
method, where the weakly monotone variational inequality (VI) corresponding to
the original min-max problem is solved through approximately solving a sequence
of strongly monotone VIs constructed by adding a strongly monotone mapping to
the original gradient mapping. We prove first-order convergence to a nearly
stationary solution of the original min-max problem of the generic algorithmic
framework and establish different rates by employing different algorithms for
solving each strongly monotone VI. Experiments verify the convergence theory
and also demonstrate the effectiveness of the proposed methods on training
GANs.Comment: In this revised version, we changed title to "First-order Convergence
Theory for Weakly-Convex-Weakly-Concave Min-max Problems" and added more
experimental result
On the Convergence Properties of Non-Euclidean Extragradient Methods for Variational Inequalities with Generalized Monotone Operators
In this paper, we study a class of generalized monotone variational
inequality (GMVI) problems whose operators are not necessarily monotone (e.g.,
pseudo-monotone). We present non-Euclidean extragradient (N-EG) methods for
computing approximate strong solutions of these problems, and demonstrate how
their iteration complexities depend on the global Lipschitz or H\"{o}lder
continuity properties for their operators and the smoothness properties for the
distance generating function used in the N-EG algorithms. We also introduce a
variant of this algorithm by incorporating a simple line-search procedure to
deal with problems with more general continuous operators. Numerical studies
are conducted to illustrate the significant advantages of the developed
algorithms over the existing ones for solving large-scale GMVI problems
Accelerated Schemes For A Class of Variational Inequalities
We propose a novel method, namely the accelerated mirror-prox (AMP) method,
for computing the weak solutions of a class of deterministic and stochastic
monotone variational inequalities (VI). The main idea of this algorithm is to
incorporate a multi-step acceleration scheme into the mirror-prox method. For
both deterministic and stochastic VIs, the developed AMP method computes the
weak solutions with optimal rate of convergence. In particular, if the monotone
operator in VI consists of the gradient of a smooth function, the rate of
convergence of the AMP method can be accelerated in terms of its dependence on
the Lipschitz constant of the smooth function. For VIs with bounded feasible
sets, the estimate of the rate of convergence of the AMP method depends on the
diameter of the feasible set. For unbounded VIs, we adopt the modified gap
function introduced by Monteiro and Svaiter for solving monotone inclusion, and
demonstrate that the rate of convergence of the AMP method depends on the
distance from the initial point to the set of strong solutions
Projected Reflected Gradient Methods for Monotone Variational Inequalities
This paper is concerned with some new projection methods for solving
variational inequality problems with monotone and Lipschitz-continuous mapping
in Hilbert space. First, we propose the projected reflected gradient algorithm
with a constant stepsize. It is similar to the projected gradient method,
namely, the method requires only one projection onto the feasible set and only
one value of the mapping per iteration. This distinguishes our method from most
other projection-type methods for variational inequalities with monotone
mapping. Also we prove that it has R-linear rate of convergence under the
strong monotonicity assumption. The usual drawback of algorithms with constant
stepsize is the requirement to know the Lipschitz constant of the mapping. To
avoid this, we modify our first algorithm so that the algorithm needs at most
two projections per iteration. In fact, our computational experience shows that
such cases with two projections are very rare. This scheme, at least
theoretically, seems to be very effective. All methods are shown to be globally
convergent to a solution of the variational inequality. Preliminary results
from numerical experiments are quite promising
A Relaxed-Projection Splitting Algorithm for Variational Inequalities in Hilbert Spaces
We introduce a relaxed-projection splitting algorithm for solving variational
inequalities in Hilbert spaces for the sum of nonsmooth maximal monotone
operators, where the feasible set is defined by a nonlinear and nonsmooth
continuous convex function inequality. In our scheme, the orthogonal
projections onto the feasible set are replaced by projections onto separating
hyperplanes. Furthermore, each iteration of the proposed method consists of
simple subgradient-like steps, which does not demand the solution of a
nontrivial subproblem, using only individual operators, which exploits the
structure of the problem. Assuming monotonicity of the individual operators and
the existence of solutions, we prove that the generated sequence converges
weakly to a solution.Comment: 18 page
Proximal extrapolated gradient methods for variational inequalities
The paper concerns with novel first-order methods for monotone variational
inequalities. They use a very simple linesearch procedure that takes into
account a local information of the operator. Also the methods do not require
Lipschitz-continuity of the operator and the linesearch procedure uses only
values of the operator. Moreover, when operator is affine our linesearch
becomes very simple, namely, it needs only vector-vector multiplication. For
all our methods we establish the ergodic convergence rate. Although the
proposed methods are very general, sometimes they may show much better
performance even for optimization problems. The reason for this is that they
often can use larger stepsizes without additional expensive computation
On the analysis of variance-reduced and randomized projection variants of single projection schemes for monotone stochastic variational inequality problems
Classical extragradient schemes and their stochastic counterpart represent a
cornerstone for resolving monotone variational inequality problems. Yet, such
schemes have a per-iteration complexity of two projections onto a convex set
and require two evaluations of the map, the former of which could be relatively
expensive if is a complicated set. We consider two related avenues where
the per-iteration complexity is significantly reduced: (i) A stochastic
projected reflected gradient method requiring a single evaluation of the map
and a single projection; and (ii) A stochastic subgradient extragradient method
that requires two evaluations of the map, a single projection onto , and a
significantly cheaper projection (onto a halfspace) computable in closed form.
Under a variance-reduced framework reliant on a sample-average of the map based
on an increasing batch-size, we prove almost sure (a.s.) convergence of the
iterates to a random point in the solution set for both schemes. Additionally,
both schemes display a non-asymptotic rate of where
denotes the number of iterations; notably, both rates match those obtained in
deterministic regimes. To address feasibility sets given by the intersection of
a large number of convex constraints, we adapt both of the aforementioned
schemes to a random projection framework. We then show that the random
projection analogs of both schemes also display a.s. convergence under a
weak-sharpness requirement; furthermore, without imposing the weak-sharpness
requirement, both schemes are characterized by a provable rate of
in terms of the gap function of the projection of the
averaged sequence onto as well as the infeasibility of this sequence.
Preliminary numerics support theoretical findings and the schemes outperform
standard extragradient schemes in terms of the per-iteration complexity
Proximal Reinforcement Learning: A New Theory of Sequential Decision Making in Primal-Dual Spaces
In this paper, we set forth a new vision of reinforcement learning developed
by us over the past few years, one that yields mathematically rigorous
solutions to longstanding important questions that have remained unresolved:
(i) how to design reliable, convergent, and robust reinforcement learning
algorithms (ii) how to guarantee that reinforcement learning satisfies
pre-specified "safety" guarantees, and remains in a stable region of the
parameter space (iii) how to design "off-policy" temporal difference learning
algorithms in a reliable and stable manner, and finally (iv) how to integrate
the study of reinforcement learning into the rich theory of stochastic
optimization. In this paper, we provide detailed answers to all these questions
using the powerful framework of proximal operators.
The key idea that emerges is the use of primal dual spaces connected through
the use of a Legendre transform. This allows temporal difference updates to
occur in dual spaces, allowing a variety of important technical advantages. The
Legendre transform elegantly generalizes past algorithms for solving
reinforcement learning problems, such as natural gradient methods, which we
show relate closely to the previously unconnected framework of mirror descent
methods. Equally importantly, proximal operator theory enables the systematic
development of operator splitting methods that show how to safely and reliably
decompose complex products of gradients that occur in recent variants of
gradient-based temporal difference learning. This key technical innovation
makes it possible to finally design "true" stochastic gradient methods for
reinforcement learning. Finally, Legendre transforms enable a variety of other
benefits, including modeling sparsity and domain geometry. Our work builds
extensively on recent work on the convergence of saddle-point algorithms, and
on the theory of monotone operators.Comment: 121 page
- …