379 research outputs found
Modified extragradient methods for solving variational inequalities
AbstractIn this paper, we propose two methods for solving variational inequalities. In the first method, we modified the extragradient method by using a new step size while the second method can be viewed as an extension of the first one by performing an additional projection step at each iteration and another optimal step length is employed to reach substantial progress in each iteration. Under certain conditions, the global convergence of two methods is proved. Preliminary numerical experiments are included to illustrate the efficiency of the proposed methods
On the Convergence of (Stochastic) Gradient Descent with Extrapolation for Non-Convex Optimization
Extrapolation is a well-known technique for solving convex optimization and
variational inequalities and recently attracts some attention for non-convex
optimization. Several recent works have empirically shown its success in some
machine learning tasks. However, it has not been analyzed for non-convex
minimization and there still remains a gap between the theory and the practice.
In this paper, we analyze gradient descent and stochastic gradient descent with
extrapolation for finding an approximate first-order stationary point in smooth
non-convex optimization problems. Our convergence upper bounds show that the
algorithms with extrapolation can be accelerated than without extrapolation
The Forward-Backward-Forward Method from continuous and discrete perspective for pseudo-monotone variational inequalities in Hilbert spaces
Tseng's forward-backward-forward algorithm is a valuable alternative for
Korpelevich's extragradient method when solving variational inequalities over a
convex and closed set governed by monotone and Lipschitz continuous operators,
as it requires in every step only one projection operation. However, it is
well-known that Korpelevich's method converges and can therefore be used also
for solving variational inequalities governed by pseudo-monotone and Lipschitz
continuous operators. In this paper, we first associate to a pseudo-monotone
variational inequality a forward-backward-forward dynamical system and carry
out an asymptotic analysis for the generated trajectories. The explicit time
discretization of this system results into Tseng's forward-backward-forward
algorithm with relaxation parameters, which we prove to converge also when it
is applied to pseudo-monotone variational inequalities. In addition, we show
that linear convergence is guaranteed under strong pseudo-monotonicity.
Numerical experiments are carried out for pseudo-monotone variational
inequalities over polyhedral sets and fractional programming problems
- …