115 research outputs found
Solving variational inequalities with Stochastic Mirror-Prox algorithm
In this paper we consider iterative methods for stochastic variational
inequalities (s.v.i.) with monotone operators. Our basic assumption is that the
operator possesses both smooth and nonsmooth components. Further, only noisy
observations of the problem data are available. We develop a novel Stochastic
Mirror-Prox (SMP) algorithm for solving s.v.i. and show that with the
convenient stepsize strategy it attains the optimal rates of convergence with
respect to the problem parameters. We apply the SMP algorithm to Stochastic
composite minimization and describe particular applications to Stochastic
Semidefinite Feasability problem and Eigenvalue minimization
Convex optimization over intersection of simple sets: improved convergence rate guarantees via an exact penalty approach
We consider the problem of minimizing a convex function over the intersection
of finitely many simple sets which are easy to project onto. This is an
important problem arising in various domains such as machine learning. The main
difficulty lies in finding the projection of a point in the intersection of
many sets. Existing approaches yield an infeasible point with an
iteration-complexity of for nonsmooth problems with no
guarantees on the in-feasibility. By reformulating the problem through exact
penalty functions, we derive first-order algorithms which not only guarantees
that the distance to the intersection is small but also improve the complexity
to and for smooth functions. For
composite and smooth problems, this is achieved through a saddle-point
reformulation where the proximal operators required by the primal-dual
algorithms can be computed in closed form. We illustrate the benefits of our
approach on a graph transduction problem and on graph matching
Semi-proximal Mirror-Prox for Nonsmooth Composite Minimization
We propose a new first-order optimisation algorithm to solve high-dimensional
non-smooth composite minimisation problems. Typical examples of such problems
have an objective that decomposes into a non-smooth empirical risk part and a
non-smooth regularisation penalty. The proposed algorithm, called Semi-Proximal
Mirror-Prox, leverages the Fenchel-type representation of one part of the
objective while handling the other part of the objective via linear
minimization over the domain. The algorithm stands in contrast with more
classical proximal gradient algorithms with smoothing, which require the
computation of proximal operators at each iteration and can therefore be
impractical for high-dimensional problems. We establish the theoretical
convergence rate of Semi-Proximal Mirror-Prox, which exhibits the optimal
complexity bounds, i.e. , for the number of calls to linear
minimization oracle. We present promising experimental results showing the
interest of the approach in comparison to competing methods
Decomposition in conic optimization with partially separable structure
Decomposition techniques for linear programming are difficult to extend to
conic optimization problems with general non-polyhedral convex cones because
the conic inequalities introduce an additional nonlinear coupling between the
variables. However in many applications the convex cones have a partially
separable structure that allows them to be characterized in terms of simpler
lower-dimensional cones. The most important example is sparse semidefinite
programming with a chordal sparsity pattern. Here partial separability derives
from the clique decomposition theorems that characterize positive semidefinite
and positive-semidefinite-completable matrices with chordal sparsity patterns.
The paper describes a decomposition method that exploits partial separability
in conic linear optimization. The method is based on Spingarn's method for
equality constrained convex optimization, combined with a fast interior-point
method for evaluating proximal operators
Sparse Non Gaussian Component Analysis by Semidefinite Programming
Sparse non-Gaussian component analysis (SNGCA) is an unsupervised method of
extracting a linear structure from a high dimensional data based on estimating
a low-dimensional non-Gaussian data component. In this paper we discuss a new
approach to direct estimation of the projector on the target space based on
semidefinite programming which improves the method sensitivity to a broad
variety of deviations from normality. We also discuss the procedures which
allows to recover the structure when its effective dimension is unknown
- …