7 research outputs found
Stochastic Frank-Wolfe for Composite Convex Minimization
A broad class of convex optimization problems can be formulated as a
semidefinite program (SDP), minimization of a convex function over the
positive-semidefinite cone subject to some affine constraints. The majority of
classical SDP solvers are designed for the deterministic setting where problem
data is readily available. In this setting, generalized conditional gradient
methods (aka Frank-Wolfe-type methods) provide scalable solutions by leveraging
the so-called linear minimization oracle instead of the projection onto the
semidefinite cone. Most problems in machine learning and modern engineering
applications, however, contain some degree of stochasticity. In this work, we
propose the first conditional-gradient-type method for solving stochastic
optimization problems under affine constraints. Our method guarantees
convergence rate in expectation on the objective
residual and on the feasibility gap
Approximate Frank-Wolfe Algorithms over Graph-structured Support Sets
In this paper, we propose approximate Frank-Wolfe (FW) algorithms to solve
convex optimization problems over graph-structured support sets where the
\textit{linear minimization oracle} (LMO) cannot be efficiently obtained in
general. We first demonstrate that two popular approximation assumptions
(\textit{additive} and \textit{multiplicative gap errors)}, are not valid for
our problem, in that no cheap gap-approximate LMO oracle exists in general.
Instead, a new \textit{approximate dual maximization oracle} (DMO) is proposed,
which approximates the inner product rather than the gap. When the objective is
-smooth, we prove that the standard FW method using a -approximate
DMO converges as in general, and as over a
-relaxation of the constraint set. Additionally, when the objective is
-strongly convex and the solution is unique, a variant of FW converges to
with the same per-iteration
complexity. Our empirical results suggest that even these improved bounds are
pessimistic, with significant improvement in recovering real-world images with
graph-structured sparsity.Comment: 30 pages, 8 figure
A Stochastic Sequential Quadratic Optimization Algorithm for Nonlinear Equality Constrained Optimization with Rank-Deficient Jacobians
A sequential quadratic optimization algorithm is proposed for solving smooth
nonlinear equality constrained optimization problems in which the objective
function is defined by an expectation of a stochastic function. The algorithmic
structure of the proposed method is based on a step decomposition strategy that
is known in the literature to be widely effective in practice, wherein each
search direction is computed as the sum of a normal step (toward linearized
feasibility) and a tangential step (toward objective decrease in the null space
of the constraint Jacobian). However, the proposed method is unique from others
in the literature in that it both allows the use of stochastic objective
gradient estimates and possesses convergence guarantees even in the setting in
which the constraint Jacobians may be rank deficient. The results of numerical
experiments demonstrate that the algorithm offers superior performance when
compared to popular alternatives
Stochastic Frank-Wolfe for Composite Convex Minimization
A broad class of convex optimization problems can be formulated as a semidefinite program (SDP), minimization of a convex function over the positive-semidefinite cone subject to some affine constraints. The majority of classical SDP solvers are designed for the deterministic setting where problem data is readily available. In this setting, generalized conditional gradient methods (aka Frank-Wolfe-type methods) provide scalable solutions by leveraging the so-called linear minimization oracle instead of the projection onto the semidefinite cone. Most problems in machine learning and modern engineering applications, however, contain some degree of stochasticity. In this work, we propose the first conditional-gradient-type method for solving stochastic optimization problems under affine constraints. Our method guarantees O(k(-1/3)) convergence rate in expectation on the objective residual and O(k (5/12)) on the feasibility gap