566 research outputs found
Riemannian Optimization via Frank-Wolfe Methods
We study projection-free methods for constrained Riemannian optimization. In
particular, we propose the Riemannian Frank-Wolfe (RFW) method. We analyze
non-asymptotic convergence rates of RFW to an optimum for (geodesically) convex
problems, and to a critical point for nonconvex objectives. We also present a
practical setting under which RFW can attain a linear convergence rate. As a
concrete example, we specialize Rfw to the manifold of positive definite
matrices and apply it to two tasks: (i) computing the matrix geometric mean
(Riemannian centroid); and (ii) computing the Bures-Wasserstein barycenter.
Both tasks involve geodesically convex interval constraints, for which we show
that the Riemannian "linear oracle" required by RFW admits a closed-form
solution; this result may be of independent interest. We further specialize RFW
to the special orthogonal group and show that here too, the Riemannian "linear
oracle" can be solved in closed form. Here, we describe an application to the
synchronization of data matrices (Procrustes problem). We complement our
theoretical results with an empirical comparison of Rfw against
state-of-the-art Riemannian optimization methods and observe that RFW performs
competitively on the task of computing Riemannian centroids.Comment: Under Review. Largely revised version, including an extended
experimental section and an application to the special orthogonal group and
the Procrustes proble
Random gradient-free minimization of convex functions
In this paper, we prove the complexity bounds for methods of Convex Optimization based only on computation of the function value. The search directions of our schemes are normally distributed random Gaussian vectors. It appears that such methods usually need at most n times more iterations than the standard gradient methods, where n is the dimension of the space of variables. This conclusion is true both for nonsmooth and smooth problems. For the later class, we present also an accelerated scheme with the expected rate of convergence O(n[ exp ]2 /k[ exp ]2), where k is the iteration counter. For Stochastic Optimization, we propose a zero-order scheme and justify its expected rate of convergence O(n/k[ exp ]1/2). We give also some bounds for the rate of convergence of the random gradient-free methods to stationary points of nonconvex functions, both for smooth and nonsmooth cases. Our theoretical results are supported by preliminary computational experiments.convex optimization, stochastic optimization, derivative-free methods, random methods, complexity bounds
Extended Formulations in Mixed-integer Convex Programming
We present a unifying framework for generating extended formulations for the
polyhedral outer approximations used in algorithms for mixed-integer convex
programming (MICP). Extended formulations lead to fewer iterations of outer
approximation algorithms and generally faster solution times. First, we observe
that all MICP instances from the MINLPLIB2 benchmark library are conic
representable with standard symmetric and nonsymmetric cones. Conic
reformulations are shown to be effective extended formulations themselves
because they encode separability structure. For mixed-integer
conic-representable problems, we provide the first outer approximation
algorithm with finite-time convergence guarantees, opening a path for the use
of conic solvers for continuous relaxations. We then connect the popular
modeling framework of disciplined convex programming (DCP) to the existence of
extended formulations independent of conic representability. We present
evidence that our approach can yield significant gains in practice, with the
solution of a number of open instances from the MINLPLIB2 benchmark library.Comment: To be presented at IPCO 201
- …