595 research outputs found
Riemannian Optimization via Frank-Wolfe Methods
We study projection-free methods for constrained Riemannian optimization. In
particular, we propose the Riemannian Frank-Wolfe (RFW) method. We analyze
non-asymptotic convergence rates of RFW to an optimum for (geodesically) convex
problems, and to a critical point for nonconvex objectives. We also present a
practical setting under which RFW can attain a linear convergence rate. As a
concrete example, we specialize Rfw to the manifold of positive definite
matrices and apply it to two tasks: (i) computing the matrix geometric mean
(Riemannian centroid); and (ii) computing the Bures-Wasserstein barycenter.
Both tasks involve geodesically convex interval constraints, for which we show
that the Riemannian "linear oracle" required by RFW admits a closed-form
solution; this result may be of independent interest. We further specialize RFW
to the special orthogonal group and show that here too, the Riemannian "linear
oracle" can be solved in closed form. Here, we describe an application to the
synchronization of data matrices (Procrustes problem). We complement our
theoretical results with an empirical comparison of Rfw against
state-of-the-art Riemannian optimization methods and observe that RFW performs
competitively on the task of computing Riemannian centroids.Comment: Under Review. Largely revised version, including an extended
experimental section and an application to the special orthogonal group and
the Procrustes proble
Simple Complexity Analysis of Simplified Direct Search
We consider the problem of unconstrained minimization of a smooth function in
the derivative-free setting using. In particular, we propose and study a
simplified variant of the direct search method (of direction type), which we
call simplified direct search (SDS). Unlike standard direct search methods,
which depend on a large number of parameters that need to be tuned, SDS depends
on a single scalar parameter only.
Despite relevant research activity in direct search methods spanning several
decades, complexity guarantees---bounds on the number of function evaluations
needed to find an approximate solution---were not established until very
recently. In this paper we give a surprisingly brief and unified analysis of
SDS for nonconvex, convex and strongly convex functions. We match the existing
complexity results for direct search in their dependence on the problem
dimension () and error tolerance (), but the overall bounds are
simpler, easier to interpret, and have better dependence on other problem
parameters. In particular, we show that for the set of directions formed by the
standard coordinate vectors and their negatives, the number of function
evaluations needed to find an -solution is (resp.
) for the problem of minimizing a convex (resp.
strongly convex) smooth function. In the nonconvex smooth case, the bound is
, with the goal being the reduction of the norm of the
gradient below .Comment: 21 pages, 5 algorithms, 1 tabl
- …