127 research outputs found
Projection-free nonconvex stochastic optimization on Riemannian manifolds
We study stochastic projection-free methods for constrained optimization of
smooth functions on Riemannian manifolds, i.e., with additional constraints
beyond the parameter domain being a manifold. Specifically, we introduce
stochastic Riemannian Frank-Wolfe methods for nonconvex and geodesically convex
problems. We present algorithms for both purely stochastic optimization and
finite-sum problems. For the latter, we develop variance-reduced methods,
including a Riemannian adaptation of the recently proposed Spider technique.
For all settings, we recover convergence rates that are comparable to the
best-known rates for their Euclidean counterparts. Finally, we discuss
applications to two classic tasks: The computation of the Karcher mean of
positive definite matrices and Wasserstein barycenters for multivariate normal
distributions. For both tasks, stochastic Fw methods yield state-of-the-art
empirical performance.Comment: Under Revie
Modified memoryless spectral-scaling Broyden family on Riemannian manifolds
This paper presents modified memoryless quasi-Newton methods based on the
spectral-scaling Broyden family on Riemannian manifolds. The method involves
adding one parameter to the search direction of the memoryless self-scaling
Broyden family on the manifold. Moreover, it uses a general map instead of
vector transport. This idea has already been proposed within a general
framework of Riemannian conjugate gradient methods where one can use vector
transport, scaled vector transport, or an inverse retraction. We show that the
search direction satisfies the sufficient descent condition under some
assumptions on the parameters. In addition, we show global convergence of the
proposed method under the Wolfe conditions. We numerically compare it with
existing methods, including Riemannian conjugate gradient methods and the
memoryless spectral-scaling Broyden family. The numerical results indicate that
the proposed method with the BFGS formula is suitable for solving an
off-diagonal cost function minimization problem on an oblique manifold.Comment: 20 pages, 8 figure
- …