2 research outputs found
Riemannian Stochastic Proximal Gradient Methods for Nonsmooth Optimization over the Stiefel Manifold
Riemannian optimization has drawn a lot of attention due to its wide
applications in practice. Riemannian stochastic first-order algorithms have
been studied in the literature to solve large-scale machine learning problems
over Riemannian manifolds. However, most of the existing Riemannian stochastic
algorithms require the objective function to be differentiable, and they do not
apply to the case where the objective function is nonsmooth. In this paper, we
present two Riemannian stochastic proximal gradient methods for minimizing
nonsmooth function over the Stiefel manifold. The two methods, named R-ProxSGD
and R-ProxSPB, are generalizations of proximal SGD and proximal SpiderBoost in
Euclidean setting to the Riemannian setting. Analysis on the incremental
first-order oracle (IFO) complexity of the proposed algorithms is provided.
Specifically, the R-ProxSPB algorithm finds an -stationary point with
IFOs in the online case, and
IFOs in the finite-sum case with
being the number of summands in the objective. Experimental results on online
sparse PCA and robust low-rank matrix completion show that our proposed methods
significantly outperform the existing methods that uses Riemannian subgradient
information
Curvature-Dependant Global Convergence Rates for Optimization on Manifolds of Bounded Geometry
We give curvature-dependant convergence rates for the optimization of weakly
convex functions defined on a manifold of 1-bounded geometry via Riemannian
gradient descent and via the dynamic trivialization algorithm. In order to do
this, we give a tighter bound on the norm of the Hessian of the Riemannian
exponential than the previously known. We compute these bounds explicitly for
some manifolds commonly used in the optimization literature such as the special
orthogonal group and the real Grassmannian. Along the way, we present
self-contained proofs of fully general bounds on the norm of the differential
of the exponential map and certain cosine inequalities on manifolds, which are
commonly used in optimization on manifolds