193 research outputs found
Smoothing algorithms for nonsmooth and nonconvex minimization over the stiefel manifold
We consider a class of nonsmooth and nonconvex optimization problems over the
Stiefel manifold where the objective function is the summation of a nonconvex
smooth function and a nonsmooth Lipschitz continuous convex function composed
with an linear mapping. We propose three numerical algorithms for solving this
problem, by combining smoothing methods and some existing algorithms for smooth
optimization over the Stiefel manifold. In particular, we approximate the
aforementioned nonsmooth convex function by its Moreau envelope in our
smoothing methods, and prove that the Moreau envelope has many favorable
properties. Thanks to this and the scheme for updating the smoothing parameter,
we show that any accumulation point of the solution sequence generated by the
proposed algorithms is a stationary point of the original optimization problem.
Numerical experiments on building graph Fourier basis are conducted to
demonstrate the efficiency of the proposed algorithms.Comment: 22 page
A Riemannian ADMM
We consider a class of Riemannian optimization problems where the objective
is the sum of a smooth function and a nonsmooth function, considered in the
ambient space. This class of problems finds important applications in machine
learning and statistics such as the sparse principal component analysis, sparse
spectral clustering, and orthogonal dictionary learning. We propose a
Riemannian alternating direction method of multipliers (ADMM) to solve this
class of problems. Our algorithm adopts easily computable steps in each
iteration. The iteration complexity of the proposed algorithm for obtaining an
-stationary point is analyzed under mild assumptions. To the best of
our knowledge, this is the first Riemannian ADMM with provable convergence
guarantee for solving Riemannian optimization problem with nonsmooth objective.
Numerical experiments are conducted to demonstrate the advantage of the
proposed method
- …