611 research outputs found
Smoothing algorithms for nonsmooth and nonconvex minimization over the stiefel manifold
We consider a class of nonsmooth and nonconvex optimization problems over the
Stiefel manifold where the objective function is the summation of a nonconvex
smooth function and a nonsmooth Lipschitz continuous convex function composed
with an linear mapping. We propose three numerical algorithms for solving this
problem, by combining smoothing methods and some existing algorithms for smooth
optimization over the Stiefel manifold. In particular, we approximate the
aforementioned nonsmooth convex function by its Moreau envelope in our
smoothing methods, and prove that the Moreau envelope has many favorable
properties. Thanks to this and the scheme for updating the smoothing parameter,
we show that any accumulation point of the solution sequence generated by the
proposed algorithms is a stationary point of the original optimization problem.
Numerical experiments on building graph Fourier basis are conducted to
demonstrate the efficiency of the proposed algorithms.Comment: 22 page
Moreau Envelope ADMM for Decentralized Weakly Convex Optimization
This paper proposes a proximal variant of the alternating direction method of
multipliers (ADMM) for distributed optimization. Although the current versions
of ADMM algorithm provide promising numerical results in producing solutions
that are close to optimal for many convex and non-convex optimization problems,
it remains unclear if they can converge to a stationary point for weakly convex
and locally non-smooth functions. Through our analysis using the Moreau
envelope function, we demonstrate that MADM can indeed converge to a stationary
point under mild conditions. Our analysis also includes computing the bounds on
the amount of change in the dual variable update step by relating the gradient
of the Moreau envelope function to the proximal function. Furthermore, the
results of our numerical experiments indicate that our method is faster and
more robust than widely-used approaches
Strongly convex functions, Moreau envelopes and the generic nature of convex functions with strong minimizers
In this work, using Moreau envelopes, we define a complete metric for the set
of proper lower semicontinuous convex functions. Under this metric, the
convergence of each sequence of convex functions is epi-convergence. We show
that the set of strongly convex functions is dense but it is only of the first
category. On the other hand, it is shown that the set of convex functions with
strong minima is of the second category
- …