2 research outputs found
Global Optimality in Distributed Low-rank Matrix Factorization
We study the convergence of a variant of distributed gradient descent (DGD)
on a distributed low-rank matrix approximation problem wherein some
optimization variables are used for consensus (as in classical DGD) and some
optimization variables appear only locally at a single node in the network. We
term the resulting algorithm DGD+LOCAL. Using algorithmic connections to
gradient descent and geometric connections to the well-behaved landscape of the
centralized low-rank matrix approximation problem, we identify sufficient
conditions where DGD+LOCAL is guaranteed to converge with exact consensus to a
global minimizer of the original centralized problem. For the distributed
low-rank matrix approximation problem, these guarantees are stronger---in terms
of consensus and optimality---than what appear in the literature for classical
DGD and more general problems
Provable Bregman-divergence based Methods for Nonconvex and Non-Lipschitz Problems
The (global) Lipschitz smoothness condition is crucial in establishing the
convergence theory for most optimization methods. Unfortunately, most machine
learning and signal processing problems are not Lipschitz smooth. This
motivates us to generalize the concept of Lipschitz smoothness condition to the
relative smoothness condition, which is satisfied by any finite-order
polynomial objective function. Further, this work develops new
Bregman-divergence based algorithms that are guaranteed to converge to a
second-order stationary point for any relatively smooth problem. In addition,
the proposed optimization methods cover both the proximal alternating
minimization and the proximal alternating linearized minimization when we
specialize the Bregman divergence to the Euclidian distance. Therefore, this
work not only develops guaranteed optimization methods for non-Lipschitz smooth
problems but also solves an open problem of showing the second-order
convergence guarantees for these alternating minimization methods