305 research outputs found
A Variance-Reduced Stochastic Gradient Tracking Algorithm for Decentralized Optimization with Orthogonality Constraints
Decentralized optimization with orthogonality constraints is found widely in
scientific computing and data science. Since the orthogonality constraints are
nonconvex, it is quite challenging to design efficient algorithms. Existing
approaches leverage the geometric tools from Riemannian optimization to solve
this problem at the cost of high sample and communication complexities. To
relieve this difficulty, based on two novel techniques that can waive the
orthogonality constraints, we propose a variance-reduced stochastic gradient
tracking (VRSGT) algorithm with the convergence rate of to a
stationary point. To the best of our knowledge, VRSGT is the first algorithm
for decentralized optimization with orthogonality constraints that reduces both
sampling and communication complexities simultaneously. In the numerical
experiments, VRSGT has a promising performance in a real-world autonomous
driving application
- …