21 research outputs found
Distributed Learning with Sparse Communications by Identification
In distributed optimization for large-scale learning, a major performance
limitation comes from the communications between the different entities. When
computations are performed by workers on local data while a coordinator machine
coordinates their updates to minimize a global loss, we present an asynchronous
optimization algorithm that efficiently reduces the communications between the
coordinator and workers. This reduction comes from a random sparsification of
the local updates. We show that this algorithm converges linearly in the
strongly convex case and also identifies optimal strongly sparse solutions. We
further exploit this identification to propose an automatic dimension
reduction, aptly sparsifying all exchanges between coordinator and workers.Comment: v2 is a significant improvement over v1 (titled "Asynchronous
Distributed Learning with Sparse Communications and Identification") with new
algorithms, results, and discussion
A generic coordinate descent solver for nonsmooth convex optimization
International audienceWe present a generic coordinate descent solver for the minimization of a nonsmooth convex objective with structure. The method can deal in particular with problems with linear constraints. The implementation makes use of efficient residual updates and automatically determines which dual variables should be duplicated. A list of basic functional atoms is pre-compiled for efficiency and a modelling language in Python allows the user to combine them at run time. So, the algorithm can be used to solve a large variety of problems including Lasso, sparse multinomial logistic regression, linear and quadratic programs