1,950 research outputs found
Networked Signal and Information Processing
The article reviews significant advances in networked signal and information
processing, which have enabled in the last 25 years extending decision making
and inference, optimization, control, and learning to the increasingly
ubiquitous environments of distributed agents. As these interacting agents
cooperate, new collective behaviors emerge from local decisions and actions.
Moreover, and significantly, theory and applications show that networked
agents, through cooperation and sharing, are able to match the performance of
cloud or federated solutions, while offering the potential for improved
privacy, increasing resilience, and saving resources
A Coordinate Descent Primal-Dual Algorithm and Application to Distributed Asynchronous Optimization
Based on the idea of randomized coordinate descent of -averaged
operators, a randomized primal-dual optimization algorithm is introduced, where
a random subset of coordinates is updated at each iteration. The algorithm
builds upon a variant of a recent (deterministic) algorithm proposed by V\~u
and Condat that includes the well known ADMM as a particular case. The obtained
algorithm is used to solve asynchronously a distributed optimization problem. A
network of agents, each having a separate cost function containing a
differentiable term, seek to find a consensus on the minimum of the aggregate
objective. The method yields an algorithm where at each iteration, a random
subset of agents wake up, update their local estimates, exchange some data with
their neighbors, and go idle. Numerical results demonstrate the attractive
performance of the method. The general approach can be naturally adapted to
other situations where coordinate descent convex optimization algorithms are
used with a random choice of the coordinates.Comment: 10 page
Networked signal and information processing
The article reviews significant advances in networked signal and information processing, which have enabled in the last 25 years extending decision making and inference, optimization, control, and learning to the increasingly ubiquitous environments of distributed agents. As these interacting agents cooperate, new collective behaviors emerge from local decisions and actions. Moreover, and significantly, theory and applications show that networked agents, through cooperation and sharing, are able to match the performance of cloud or federated solutions, while offering the potential for improved privacy, increasing resilience, and saving resources
On the Influence of Bias-Correction on Distributed Stochastic Optimization
Various bias-correction methods such as EXTRA, gradient tracking methods, and
exact diffusion have been proposed recently to solve distributed {\em
deterministic} optimization problems. These methods employ constant step-sizes
and converge linearly to the {\em exact} solution under proper conditions.
However, their performance under stochastic and adaptive settings is less
explored. It is still unknown {\em whether}, {\em when} and {\em why} these
bias-correction methods can outperform their traditional counterparts (such as
consensus and diffusion) with noisy gradient and constant step-sizes.
This work studies the performance of exact diffusion under the stochastic and
adaptive setting, and provides conditions under which exact diffusion has
superior steady-state mean-square deviation (MSD) performance than traditional
algorithms without bias-correction. In particular, it is proven that this
superiority is more evident over sparsely-connected network topologies such as
lines, cycles, or grids. Conditions are also provided under which exact
diffusion method match or may even degrade the performance of traditional
methods. Simulations are provided to validate the theoretical findings.Comment: 17 pages, 9 figure, submitted for publicatio
- …