195,004 research outputs found
Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling
The goal of decentralized optimization over a network is to optimize a global
objective formed by a sum of local (possibly nonsmooth) convex functions using
only local computation and communication. It arises in various application
domains, including distributed tracking and localization, multi-agent
co-ordination, estimation in sensor networks, and large-scale optimization in
machine learning. We develop and analyze distributed algorithms based on dual
averaging of subgradients, and we provide sharp bounds on their convergence
rates as a function of the network size and topology. Our method of analysis
allows for a clear separation between the convergence of the optimization
algorithm itself and the effects of communication constraints arising from the
network structure. In particular, we show that the number of iterations
required by our algorithm scales inversely in the spectral gap of the network.
The sharpness of this prediction is confirmed both by theoretical lower bounds
and simulations for various networks. Our approach includes both the cases of
deterministic optimization and communication, as well as problems with
stochastic optimization and/or communication.Comment: 40 pages, 4 figure
Guest Editorial: Nonlinear Optimization of Communication Systems
Linear programming and other classical optimization techniques have found important applications in communication systems for many decades. Recently, there has been a surge in research activities that utilize the latest developments in nonlinear optimization to tackle a much wider scope of work in the analysis and design of communication systems. These activities involve every “layer” of the protocol stack and the principles of layered network architecture itself, and have made intellectual and practical impacts significantly beyond the established frameworks of optimization of communication systems in the early 1990s. These recent results are driven by new demands in the areas of communications and networking, as well as new tools emerging from optimization theory. Such tools include the powerful theories and highly efficient computational algorithms for nonlinear convex optimization, together with global solution methods and relaxation techniques for nonconvex optimization
- …