790 research outputs found
Gossip Dual Averaging for Decentralized Optimization of Pairwise Functions
In decentralized networks (of sensors, connected objects, etc.), there is an
important need for efficient algorithms to optimize a global cost function, for
instance to learn a global model from the local data collected by each
computing unit. In this paper, we address the problem of decentralized
minimization of pairwise functions of the data points, where these points are
distributed over the nodes of a graph defining the communication topology of
the network. This general problem finds applications in ranking, distance
metric learning and graph inference, among others. We propose new gossip
algorithms based on dual averaging which aims at solving such problems both in
synchronous and asynchronous settings. The proposed framework is flexible
enough to deal with constrained and regularized variants of the optimization
problem. Our theoretical analysis reveals that the proposed algorithms preserve
the convergence rate of centralized dual averaging up to an additive bias term.
We present numerical simulations on Area Under the ROC Curve (AUC) maximization
and metric learning problems which illustrate the practical interest of our
approach
Cooperative Convex Optimization in Networked Systems: Augmented Lagrangian Algorithms with Directed Gossip Communication
We study distributed optimization in networked systems, where nodes cooperate
to find the optimal quantity of common interest, x=x^\star. The objective
function of the corresponding optimization problem is the sum of private (known
only by a node,) convex, nodes' objectives and each node imposes a private
convex constraint on the allowed values of x. We solve this problem for generic
connected network topologies with asymmetric random link failures with a novel
distributed, decentralized algorithm. We refer to this algorithm as AL-G
(augmented Lagrangian gossiping,) and to its variants as AL-MG (augmented
Lagrangian multi neighbor gossiping) and AL-BG (augmented Lagrangian broadcast
gossiping.) The AL-G algorithm is based on the augmented Lagrangian dual
function. Dual variables are updated by the standard method of multipliers, at
a slow time scale. To update the primal variables, we propose a novel,
Gauss-Seidel type, randomized algorithm, at a fast time scale. AL-G uses
unidirectional gossip communication, only between immediate neighbors in the
network and is resilient to random link failures. For networks with reliable
communication (i.e., no failures,) the simplified, AL-BG (augmented Lagrangian
broadcast gossiping) algorithm reduces communication, computation and data
storage cost. We prove convergence for all proposed algorithms and demonstrate
by simulations the effectiveness on two applications: l_1-regularized logistic
regression for classification and cooperative spectrum sensing for cognitive
radio networks.Comment: 28 pages, journal; revise
Gossip Algorithms for Distributed Signal Processing
Gossip algorithms are attractive for in-network processing in sensor networks
because they do not require any specialized routing, there is no bottleneck or
single point of failure, and they are robust to unreliable wireless network
conditions. Recently, there has been a surge of activity in the computer
science, control, signal processing, and information theory communities,
developing faster and more robust gossip algorithms and deriving theoretical
performance guarantees. This article presents an overview of recent work in the
area. We describe convergence rate results, which are related to the number of
transmitted messages and thus the amount of energy consumed in the network for
gossiping. We discuss issues related to gossiping over wireless links,
including the effects of quantization and noise, and we illustrate the use of
gossip algorithms for canonical signal processing tasks including distributed
estimation, source localization, and compression.Comment: Submitted to Proceedings of the IEEE, 29 page
Geographic Gossip: Efficient Averaging for Sensor Networks
Gossip algorithms for distributed computation are attractive due to their
simplicity, distributed nature, and robustness in noisy and uncertain
environments. However, using standard gossip algorithms can lead to a
significant waste in energy by repeatedly recirculating redundant information.
For realistic sensor network model topologies like grids and random geometric
graphs, the inefficiency of gossip schemes is related to the slow mixing times
of random walks on the communication graph. We propose and analyze an
alternative gossiping scheme that exploits geographic information. By utilizing
geographic routing combined with a simple resampling method, we demonstrate
substantial gains over previously proposed gossip protocols. For regular graphs
such as the ring or grid, our algorithm improves standard gossip by factors of
and respectively. For the more challenging case of random
geometric graphs, our algorithm computes the true average to accuracy
using radio
transmissions, which yields a factor improvement over
standard gossip algorithms. We illustrate these theoretical results with
experimental comparisons between our algorithm and standard methods as applied
to various classes of random fields.Comment: To appear, IEEE Transactions on Signal Processin
Autonomous Demand Side Management Based on Energy Consumption Scheduling and Instantaneous Load Billing: An Aggregative Game Approach
In this paper, we investigate a practical demand side management scenario
where the selfish consumers compete to minimize their individual energy cost
through scheduling their future energy consumption profiles. We propose an
instantaneous load billing scheme to effectively convince the consumers to
shift their peak-time consumption and to fairly charge the consumers for their
energy consumption. For the considered DSM scenario, an aggregative game is
first formulated to model the strategic behaviors of the selfish consumers. By
resorting to the variational inequality theory, we analyze the conditions for
the existence and uniqueness of the Nash equilibrium (NE) of the formulated
game. Subsequently, for the scenario where there is a central unit calculating
and sending the real-time aggregated load to all consumers, we develop a one
timescale distributed iterative proximal-point algorithm with provable
convergence to achieve the NE of the formulated game. Finally, considering the
alternative situation where the central unit does not exist, but the consumers
are connected and they would like to share their estimated information with
others, we present a distributed agreement-based algorithm, by which the
consumers can achieve the NE of the formulated game through exchanging
information with their immediate neighbors.Comment: 11 pages, 7 figure
- âŠ