13,176 research outputs found
Differentially Private Consensus-Based Distributed Optimization
Data privacy is an important concern in learning, when datasets contain
sensitive information about individuals. This paper considers consensus-based
distributed optimization under data privacy constraints. Consensus-based
optimization consists of a set of computational nodes arranged in a graph, each
having a local objective that depends on their local data, where in every step
nodes take a linear combination of their neighbors' messages, as well as taking
a new gradient step. Since the algorithm requires exchanging messages that
depend on local data, private information gets leaked at every step. Taking
-differential privacy (DP) as our criterion, we consider
the strategy where the nodes add random noise to their messages before
broadcasting it, and show that the method achieves convergence with a bounded
mean-squared error, while satisfying -DP. By relaxing the
more stringent -DP requirement in previous work, we strengthen a
known convergence result in the literature. We conclude the paper with
numerical results demonstrating the effectiveness of our methods for mean
estimation
Local Thresholding on Distributed Hash Tables
We present a binary routing tree protocol for distributed hash table
overlays. Using this protocol each peer can independently route messages to its
parent and two descendants on the fly without any maintenance, global context,
and synchronization. The protocol is then extended to support tree change
notification with similar efficiency. The resulting tree is almost perfectly
dense and balanced, and has O(1) stretch if the distributed hash table is
symmetric Chord. We use the tree routing protocol to overcome the main
impediment for implementation of local thresholding algorithms in peer-to-peer
systems -- their requirement for cycle free routing. Direct comparison of a
gossip-based algorithm and a corresponding local thresholding algorithm on a
majority voting problem reveals that the latter obtains superior accuracy using
a fraction of the communication overhead
Toward Creating Subsurface Camera
In this article, the framework and architecture of Subsurface Camera (SAMERA)
is envisioned and described for the first time. A SAMERA is a geophysical
sensor network that senses and processes geophysical sensor signals, and
computes a 3D subsurface image in-situ in real-time. The basic mechanism is:
geophysical waves propagating/reflected/refracted through subsurface enter a
network of geophysical sensors, where a 2D or 3D image is computed and
recorded; a control software may be connected to this network to allow view of
the 2D/3D image and adjustment of settings such as resolution, filter,
regularization and other algorithm parameters. System prototypes based on
seismic imaging have been designed. SAMERA technology is envisioned as a game
changer to transform many subsurface survey and monitoring applications,
including oil/gas exploration and production, subsurface infrastructures and
homeland security, wastewater and CO2 sequestration, earthquake and volcano
hazard monitoring. The system prototypes for seismic imaging have been built.
Creating SAMERA requires an interdisciplinary collaboration and transformation
of sensor networks, signal processing, distributed computing, and geophysical
imaging.Comment: 15 pages, 7 figure
Consensus Needs Broadcast in Noiseless Models but can be Exponentially Easier in the Presence of Noise
Consensus and Broadcast are two fundamental problems in distributed
computing, whose solutions have several applications. Intuitively, Consensus
should be no harder than Broadcast, and this can be rigorously established in
several models. Can Consensus be easier than Broadcast?
In models that allow noiseless communication, we prove a reduction of (a
suitable variant of) Broadcast to binary Consensus, that preserves the
communication model and all complexity parameters such as randomness, number of
rounds, communication per round, etc., while there is a loss in the success
probability of the protocol. Using this reduction, we get, among other
applications, the first logarithmic lower bound on the number of rounds needed
to achieve Consensus in the uniform GOSSIP model on the complete graph. The
lower bound is tight and, in this model, Consensus and Broadcast are
equivalent.
We then turn to distributed models with noisy communication channels that
have been studied in the context of some bio-inspired systems. In such models,
only one noisy bit is exchanged when a communication channel is established
between two nodes, and so one cannot easily simulate a noiseless protocol by
using error-correcting codes. An lower bound on the
number of rounds needed for Broadcast is proved by Boczkowski et al. [PLOS
Comp. Bio. 2018] in one such model (noisy uniform PULL, where is a
parameter that measures the amount of noise). In such model, we prove a new
bound for Broadcast and a
bound for binary Consensus, thus establishing an
exponential gap between the number of rounds necessary for Consensus versus
Broadcast
Naming Games in Two-Dimensional and Small-World-Connected Random Geometric Networks
We investigate a prototypical agent-based model, the Naming Game, on
two-dimensional random geometric networks. The Naming Game [A. Baronchelli et
al., J. Stat. Mech.: Theory Exp. (2006) P06014.] is a minimal model, employing
local communications that captures the emergence of shared communication
schemes (languages) in a population of autonomous semiotic agents. Implementing
the Naming Games with local broadcasts on random geometric graphs, serves as a
model for agreement dynamics in large-scale, autonomously operating wireless
sensor networks. Further, it captures essential features of the scaling
properties of the agreement process for spatially-embedded autonomous agents.
Among the relevant observables capturing the temporal properties of the
agreement process, we investigate the cluster-size distribution and the
distribution of the agreement times, both exhibiting dynamic scaling. We also
present results for the case when a small density of long-range communication
links are added on top of the random geometric graph, resulting in a
"small-world"-like network and yielding a significantly reduced time to reach
global agreement. We construct a finite-size scaling analysis for the agreement
times in this case
DPCrowd: Privacy-preserving and Communication-efficient Decentralized Statistical Estimation for Real-time Crowd-sourced Data
In Internet of Things (IoT) driven smart-world systems, real-time
crowd-sourced databases from multiple distributed servers can be aggregated to
extract dynamic statistics from a larger population, thus providing more
reliable knowledge for our society. Particularly, multiple distributed servers
in a decentralized network can realize real-time collaborative statistical
estimation by disseminating statistics from their separate databases. Despite
no raw data sharing, the real-time statistics could still expose the data
privacy of crowd-sourcing participants. For mitigating the privacy concern,
while traditional differential privacy (DP) mechanism can be simply implemented
to perturb the statistics in each timestamp and independently for each
dimension, this may suffer a great utility loss from the real-time and
multi-dimensional crowd-sourced data. Also, the real-time broadcasting would
bring significant overheads in the whole network. To tackle the issues, we
propose a novel privacy-preserving and communication-efficient decentralized
statistical estimation algorithm (DPCrowd), which only requires intermittently
sharing the DP protected parameters with one-hop neighbors by exploiting the
temporal correlations in real-time crowd-sourced data. Then, with further
consideration of spatial correlations, we develop an enhanced algorithm,
DPCrowd+, to deal with multi-dimensional infinite crowd-data streams. Extensive
experiments on several datasets demonstrate that our proposed schemes DPCrowd
and DPCrowd+ can significantly outperform existing schemes in providing
accurate and consensus estimation with rigorous privacy protection and great
communication efficiency
CIoTA: Collaborative IoT Anomaly Detection via Blockchain
Due to their rapid growth and deployment, Internet of things (IoT) devices
have become a central aspect of our daily lives. However, they tend to have
many vulnerabilities which can be exploited by an attacker. Unsupervised
techniques, such as anomaly detection, can help us secure the IoT devices.
However, an anomaly detection model must be trained for a long time in order to
capture all benign behaviors. This approach is vulnerable to adversarial
attacks since all observations are assumed to be benign while training the
anomaly detection model.
In this paper, we propose CIoTA, a lightweight framework that utilizes the
blockchain concept to perform distributed and collaborative anomaly detection
for devices with limited resources. CIoTA uses blockchain to incrementally
update a trusted anomaly detection model via self-attestation and consensus
among IoT devices. We evaluate CIoTA on our own distributed IoT simulation
platform, which consists of 48 Raspberry Pis, to demonstrate CIoTA's ability to
enhance the security of each device and the security of the network as a whole.Comment: Appears in the workshop on Decentralized IoT Security and Standards
(DISS) of the Network and Distributed Systems Security Symposium (NDSS) 201
Leveraging Communication Topologies Between Learning Agents in Deep Reinforcement Learning
A common technique to improve learning performance in deep reinforcement
learning (DRL) and many other machine learning algorithms is to run multiple
learning agents in parallel. A neglected component in the development of these
algorithms has been how best to arrange the learning agents involved to improve
distributed search. Here we draw upon results from the networked optimization
literatures suggesting that arranging learning agents in communication networks
other than fully connected topologies (the implicit way agents are commonly
arranged in) can improve learning. We explore the relative performance of four
popular families of graphs and observe that one such family (Erdos-Renyi random
graphs) empirically outperforms the de facto fully-connected communication
topology across several DRL benchmark tasks. Additionally, we observe that 1000
learning agents arranged in an Erdos-Renyi graph can perform as well as 3000
agents arranged in the standard fully-connected topology, showing the large
learning improvement possible when carefully designing the topology over which
agents communicate. We complement these empirical results with a theoretical
investigation of why our alternate topologies perform better. Overall, our work
suggests that distributed machine learning algorithms could be made more
effective if the communication topology between learning agents was optimized.Comment: arXiv admin note: substantial text overlap with arXiv:1811.1255
Naming Games in Spatially-Embedded Random Networks
We investigate a prototypical agent-based model, the Naming Game, on random
geometric networks. The Naming Game is a minimal model, employing local
communications that captures the emergence of shared communication schemes
(languages) in a population of autonomous semiotic agents. Implementing the
Naming Games on random geometric graphs, local communications being local
broadcasts, serves as a model for agreement dynamics in large-scale,
autonomously operating wireless sensor networks. Further, it captures essential
features of the scaling properties of the agreement process for
spatially-embedded autonomous agents. We also present results for the case when
a small density of long-range communication links are added on top of the
random geometric graph, resulting in a "small-world"-like network and yielding
a significantly reduced time to reach global agreement.Comment: We have found a programming error in our code used to generate the
results of the earlier version. We have corrected the error, reran all
simulations, and regenerated all data plots. While the qualitative behavior
of the model has not changed, the numerical values of some of the scaling
exponents did. 7 figure
Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs
We present a simple neural rendering architecture that helps variational
autoencoders (VAEs) learn disentangled representations. Instead of the
deconvolutional network typically used in the decoder of VAEs, we tile
(broadcast) the latent vector across space, concatenate fixed X- and
Y-"coordinate" channels, and apply a fully convolutional network with 1x1
stride. This provides an architectural prior for dissociating positional from
non-positional features in the latent distribution of VAEs, yet without
providing any explicit supervision to this effect. We show that this
architecture, which we term the Spatial Broadcast decoder, improves
disentangling, reconstruction accuracy, and generalization to held-out regions
in data space. It provides a particularly dramatic benefit when applied to
datasets with small objects. We also emphasize a method for visualizing learned
latent spaces that helped us diagnose our models and may prove useful for
others aiming to assess data representations. Finally, we show the Spatial
Broadcast Decoder is complementary to state-of-the-art (SOTA) disentangling
techniques and when incorporated improves their performance
- …