2,906 research outputs found
Dynamic Algorithms for the Massively Parallel Computation Model
The Massive Parallel Computing (MPC) model gained popularity during the last
decade and it is now seen as the standard model for processing large scale
data. One significant shortcoming of the model is that it assumes to work on
static datasets while, in practice, real-world datasets evolve continuously. To
overcome this issue, in this paper we initiate the study of dynamic algorithms
in the MPC model.
We first discuss the main requirements for a dynamic parallel model and we
show how to adapt the classic MPC model to capture them. Then we analyze the
connection between classic dynamic algorithms and dynamic algorithms in the MPC
model. Finally, we provide new efficient dynamic MPC algorithms for a variety
of fundamental graph problems, including connectivity, minimum spanning tree
and matching.Comment: Accepted to the 31st ACM Symposium on Parallelism in Algorithms and
Architectures (SPAA 2019
Streaming Graph Challenge: Stochastic Block Partition
An important objective for analyzing real-world graphs is to achieve scalable
performance on large, streaming graphs. A challenging and relevant example is
the graph partition problem. As a combinatorial problem, graph partition is
NP-hard, but existing relaxation methods provide reasonable approximate
solutions that can be scaled for large graphs. Competitive benchmarks and
challenges have proven to be an effective means to advance state-of-the-art
performance and foster community collaboration. This paper describes a graph
partition challenge with a baseline partition algorithm of sub-quadratic
complexity. The algorithm employs rigorous Bayesian inferential methods based
on a statistical model that captures characteristics of the real-world graphs.
This strong foundation enables the algorithm to address limitations of
well-known graph partition approaches such as modularity maximization. This
paper describes various aspects of the challenge including: (1) the data sets
and streaming graph generator, (2) the baseline partition algorithm with
pseudocode, (3) an argument for the correctness of parallelizing the Bayesian
inference, (4) different parallel computation strategies such as node-based
parallelism and matrix-based parallelism, (5) evaluation metrics for partition
correctness and computational requirements, (6) preliminary timing of a
Python-based demonstration code and the open source C++ code, and (7)
considerations for partitioning the graph in streaming fashion. Data sets and
source code for the algorithm as well as metrics, with detailed documentation
are available at GraphChallenge.org.Comment: To be published in 2017 IEEE High Performance Extreme Computing
Conference (HPEC
Input-Dynamic Distributed Algorithms for Communication Networks
Consider a distributed task where the communication network is fixed but the
local inputs given to the nodes of the distributed system may change over time.
In this work, we explore the following question: if some of the local inputs
change, can an existing solution be updated efficiently, in a dynamic and
distributed manner?
To address this question, we define the batch dynamic CONGEST model in which
we are given a bandwidth-limited communication network and a dynamic edge
labelling defines the problem input. The task is to maintain a solution to a
graph problem on the labeled graph under batch changes. We investigate, when a
batch of edge label changes arrive,
-- how much time as a function of we need to update an existing
solution, and
-- how much information the nodes have to keep in local memory between
batches in order to update the solution quickly.
Our work lays the foundations for the theory of input-dynamic distributed
network algorithms. We give a general picture of the complexity landscape in
this model, design both universal algorithms and algorithms for concrete
problems, and present a general framework for lower bounds. In particular, we
derive non-trivial upper bounds for two selected, contrasting problems:
maintaining a minimum spanning tree and detecting cliques
- …