116 research outputs found
Equivalence Classes and Conditional Hardness in Massively Parallel Computations
The Massively Parallel Computation (MPC) model serves as a common abstraction of many modern large-scale data processing frameworks, and has been receiving increasingly more attention over the past few years, especially in the context of classical graph problems. So far, the only way to argue lower bounds for this model is to condition on conjectures about the hardness of some specific problems, such as graph connectivity on promise graphs that are either one cycle or two cycles, usually called the one cycle vs. two cycles problem. This is unlike the traditional arguments based on conjectures about complexity classes (e.g., P ? NP), which are often more robust in the sense that refuting them would lead to groundbreaking algorithms for a whole bunch of problems.
In this paper we present connections between problems and classes of problems that allow the latter type of arguments. These connections concern the class of problems solvable in a sublogarithmic amount of rounds in the MPC model, denoted by MPC(o(log N)), and some standard classes concerning space complexity, namely L and NL, and suggest conjectures that are robust in the sense that refuting them would lead to many surprisingly fast new algorithms in the MPC model. We also obtain new conditional lower bounds, and prove new reductions and equivalences between problems in the MPC model
On the Distributed Complexity of Large-Scale Graph Computations
Motivated by the increasing need to understand the distributed algorithmic
foundations of large-scale graph computations, we study some fundamental graph
problems in a message-passing model for distributed computing where
machines jointly perform computations on graphs with nodes (typically, ). The input graph is assumed to be initially randomly partitioned among
the machines, a common implementation in many real-world systems.
Communication is point-to-point, and the goal is to minimize the number of
communication {\em rounds} of the computation.
Our main contribution is the {\em General Lower Bound Theorem}, a theorem
that can be used to show non-trivial lower bounds on the round complexity of
distributed large-scale data computations. The General Lower Bound Theorem is
established via an information-theoretic approach that relates the round
complexity to the minimal amount of information required by machines to solve
the problem. Our approach is generic and this theorem can be used in a
"cookbook" fashion to show distributed lower bounds in the context of several
problems, including non-graph problems. We present two applications by showing
(almost) tight lower bounds for the round complexity of two fundamental graph
problems, namely {\em PageRank computation} and {\em triangle enumeration}. Our
approach, as demonstrated in the case of PageRank, can yield tight lower bounds
for problems (including, and especially, under a stochastic partition of the
input) where communication complexity techniques are not obvious.
Our approach, as demonstrated in the case of triangle enumeration, can yield
stronger round lower bounds as well as message-round tradeoffs compared to
approaches that use communication complexity techniques
A Lower Bound Technique for Communication in BSP
Communication is a major factor determining the performance of algorithms on
current computing systems; it is therefore valuable to provide tight lower
bounds on the communication complexity of computations. This paper presents a
lower bound technique for the communication complexity in the bulk-synchronous
parallel (BSP) model of a given class of DAG computations. The derived bound is
expressed in terms of the switching potential of a DAG, that is, the number of
permutations that the DAG can realize when viewed as a switching network. The
proposed technique yields tight lower bounds for the fast Fourier transform
(FFT), and for any sorting and permutation network. A stronger bound is also
derived for the periodic balanced sorting network, by applying this technique
to suitable subnetworks. Finally, we demonstrate that the switching potential
captures communication requirements even in computational models different from
BSP, such as the I/O model and the LPRAM
Matching on the Line Admits No o(?log n)-Competitive Algorithm
We present a simple proof that the competitive ratio of any randomized online matching algorithm for the line exceeds ?{log?(n +1)}/15 for all n = 2?-1: i ? ?, settling a 25-year-old open question
Communication Lower Bounds for Distributed-Memory Computations
In this paper we propose a new approach to the study of the communication requirements of distributed computations, which advocates for the removal of the restrictive assumptions under which earlier results were derived. We illustrate our approach by giving tight lower bounds on the communication complexity required to solve several computational problems in a distributed-memory parallel machine, namely standard matrix multiplication, stencil computations, comparison sorting, and the Fast Fourier Transform. Our bounds rely only on a mild assumption on work distribution, and significantly strengthen previous results which require either the computation to be balanced among the processors, or specific initial distributions of the input data, or an upper bound on the size of processors\u27 local memories
A time- and message-optimal distributed algorithm for minimum spanning trees
This paper presents a randomized Las Vegas distributed algorithm that
constructs a minimum spanning tree (MST) in weighted networks with optimal (up
to polylogarithmic factors) time and message complexity. This algorithm runs in
time and exchanges messages (both with
high probability), where is the number of nodes of the network, is the
diameter, and is the number of edges. This is the first distributed MST
algorithm that matches \emph{simultaneously} the time lower bound of
[Elkin, SIAM J. Comput. 2006] and the message
lower bound of [Kutten et al., J.ACM 2015] (which both apply to
randomized algorithms).
The prior time and message lower bounds are derived using two completely
different graph constructions; the existing lower bound construction that shows
one lower bound {\em does not} work for the other. To complement our algorithm,
we present a new lower bound graph construction for which any distributed MST
algorithm requires \emph{both} rounds and
messages
Fast Distributed Algorithms for Connectivity and MST in Large Graphs
Motivated by the increasing need to understand the algorithmic foundations of
distributed large-scale graph computations, we study a number of fundamental
graph problems in a message-passing model for distributed computing where machines jointly perform computations on graphs with nodes
(typically, ). The input graph is assumed to be initially randomly
partitioned among the machines, a common implementation in many real-world
systems. Communication is point-to-point, and the goal is to minimize the
number of communication rounds of the computation.
Our main result is an (almost) optimal distributed randomized algorithm for
graph connectivity. Our algorithm runs in rounds
( notation hides a \poly\log(n) factor and an additive
\poly\log(n) term). This improves over the best previously known bound of
[Klauck et al., SODA 2015], and is optimal (up to a
polylogarithmic factor) in view of an existing lower bound of
. Our improved algorithm uses a bunch of techniques,
including linear graph sketching, that prove useful in the design of efficient
distributed graph algorithms. Using the connectivity algorithm as a building
block, we then present fast randomized algorithms for computing minimum
spanning trees, (approximate) min-cuts, and for many graph verification
problems. All these algorithms take rounds, and are optimal
up to polylogarithmic factors. We also show an almost matching lower bound of
rounds for many graph verification problems by
leveraging lower bounds in random-partition communication complexity
On Characterizing the Data Access Complexity of Programs
Technology trends will cause data movement to account for the majority of
energy expenditure and execution time on emerging computers. Therefore,
computational complexity will no longer be a sufficient metric for comparing
algorithms, and a fundamental characterization of data access complexity will
be increasingly important. The problem of developing lower bounds for data
access complexity has been modeled using the formalism of Hong & Kung's
red/blue pebble game for computational directed acyclic graphs (CDAGs).
However, previously developed approaches to lower bounds analysis for the
red/blue pebble game are very limited in effectiveness when applied to CDAGs of
real programs, with computations comprised of multiple sub-computations with
differing DAG structure. We address this problem by developing an approach for
effectively composing lower bounds based on graph decomposition. We also
develop a static analysis algorithm to derive the asymptotic data-access lower
bounds of programs, as a function of the problem size and cache size
- …