21 research outputs found
On The Communication Complexity of Linear Algebraic Problems in the Message Passing Model
We study the communication complexity of linear algebraic problems over
finite fields in the multi-player message passing model, proving a number of
tight lower bounds. Specifically, for a matrix which is distributed among a
number of players, we consider the problem of determining its rank, of
computing entries in its inverse, and of solving linear equations. We also
consider related problems such as computing the generalized inner product of
vectors held on different servers. We give a general framework for reducing
these multi-player problems to their two-player counterparts, showing that the
randomized -player communication complexity of these problems is at least
times the randomized two-player communication complexity. Provided the
problem has a certain amount of algebraic symmetry, which we formally define,
we can show the hardest input distribution is a symmetric distribution, and
therefore apply a recent multi-player lower bound technique of Phillips et al.
Further, we give new two-player lower bounds for a number of these problems. In
particular, our optimal lower bound for the two-player version of the matrix
rank problem resolves an open question of Sun and Wang.
A common feature of our lower bounds is that they apply even to the special
"threshold promise" versions of these problems, wherein the underlying
quantity, e.g., rank, is promised to be one of just two values, one on each
side of some critical threshold. These kinds of promise problems are
commonplace in the literature on data streaming as sources of hardness for
reductions giving space lower bounds
Tribes Is Hard in the Message Passing Model
We consider the point-to-point message passing model of communication in
which there are processors with individual private inputs, each -bit
long. Each processor is located at the node of an underlying undirected graph
and has access to private random coins. An edge of the graph is a private
channel of communication between its endpoints. The processors have to compute
a given function of all their inputs by communicating along these channels.
While this model has been widely used in distributed computing, strong lower
bounds on the amount of communication needed to compute simple functions have
just begun to appear. In this work, we prove a tight lower bound of
on the communication needed for computing the Tribes function,
when the underlying graph is a star of nodes that has leaves with
inputs and a center with no input. Lower bound on this topology easily implies
comparable bounds for others. Our lower bounds are obtained by building upon
the recent information theoretic techniques of Braverman et.al (FOCS'13) and
combining it with the earlier work of Jayram, Kumar and Sivakumar (STOC'03).
This approach yields information complexity bounds that is of independent
interest
On the Distributed Complexity of Large-Scale Graph Computations
Motivated by the increasing need to understand the distributed algorithmic
foundations of large-scale graph computations, we study some fundamental graph
problems in a message-passing model for distributed computing where
machines jointly perform computations on graphs with nodes (typically, ). The input graph is assumed to be initially randomly partitioned among
the machines, a common implementation in many real-world systems.
Communication is point-to-point, and the goal is to minimize the number of
communication {\em rounds} of the computation.
Our main contribution is the {\em General Lower Bound Theorem}, a theorem
that can be used to show non-trivial lower bounds on the round complexity of
distributed large-scale data computations. The General Lower Bound Theorem is
established via an information-theoretic approach that relates the round
complexity to the minimal amount of information required by machines to solve
the problem. Our approach is generic and this theorem can be used in a
"cookbook" fashion to show distributed lower bounds in the context of several
problems, including non-graph problems. We present two applications by showing
(almost) tight lower bounds for the round complexity of two fundamental graph
problems, namely {\em PageRank computation} and {\em triangle enumeration}. Our
approach, as demonstrated in the case of PageRank, can yield tight lower bounds
for problems (including, and especially, under a stochastic partition of the
input) where communication complexity techniques are not obvious.
Our approach, as demonstrated in the case of triangle enumeration, can yield
stronger round lower bounds as well as message-round tradeoffs compared to
approaches that use communication complexity techniques
The Range of Topological Effects on Communication
We continue the study of communication cost of computing functions when
inputs are distributed among processors, each of which is located at one
vertex of a network/graph called a terminal. Every other node of the network
also has a processor, with no input. The communication is point-to-point and
the cost is the total number of bits exchanged by the protocol, in the worst
case, on all edges.
Chattopadhyay, Radhakrishnan and Rudra (FOCS'14) recently initiated a study
of the effect of topology of the network on the total communication cost using
tools from embeddings. Their techniques provided tight bounds for simple
functions like Element-Distinctness (ED), which depend on the 1-median of the
graph. This work addresses two other kinds of natural functions. We show that
for a large class of natural functions like Set-Disjointness the communication
cost is essentially times the cost of the optimal Steiner tree connecting
the terminals. Further, we show for natural composed functions like and , the naive protocols
suggested by their definition is optimal for general networks. Interestingly,
the bounds for these functions depend on more involved topological parameters
that are a combination of Steiner tree and 1-median costs.
To obtain our results, we use some new tools in addition to ones used in
Chattopadhyay et. al. These include (i) viewing the communication constraints
via a linear program; (ii) using tools from the theory of tree embeddings to
prove topology sensitive direct sum results that handle the case of composed
functions and (iii) representing the communication constraints of certain
problems as a family of collection of multiway cuts, where each multiway cut
simulates the hardness of computing the function on the star topology
On The Multiparty Communication Complexity of Testing Triangle-Freeness
In this paper we initiate the study of property testing in simultaneous and
non-simultaneous multi-party communication complexity, focusing on testing
triangle-freeness in graphs. We consider the model,
where we have players receiving private inputs, and a coordinator who
receives no input; the coordinator can communicate with all the players, but
the players cannot communicate with each other. In this model, we ask: if an
input graph is divided between the players, with each player receiving some of
the edges, how many bits do the players and the coordinator need to exchange to
determine if the graph is triangle-free, or from triangle-free?
For general communication protocols, we show that
bits are sufficient to test triangle-freeness in
graphs of size with average degree (the degree need not be known in
advance). For protocols, where there is only one
communication round, we give a protocol that uses bits
when and when ; here, again, the average degree does not need to be
known in advance. We show that for average degree , our simultaneous
protocol is asymptotically optimal up to logarithmic factors. For higher
degrees, we are not able to give lower bounds on testing triangle-freeness, but
we give evidence that the problem is hard by showing that finding an edge that
participates in a triangle is hard, even when promised that at least a constant
fraction of the edges must be removed in order to make the graph triangle-free.Comment: To Appear in PODC 201
Towards Tight Communication Lower Bounds for Distributed Optimisation
We consider a standard distributed optimisation setting where machines,
each holding a -dimensional function , aim to jointly minimise the sum
of the functions . This problem arises naturally in
large-scale distributed optimisation, where a standard solution is to apply
variants of (stochastic) gradient descent. We focus on the communication
complexity of this problem: our main result provides the first fully
unconditional bounds on total number of bits which need to be sent and received
by the machines to solve this problem under point-to-point communication,
within a given error-tolerance. Specifically, we show that total bits need to be communicated between the machines to find
an additive -approximation to the minimum of . The result holds for both deterministic and randomised algorithms, and,
importantly, requires no assumptions on the algorithm structure. The lower
bound is tight under certain restrictions on parameter values, and is matched
within constant factors for quadratic objectives by a new variant of quantised
gradient descent, which we describe and analyse. Our results bring over tools
from communication complexity to distributed optimisation, which has potential
for further applications