35,433 research outputs found
On Minimizing Data-read and Download for Storage-Node Recovery
We consider the problem of efficient recovery of the data stored in any
individual node of a distributed storage system, from the rest of the nodes.
Applications include handling failures and degraded reads. We measure
efficiency in terms of the amount of data-read and the download required. To
minimize the download, we focus on the minimum bandwidth setting of the
'regenerating codes' model for distributed storage. Under this model, the
system has a total of n nodes, and the data stored in any node must be
(efficiently) recoverable from any d of the other (n-1) nodes. Lower bounds on
the two metrics under this model were derived previously; it has also been
shown that these bounds are achievable for the amount of data-read and download
when d=n-1, and for the amount of download alone when d<n-1.
In this paper, we complete this picture by proving the converse result, that
when d<n-1, these lower bounds are strictly loose with respect to the amount of
read required. The proof is information-theoretic, and hence applies to
non-linear codes as well. We also show that under two (practical) relaxations
of the problem setting, these lower bounds can be met for both read and
download simultaneously.Comment: IEEE Communications Letter
On Characterizing the Data Movement Complexity of Computational DAGs for Parallel Execution
Technology trends are making the cost of data movement increasingly dominant,
both in terms of energy and time, over the cost of performing arithmetic
operations in computer systems. The fundamental ratio of aggregate data
movement bandwidth to the total computational power (also referred to the
machine balance parameter) in parallel computer systems is decreasing. It is
there- fore of considerable importance to characterize the inherent data
movement requirements of parallel algorithms, so that the minimal architectural
balance parameters required to support it on future systems can be well
understood. In this paper, we develop an extension of the well-known red-blue
pebble game to develop lower bounds on the data movement complexity for the
parallel execution of computational directed acyclic graphs (CDAGs) on parallel
systems. We model multi-node multi-core parallel systems, with the total
physical memory distributed across the nodes (that are connected through some
interconnection network) and in a multi-level shared cache hierarchy for
processors within a node. We also develop new techniques for lower bound
characterization of non-homogeneous CDAGs. We demonstrate the use of the
methodology by analyzing the CDAGs of several numerical algorithms, to develop
lower bounds on data movement for their parallel execution
On the Efficiency of Classical and Quantum Secure Function Evaluation
We provide bounds on the efficiency of secure one-sided output two-party
computation of arbitrary finite functions from trusted distributed randomness
in the statistical case. From these results we derive bounds on the efficiency
of protocols that use different variants of OT as a black-box. When applied to
implementations of OT, these bounds generalize most known results to the
statistical case. Our results hold in particular for transformations between a
finite number of primitives and for any error. In the second part we study the
efficiency of quantum protocols implementing OT. While most classical lower
bounds for perfectly secure reductions of OT to distributed randomness still
hold in the quantum setting, we present a statistically secure protocol that
violates these bounds by an arbitrarily large factor. We then prove a weaker
lower bound that does hold in the statistical quantum setting and implies that
even quantum protocols cannot extend OT. Finally, we present two lower bounds
for reductions of OT to commitments and a protocol based on string commitments
that is optimal with respect to both of these bounds
Equivalence Classes and Conditional Hardness in Massively Parallel Computations
The Massively Parallel Computation (MPC) model serves as a common abstraction of many modern large-scale data processing frameworks, and has been receiving increasingly more attention over the past few years, especially in the context of classical graph problems. So far, the only way to argue lower bounds for this model is to condition on conjectures about the hardness of some specific problems, such as graph connectivity on promise graphs that are either one cycle or two cycles, usually called the one cycle vs. two cycles problem. This is unlike the traditional arguments based on conjectures about complexity classes (e.g., P ? NP), which are often more robust in the sense that refuting them would lead to groundbreaking algorithms for a whole bunch of problems.
In this paper we present connections between problems and classes of problems that allow the latter type of arguments. These connections concern the class of problems solvable in a sublogarithmic amount of rounds in the MPC model, denoted by MPC(o(log N)), and some standard classes concerning space complexity, namely L and NL, and suggest conjectures that are robust in the sense that refuting them would lead to many surprisingly fast new algorithms in the MPC model. We also obtain new conditional lower bounds, and prove new reductions and equivalences between problems in the MPC model
On the Complexity of List Ranking in the Parallel External Memory Model
We study the problem of list ranking in the parallel external memory (PEM)
model. We observe an interesting dual nature for the hardness of the problem
due to limited information exchange among the processors about the structure of
the list, on the one hand, and its close relationship to the problem of
permuting data, which is known to be hard for the external memory models, on
the other hand.
By carefully defining the power of the computational model, we prove a
permuting lower bound in the PEM model. Furthermore, we present a stronger
\Omega(log^2 N) lower bound for a special variant of the problem and for a
specific range of the model parameters, which takes us a step closer toward
proving a non-trivial lower bound for the list ranking problem in the
bulk-synchronous parallel (BSP) and MapReduce models. Finally, we also present
an algorithm that is tight for a larger range of parameters of the model than
in prior work
Route Planning in Transportation Networks
We survey recent advances in algorithms for route planning in transportation
networks. For road networks, we show that one can compute driving directions in
milliseconds or less even at continental scale. A variety of techniques provide
different trade-offs between preprocessing effort, space requirements, and
query time. Some algorithms can answer queries in a fraction of a microsecond,
while others can deal efficiently with real-time traffic. Journey planning on
public transportation systems, although conceptually similar, is a
significantly harder problem due to its inherent time-dependent and
multicriteria nature. Although exact algorithms are fast enough for interactive
queries on metropolitan transit systems, dealing with continent-sized instances
requires simplifications or heavy preprocessing. The multimodal route planning
problem, which seeks journeys combining schedule-based transportation (buses,
trains) with unrestricted modes (walking, driving), is even harder, relying on
approximate solutions even for metropolitan inputs.Comment: This is an updated version of the technical report MSR-TR-2014-4,
previously published by Microsoft Research. This work was mostly done while
the authors Daniel Delling, Andrew Goldberg, and Renato F. Werneck were at
Microsoft Research Silicon Valle
- …