2,412 research outputs found
Theoretically Efficient Parallel Graph Algorithms Can Be Fast and Scalable
There has been significant recent interest in parallel graph processing due
to the need to quickly analyze the large graphs available today. Many graph
codes have been designed for distributed memory or external memory. However,
today even the largest publicly-available real-world graph (the Hyperlink Web
graph with over 3.5 billion vertices and 128 billion edges) can fit in the
memory of a single commodity multicore server. Nevertheless, most experimental
work in the literature report results on much smaller graphs, and the ones for
the Hyperlink graph use distributed or external memory. Therefore, it is
natural to ask whether we can efficiently solve a broad class of graph problems
on this graph in memory.
This paper shows that theoretically-efficient parallel graph algorithms can
scale to the largest publicly-available graphs using a single machine with a
terabyte of RAM, processing them in minutes. We give implementations of
theoretically-efficient parallel algorithms for 20 important graph problems. We
also present the optimizations and techniques that we used in our
implementations, which were crucial in enabling us to process these large
graphs quickly. We show that the running times of our implementations
outperform existing state-of-the-art implementations on the largest real-world
graphs. For many of the problems that we consider, this is the first time they
have been solved on graphs at this scale. We have made the implementations
developed in this work publicly-available as the Graph-Based Benchmark Suite
(GBBS).Comment: This is the full version of the paper appearing in the ACM Symposium
on Parallelism in Algorithms and Architectures (SPAA), 201
Space and Time Efficient Parallel Graph Decomposition, Clustering, and Diameter Approximation
We develop a novel parallel decomposition strategy for unweighted, undirected
graphs, based on growing disjoint connected clusters from batches of centers
progressively selected from yet uncovered nodes. With respect to similar
previous decompositions, our strategy exercises a tighter control on both the
number of clusters and their maximum radius.
We present two important applications of our parallel graph decomposition:
(1) -center clustering approximation; and (2) diameter approximation. In
both cases, we obtain algorithms which feature a polylogarithmic approximation
factor and are amenable to a distributed implementation that is geared for
massive (long-diameter) graphs. The total space needed for the computation is
linear in the problem size, and the parallel depth is substantially sublinear
in the diameter for graphs with low doubling dimension. To the best of our
knowledge, ours are the first parallel approximations for these problems which
achieve sub-diameter parallel time, for a relevant class of graphs, using only
linear space. Besides the theoretical guarantees, our algorithms allow for a
very simple implementation on clustered architectures: we report on extensive
experiments which demonstrate their effectiveness and efficiency on large
graphs as compared to alternative known approaches.Comment: 14 page
Fast Shortest Path Distance Estimation in Large Networks
We study the problem of preprocessing a large graph so that point-to-point shortest-path queries can be answered very fast. Computing shortest paths is a well studied problem, but exact algorithms do not scale to huge graphs encountered on the web, social networks, and other applications.
In this paper we focus on approximate methods for distance estimation, in particular using landmark-based distance indexing. This approach involves selecting a subset of nodes as landmarks and computing (offline) the distances from each node in the graph to those landmarks. At runtime, when the distance between a pair of nodes is needed, we can estimate it quickly by combining the precomputed distances of the two nodes to the landmarks.
We prove that selecting the optimal set of landmarks is an NP-hard problem, and thus heuristic solutions need to be employed. Given a budget of memory for the index, which translates directly into a budget of landmarks, different landmark selection strategies can yield dramatically different results in terms of accuracy. A number of simple methods that scale well to large graphs are therefore developed and experimentally compared. The simplest methods choose central nodes of the graph, while the more elaborate ones select central nodes that are also far away from one another. The efficiency of the suggested techniques is tested experimentally using five different real world graphs with millions of edges; for a given accuracy, they require as much as 250 times less space than the current approach in the literature which considers selecting landmarks at random.
Finally, we study applications of our method in two problems arising naturally in large-scale networks, namely, social search and community detection.Yahoo! Research (internship
04301 Abstracts Collection -- Cache-Oblivious and Cache-Aware Algorithms
The Dagstuhl Seminar 04301 ``Cache-Oblivious and Cache-Aware Algorithms\u27\u27 was held
in the International Conference and Research Center (IBFI), Schloss Dagstuhl, from 18.07.2004 to 23.07.2004.
During the seminar, several participants presented their current
research, and ongoing work and open problems were discussed. Abstracts of
the presentations given during the seminar as well as abstracts of
seminar results and ideas are put together in this paper. The first section
describes the seminar topics and goals in general.
Links to extended abstracts or full papers are provided, if available
Exact Distance Oracles for Planar Graphs
We present new and improved data structures that answer exact node-to-node
distance queries in planar graphs. Such data structures are also known as
distance oracles. For any directed planar graph on n nodes with non-negative
lengths we obtain the following:
* Given a desired space allocation , we show how to
construct in time a data structure of size that answers
distance queries in time per query.
As a consequence, we obtain an improvement over the fastest algorithm for
k-many distances in planar graphs whenever .
* We provide a linear-space exact distance oracle for planar graphs with
query time for any constant eps>0. This is the first such data
structure with provable sublinear query time.
* For edge lengths at least one, we provide an exact distance oracle of space
such that for any pair of nodes at distance D the query time is
. Comparable query performance had been observed
experimentally but has never been explained theoretically.
Our data structures are based on the following new tool: given a
non-self-crossing cycle C with nodes, we can preprocess G in
time to produce a data structure of size that can
answer the following queries in time: for a query node u, output
the distance from u to all the nodes of C. This data structure builds on and
extends a related data structure of Klein (SODA'05), which reports distances to
the boundary of a face, rather than a cycle.
The best distance oracles for planar graphs until the current work are due to
Cabello (SODA'06), Djidjev (WG'96), and Fakcharoenphol and Rao (FOCS'01). For
and space , we essentially improve the query
time from to .Comment: To appear in the proceedings of the 23rd ACM-SIAM Symposium on
Discrete Algorithms, SODA 201
Efficient Algorithms and Data Structures for Massive Data Sets
For many algorithmic problems, traditional algorithms that optimise on the
number of instructions executed prove expensive on I/Os. Novel and very
different design techniques, when applied to these problems, can produce
algorithms that are I/O efficient. This thesis adds to the growing chorus of
such results. The computational models we use are the external memory model and
the W-Stream model.
On the external memory model, we obtain the following results. (1) An I/O
efficient algorithm for computing minimum spanning trees of graphs that
improves on the performance of the best known algorithm. (2) The first external
memory version of soft heap, an approximate meldable priority queue. (3) Hard
heap, the first meldable external memory priority queue that matches the
amortised I/O performance of the known external memory priority queues, while
allowing a meld operation at the same amortised cost. (4) I/O efficient exact,
approximate and randomised algorithms for the minimum cut problem, which has
not been explored before on the external memory model. (5) Some lower and upper
bounds on I/Os for interval graphs.
On the W-Stream model, we obtain the following results. (1) Algorithms for
various tree problems and list ranking that match the performance of the best
known algorithms and are easier to implement than them. (2) Pass efficient
algorithms for sorting, and the maximal independent set problems, that improve
on the best known algorithms. (3) Pass efficient algorithms for the graphs
problems of finding vertex-colouring, approximate single source shortest paths,
maximal matching, and approximate weighted vertex cover. (4) Lower bounds on
passes for list ranking and maximal matching.
We propose two variants of the W-Stream model, and design algorithms for the
maximal independent set, vertex-colouring, and planar graph single source
shortest paths problems on those models.Comment: PhD Thesis (144 pages
A Practical Parallel Algorithm for Diameter Approximation of Massive Weighted Graphs
We present a space and time efficient practical parallel algorithm for
approximating the diameter of massive weighted undirected graphs on distributed
platforms supporting a MapReduce-like abstraction. The core of the algorithm is
a weighted graph decomposition strategy generating disjoint clusters of bounded
weighted radius. Theoretically, our algorithm uses linear space and yields a
polylogarithmic approximation guarantee; moreover, for important practical
classes of graphs, it runs in a number of rounds asymptotically smaller than
those required by the natural approximation provided by the state-of-the-art
-stepping SSSP algorithm, which is its only practical linear-space
competitor in the aforementioned computational scenario. We complement our
theoretical findings with an extensive experimental analysis on large benchmark
graphs, which demonstrates that our algorithm attains substantial improvements
on a number of key performance indicators with respect to the aforementioned
competitor, while featuring a similar approximation ratio (a small constant
less than 1.4, as opposed to the polylogarithmic theoretical bound)
- âŠ