179 research outputs found
Design and analysis of sequential and parallel single-source shortest-paths algorithms
We study the performance of algorithms for the Single-Source Shortest-Paths (SSSP) problem on graphs with n nodes and m edges with nonnegative random weights. All previously known SSSP algorithms for directed graphs required superlinear time. Wie give the first SSSP algorithms that provably achieve linear O(n-m)average-case execution time on arbitrary directed graphs with random edge weights. For independent edge weights, the linear-time bound holds with high probability, too. Additionally, our result implies improved average-case bounds for the All-Pairs Shortest-Paths (APSP) problem on sparse graphs, and it yields the first theoretical average-case analysis for the "Approximate Bucket Implementation" of Dijkstra\u27s SSSP algorithm (ABI-Dijkstra). Futhermore, we give constructive proofs for the existence of graph classes with random edge weights on which ABI-Dijkstra and several other well-known SSSP algorithms require superlinear average-case time. Besides the classical sequential (single processor) model of computation we also consider parallel computing: we give the currently fastest average-case linear-work parallel SSSP algorithms for large graph classes with random edge weights, e.g., sparse rondom graphs and graphs modeling the WWW, telephone calls or social networks.In dieser Arbeit untersuchen wir die Laufzeiten von Algorithmen für das Kürzeste-Wege Problem (Single-Source Shortest-Paths, SSSP) auf Graphen mit n Knoten, M Kanten und nichtnegativen zufälligen Kantengewichten. Alle bisherigen SSSP Algorithmen benötigen auf gerichteten Graphen superlineare Zeit. Wir stellen den ersten SSSP Algorithmus vor, der auf beliebigen gerichteten Graphen mit zufälligen Kantengewichten eine beweisbar lineare average-case-Komplexität
O(n+m)aufweist. Sind die Kantengewichte unabhängig, so wird die lineare Zeitschranke auch mit hoher Wahrscheinlichkeit eingehalten. Außerdem impliziert unser Ergebnis verbesserte average-case-Schranken für das All-Pairs Shortest-Paths (APSP) Problem auf dünnen Graphen und liefert die erste theoretische average-case-Analyse für die "Approximate Bucket Implementierung" von Dijkstras SSSP Algorithmus (ABI-Dijkstra). Weiterhin führen wir konstruktive Existenzbeweise für Graphklassen mit zufälligen Kantengewichten, auf denen ABI-Dijkstra und mehrere andere bekannte SSSP Algorithmen durchschnittlich superlineare Zeit benötigen. Neben dem klassischen seriellen (Ein-Prozessor) Berechnungsmodell betrachten wir auch Parallelverarbeitung; für umfangreiche Graphklassen mit zufälligen Kantengewichten wie z.B. dünne Zufallsgraphen oder Modelle für das WWW, Telefonanrufe oder soziale Netzwerke stellen wir die derzeit schnellsten parallelen SSSP Algorithmen mit durchschnittlich linearer Arbeit vor
Parallelizing Sequential Programs With Statistical Accuracy Tests
We present QuickStep, a novel system for parallelizing sequential programs. QuickStep deploys a set of parallelization transformations that together induce a search space of candidate parallel programs. Given a sequential program, representative inputs, and an accuracy requirement, QuickStep uses performance measurements, profiling information, and statistical accuracy tests on the outputs of candidate parallel programs to guide its search for a parallelizationthat maximizes performance while preserving acceptable accuracy. When the search completes, QuickStep produces an interactive report that summarizes the applied parallelization transformations, performance, and accuracy results for the automatically generated candidate parallel programs. In our envisioned usage scenarios, the developer examines this report to evaluate the acceptability of the final parallelization and to obtain insight into how the original sequential program responds to different parallelization strategies. Itis also possible for the developer (or even a user of the program who has no software development expertise whatsoever) to simply use the best parallelization out of the box without examining the report or further investigating the parallelization. Results from our benchmark set of applications show that QuickStep can automatically generate accurate and efficient parallel programs---the automatically generated parallel versions of five of our six benchmark applications run between 5.0 and 7.7 times faster on 8 cores than the original sequential versions. Moreover, a comparison with the Intel icc compiler highlights how QuickStep can effectively parallelize applications with features (such as the use of modern object-oriented programming constructs or desirable parallelizations with infrequent but acceptable data races) that place them inherently beyond the reach of standard approaches
Black-Box Parallelization for Machine Learning
The landscape of machine learning applications is changing rapidly: large centralized datasets are replaced by high volume, high velocity data streams generated by a vast number of geographically distributed, loosely connected devices, such as mobile phones, smart sensors, autonomous vehicles or industrial machines. Current learning approaches centralize the data and process it in parallel in a cluster or computing center. This has three major disadvantages: (i) it does not scale well with the number of data-generating devices since their growth exceeds that of computing centers, (ii) the communication costs for centralizing the data are prohibitive in many applications, and (iii) it requires sharing potentially privacy-sensitive data. Pushing computation towards the data-generating devices alleviates these problems and allows to employ their otherwise unused computing power. However, current parallel learning approaches are designed for tightly integrated systems with low latency and high bandwidth, not for loosely connected distributed devices. Therefore, I propose a new paradigm for parallelization that treats the learning algorithm as a black box, training local models on distributed devices and aggregating them into a single strong one. Since this requires only exchanging models instead of actual data, the approach is highly scalable, communication-efficient, and privacy-preserving. Following this paradigm, this thesis develops black-box parallelizations for two broad classes of learning algorithms. One approach can be applied to incremental learning algorithms, i.e., those that improve a model in iterations. Based on the utility of aggregations it schedules communication dynamically, adapting it to the hardness of the learning problem. In practice, this leads to a reduction in communication by orders of magnitude. It is analyzed for (i) online learning, in particular in the context of in-stream learning, which allows to guarantee optimal regret and for (ii) batch learning based on empirical risk minimization where optimal convergence can be guaranteed. The other approach is applicable to non-incremental algorithms as well. It uses a novel aggregation method based on the Radon point that allows to achieve provably high model quality with only a single aggregation. This is achieved in polylogarithmic runtime on quasi-polynomially many processors. This relates parallel machine learning to Nick's class of parallel decision problems and is a step towards answering a fundamental open problem about the abilities and limitations of efficient parallel learning algorithms. An empirical study on real distributed systems confirms the potential of the approaches in realistic application scenarios
A parallel Heap-Cell Method for Eikonal equations
Numerous applications of Eikonal equations prompted the development of many
efficient numerical algorithms. The Heap-Cell Method (HCM) is a recent serial
two-scale technique that has been shown to have advantages over other serial
state-of-the-art solvers for a wide range of problems. This paper presents a
parallelization of HCM for a shared memory architecture. The numerical
experiments in show that the parallel HCM exhibits good algorithmic
behavior and scales well, resulting in a very fast and practical solver.
We further explore the influence on performance and scaling of data
precision, early termination criteria, and the hardware architecture. A shorter
version of this manuscript (omitting these more detailed tests) has been
submitted to SIAM Journal on Scientific Computing in 2012.Comment: (a minor update to address the reviewers' comments) 31 pages; 15
figures; this is an expanded version of a paper accepted by SIAM Journal on
Scientific Computin
Data distribution and performance optimization models for parallel data mining
Ankara : The Department of Computer Engineering and the Graduate School of Engineering and Science of Bilkent University, 2013.Thesis (Ph. D.) -- Bilkent University, 2013.Includes bibliographical references leaves 117-128.We have embarked upon a multitude of approaches to improve the efficiency of
selected fundamental tasks in data mining. The present thesis is concerned with
improving the efficiency of parallel processing methods for large amounts of data.
We have devised new parallel frequent itemset mining algorithms that work on
both sparse and dense datasets, and 1-D and 2-D parallel algorithms for the
all-pairs similarity problem.
Two new parallel frequent itemset mining (FIM) algorithms named NoClique
and NoClique2 parallelize our sequential vertical frequent itemset mining algorithm
named bitdrill, and uses a method based on graph partitioning by vertex
separator (GPVS) to distribute and selectively replicate items. The method operates
on a graph where vertices correspond to frequent items and edges correspond
to frequent itemsets of size two. We show that partitioning this graph by a vertex
separator is sufficient to decide a distribution of the items such that the
sub-databases determined by the item distribution can be mined independently.
This distribution entails an amount of data replication, which may be reduced
by setting appropriate weights to vertices. The data distribution scheme is used
in the design of two new parallel frequent itemset mining algorithms. Both algorithms
replicate the items that correspond to the separator. NoClique replicates
the work induced by the separator and NoClique2 computes the same work collectively.
Computational load balancing and minimization of redundant or collective
work may be achieved by assigning appropriate load estimates to vertices. The
performance is compared to another parallelization that replicates all items, and
ParDCI algorithm. We introduce another parallel FIM method using a variation of item distribution
with selective item replication. We extend the GPVS model for parallel
FIM we have proposed earlier, by relaxing the condition of independent mining.
Instead of finding independently mined item sets, we may minimize the amount of
communication and partition the candidates in a fine-grained manner. We introduce
a hypergraph partitioning model of the parallel computation where vertices
correspond to candidates and hyperedges correspond to items. A load estimate is
assigned to each candidate with vertex weights, and item frequencies are given as
hyperedge weights. The model is shown to minimize data replication and balance
load accurately. We also introduce a re-partitioning model since we can generate
only so many levels of candidates at once, using fixed vertices to model previous
item distribution/replication. Experiments show that we improve over the higher
load imbalance of NoClique2 algorithm for the same problem instances at the
cost of additional parallel overhead.
For the all-pairs similarity problem, we extend recent efficient sequential algorithms
to a parallel setting, and obtain document-wise and term-wise parallelizations
of a fast sequential algorithm, as well as an elegant combination of two
algorithms that yield a 2-D distribution of the data. Two effective algorithmic
optimizations for the term-wise case are reported that make the term-wise parallelization
feasible. These optimizations exploit local pruning and block processing
of a number of vectors, in order to decrease communication costs, the number of
candidates, and communication/computation imbalance. The correctness of local
pruning is proven. Also, a recursive term-wise parallelization is introduced. The
performance of the algorithms are shown to be favorable in extensive experiments,
as well as the utility of two major optimizations.Ă–zkural, ErayPh.D
-
We study the performance of algorithms for the Single-Source Shortest-Paths (SSSP) problem on graphs with n nodes and m edges with nonnegative random weights. All previously known SSSP algorithms for directed graphs required superlinear time. Wie give the first SSSP algorithms that provably achieve linear O(n-m)average-case execution time on arbitrary directed graphs with random edge weights. For independent edge weights, the linear-time bound holds with high probability, too. Additionally, our result implies improved average-case bounds for the All-Pairs Shortest-Paths (APSP) problem on sparse graphs, and it yields the first theoretical average-case analysis for the "Approximate Bucket Implementation" of Dijkstra's SSSP algorithm (ABI-Dijkstra). Futhermore, we give constructive proofs for the existence of graph classes with random edge weights on which ABI-Dijkstra and several other well-known SSSP algorithms require superlinear average-case time. Besides the classical sequential (single processor) model of computation we also consider parallel computing: we give the currently fastest average-case linear-work parallel SSSP algorithms for large graph classes with random edge weights, e.g., sparse rondom graphs and graphs modeling the WWW, telephone calls or social networks.In dieser Arbeit untersuchen wir die Laufzeiten von Algorithmen für das Kürzeste-Wege Problem (Single-Source Shortest-Paths, SSSP) auf Graphen mit n Knoten, M Kanten und nichtnegativen zufälligen Kantengewichten. Alle bisherigen SSSP Algorithmen benötigen auf gerichteten Graphen superlineare Zeit. Wir stellen den ersten SSSP Algorithmus vor, der auf beliebigen gerichteten Graphen mit zufälligen Kantengewichten eine beweisbar lineare average-case-Komplexität
O(n+m)aufweist. Sind die Kantengewichte unabhängig, so wird die lineare Zeitschranke auch mit hoher Wahrscheinlichkeit eingehalten. Außerdem impliziert unser Ergebnis verbesserte average-case-Schranken für das All-Pairs Shortest-Paths (APSP) Problem auf dünnen Graphen und liefert die erste theoretische average-case-Analyse für die "Approximate Bucket Implementierung" von Dijkstras SSSP Algorithmus (ABI-Dijkstra). Weiterhin führen wir konstruktive Existenzbeweise für Graphklassen mit zufälligen Kantengewichten, auf denen ABI-Dijkstra und mehrere andere bekannte SSSP Algorithmen durchschnittlich superlineare Zeit benötigen. Neben dem klassischen seriellen (Ein-Prozessor) Berechnungsmodell betrachten wir auch Parallelverarbeitung; für umfangreiche Graphklassen mit zufälligen Kantengewichten wie z.B. dünne Zufallsgraphen oder Modelle für das WWW, Telefonanrufe oder soziale Netzwerke stellen wir die derzeit schnellsten parallelen SSSP Algorithmen mit durchschnittlich linearer Arbeit vor
- …