137 research outputs found
An introduction to Graph Data Management
A graph database is a database where the data structures for the schema
and/or instances are modeled as a (labeled)(directed) graph or generalizations
of it, and where querying is expressed by graph-oriented operations and type
constructors. In this article we present the basic notions of graph databases,
give an historical overview of its main development, and study the main current
systems that implement them
GiViP: A Visual Profiler for Distributed Graph Processing Systems
Analyzing large-scale graphs provides valuable insights in different
application scenarios. While many graph processing systems working on top of
distributed infrastructures have been proposed to deal with big graphs, the
tasks of profiling and debugging their massive computations remain time
consuming and error-prone. This paper presents GiViP, a visual profiler for
distributed graph processing systems based on a Pregel-like computation model.
GiViP captures the huge amount of messages exchanged throughout a computation
and provides an interactive user interface for the visual analysis of the
collected data. We show how to take advantage of GiViP to detect anomalies
related to the computation and to the infrastructure, such as slow computing
units and anomalous message patterns.Comment: Appears in the Proceedings of the 25th International Symposium on
Graph Drawing and Network Visualization (GD 2017
Parallel Processing of Large Graphs
More and more large data collections are gathered worldwide in various IT
systems. Many of them possess the networked nature and need to be processed and
analysed as graph structures. Due to their size they require very often usage
of parallel paradigm for efficient computation. Three parallel techniques have
been compared in the paper: MapReduce, its map-side join extension and Bulk
Synchronous Parallel (BSP). They are implemented for two different graph
problems: calculation of single source shortest paths (SSSP) and collective
classification of graph nodes by means of relational influence propagation
(RIP). The methods and algorithms are applied to several network datasets
differing in size and structural profile, originating from three domains:
telecommunication, multimedia and microblog. The results revealed that
iterative graph processing with the BSP implementation always and
significantly, even up to 10 times outperforms MapReduce, especially for
algorithms with many iterations and sparse communication. Also MapReduce
extension based on map-side join usually noticeably presents better efficiency,
although not as much as BSP. Nevertheless, MapReduce still remains the good
alternative for enormous networks, whose data structures do not fit in local
memories.Comment: Preprint submitted to Future Generation Computer System
Spinning Fast Iterative Data Flows
Parallel dataflow systems are a central part of most analytic pipelines for
big data. The iterative nature of many analysis and machine learning
algorithms, however, is still a challenge for current systems. While certain
types of bulk iterative algorithms are supported by novel dataflow frameworks,
these systems cannot exploit computational dependencies present in many
algorithms, such as graph algorithms. As a result, these algorithms are
inefficiently executed and have led to specialized systems based on other
paradigms, such as message passing or shared memory. We propose a method to
integrate incremental iterations, a form of workset iterations, with parallel
dataflows. After showing how to integrate bulk iterations into a dataflow
system and its optimizer, we present an extension to the programming model for
incremental iterations. The extension alleviates for the lack of mutable state
in dataflows and allows for exploiting the sparse computational dependencies
inherent in many iterative algorithms. The evaluation of a prototypical
implementation shows that those aspects lead to up to two orders of magnitude
speedup in algorithm runtime, when exploited. In our experiments, the improved
dataflow system is highly competitive with specialized systems while
maintaining a transparent and unified dataflow abstraction.Comment: VLDB201
The Future is Big Graphs! A Community View on Graph Processing Systems
Graphs are by nature unifying abstractions that can leverage
interconnectedness to represent, explore, predict, and explain real- and
digital-world phenomena. Although real users and consumers of graph instances
and graph workloads understand these abstractions, future problems will require
new abstractions and systems. What needs to happen in the next decade for big
graph processing to continue to succeed?Comment: 12 pages, 3 figures, collaboration between the large-scale systems
and data management communities, work started at the Dagstuhl Seminar 19491
on Big Graph Processing Systems, to be published in the Communications of the
AC
- …