1 research outputs found
Benchmarking Graph Data Management and Processing Systems: A Survey
The development of scalable, representative, and widely adopted benchmarks
for graph data systems have been a question for which answers has been sought
for decades. We conduct an in-depth study of the existing literature on
benchmarks for graph data management and processing, covering 20 different
benchmarks developed during the last 15 years. We categorize the benchmarks
into three areas focusing on benchmarks for graph processing systems, graph
database benchmarks, and bigdata benchmarks with graph processing workloads.
This systematic approach allows us to identify multiple issues existing in this
area, including i) few benchmarks exist which can produce high workload
scenarios, ii) no significant work done on benchmarking graph stream processing
as well as graph based machine learning, iii) benchmarks tend to use
conventional metrics despite new meaningful metrics have been around for years,
iv) increasing number of big data benchmarks appear with graph processing
workloads. Following these observations, we conclude the survey by describing
key challenges for future research on graph data systems benchmarking.Comment: 26 pages, 5 figure