Lu, Jiaheng

English

Helsingin yliopiston digitaalinen arkisto

Towards Benchmarking Multi-Model DatabasesJiaheng LuDepartment of Computer Science, University of Helsinki, Finlandjiaheng.lu@helsinki.fi1. INTRODUCTIONAs more businesses realized that data, in all forms and sizes, iscritical to making the best possible decisions, we see the contin-ued growth of systems that support massive volume of relational ornon-relational forms of data. Unlike traditional database manage-ment systems which are organized around a single data model thatdetermines how data can be organized, stored, and manipulated, amulti-model database is designed to support multiple data modelsagainst a single, integrated backend. For example, document, graph,relational, and key-value models are examples of data models thatmay be supported by a multi-model database. Nothing shows thepicture more starkly than looking at Gartner Magic quadrant foroperational database management systems, which assumes that, by2017, all leading operational DBMSs will offer multiple data models,relational and NoSQL, in a single DBMS platform. Having a singledata platform for managing both well-structured data and NoSQLdata is beneficial to users; this approach reduces significantly in-tegration, migration, development, maintenance, and operationalissues.Benchmarking is a common practice for the evaluation of thedatabase systems, as more and more platforms are proposed to dealwith multi-model data, it becomes important to have benchmarksthat can be used to evaluate performance and usability of the nextgeneration of multi-model database systems. A number of bench-marks have been proposed that can be used to evaluate big datasystems (e.g. YCSB, BigBench, TPCx-BB, Bigframe). Unfortu-nately, those general-purpose big data benchmarks are not designedfor the evaluation of multi-model databases. In this abstract, weargue that thorough evaluation of multi-model database systemsimposes several new challenges that need to be overcome. First, theinput and output of existing multi-model databases are quite diverse.Since there is no standard multi-model query language availablenow, publicly available implementations of benchmarking data andqueries for different systems should be developed, shared, unifiedand optimized. Second, unlike the relation world, NoSQL systemsfollow “data first, schema later or never” paradigm. For a rigor-ous evaluation, it must be possible to control (and systematicallyvary) input schema and the complexity of a schema evolution formulti-model data. And the benchmark must promote productivityThis article is published under a Creative Commons Attribution Li-cense(http://creativecommons.org/licenses/by/3.0/), which permits distri-bution and reproduction in any medium as well as allowing derivativeworks, provided that you attribute the original work to the author(s) andCIDR 2017. 8th Biennial Conference on Innovative Data Systems Research(CIDR’17). January 8-11, 2017, Chaminade, California, USA..Figure 1: Data models of UDBMS benchmarkby enabling the creation of a large number of multi-model datawith varied schema using little manual effort. Finally, multi-modeldatabases are supposed to support the cross-model transaction andconsistency. Therefore, novel consistency metrics which describeconsistency behavior for different models of data must be proposedin a precise way.2. OUR UDBMS BENCHMARKTo realize our vision we propose UDBMS-benchmark, a bench-mark system for Unified DataBase Management Systems, with thefollowing new features:Multi-model data: A high level overview of the data model ispresented in Figure 1, which includes relational data, XML data,graph data, JSON data and key-value data.Multi-model schema evolution: UDBMS-benchmark automatesthe schema evolution process for multi-model data. The change ofschema can affect the usability of history queries.Multi-model transaction and consistency: One transaction caninvolve multiple data models. For example, an update of order infor-mation may affect JSON files (Orders, Product), key-value mes-sages (Feedback) and XML data (Invoice). UDBMS-benchmarkdevelops consistency metrics of ACID and eventual consistency withmulti-model data and accurately determines consistency behaviorvia experiments with actually deployed systems.Multi-model data conversion: An ideal multi-model databaseshould support the model conversion between relation and NoSQLdata. Therefore, data generators must support the creation of reason-able gold standard outputs for different transformation tasks.To sum up, our proposed system involves novel features to give acomprehensive and rigorous evaluation for multi-model databases.Once completed, this benchmark will provide the community with arich set of examples for multi-model data, query and transaction thatcan be studied to improve multi-model data management, as well asto ease the evaluation of diverse systems. This benchmark can bedownloaded from http://udbms.cs.helsinki.fi/bench/

Towards Benchmarking Multi-Model Databases

https://helda.helsinki.fi/bitstream/10138/313313/1/CIDR2017.pdf

Towards Benchmarking Multi-Model Databases

Abstract

Similar works

Full text

Available Versions

Helsingin yliopiston digitaalinen arkisto