UniBench: A Benchmark for Multi-Model Database Management Systems

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

Abstract

Unlike traditional database management systems which are organized around a single data model, a multi-model database (MMDB) utilizes a single, integrated back-end to support multiple data models, such as document, graph, relational, and key-value. As more and more platforms are proposed to deal with multi-model data, it becomes crucial to establish a benchmark for evaluating the performance and usability of MMDBs. Previous benchmarks, however, are inadequate for such scenario because they lack a comprehensive consideration for multiple models of data. In this paper, we present a benchmark, called UniBench, with the goal of facilitating a holistic and rigorous evaluation of MMDBs. UniBench consists of a mixed data model, a synthetic multi-model data generator, and a set of core workloads. Specifically, the data model simulates an emerging application: Social Commerce, a Web-based application combining E-commerce and social media. The data generator provides diverse data format including JSON, XML, key-value, tabular, and graph. The workloads are comprised of a set of multi-model queries and transactions, aiming to cover essential aspects of multi-model data management. We implemented all workloads on ArangoDB and OrientDB to illustrate the feasibility of our proposed benchmarking system and show the learned lessons through the evaluation of these two multi-model databases. The source code and data of this benchmark can be downloaded at http://udbms.cs.helsinki.fi/bench/.
Original languageEnglish
Title of host publicationPerformance Evaluation and Benchmarking for the Era of Artificial Intelligence : 10th TPC Technology Conference, TPCTC 2018, Rio de Janeiro, Brazil, August 27–31, 2018, Revised Selected Papers
EditorsRaghunath Nambiar, Meikel Poess
Number of pages16
Place of PublicationCham
PublisherSpringer
Publication date2019
Pages7-23
ISBN (Print)978-3-030-11403-9
ISBN (Electronic)978-3-030-11404-6
DOIs
Publication statusPublished - 2019
MoE publication typeA4 Article in conference proceedings
EventTPC Technology Conference on Performance Evaluation & Benchmarking - Rio De Janeiro, Brazil
Duration: 27 Aug 201731 Aug 2017
Conference number: 10

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume11135
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Fields of Science

  • 113 Computer and information sciences

Cite this

Zhang, C., Lu, J., Xu, P., & Chen, Y. (2019). UniBench: A Benchmark for Multi-Model Database Management Systems. In R. Nambiar, & M. Poess (Eds.), Performance Evaluation and Benchmarking for the Era of Artificial Intelligence: 10th TPC Technology Conference, TPCTC 2018, Rio de Janeiro, Brazil, August 27–31, 2018, Revised Selected Papers (pp. 7-23). (Lecture Notes in Computer Science; Vol. 11135). Cham: Springer. https://doi.org/10.1007/978-3-030-11404-6_2
Zhang, Chao ; Lu, Jiaheng ; Xu, Pengfei ; Chen, Yuxing. / UniBench: A Benchmark for Multi-Model Database Management Systems. Performance Evaluation and Benchmarking for the Era of Artificial Intelligence: 10th TPC Technology Conference, TPCTC 2018, Rio de Janeiro, Brazil, August 27–31, 2018, Revised Selected Papers. editor / Raghunath Nambiar ; Meikel Poess. Cham : Springer, 2019. pp. 7-23 (Lecture Notes in Computer Science).
@inproceedings{e8e73775f4224d57a10bba18f36a1c29,
title = "UniBench: A Benchmark for Multi-Model Database Management Systems",
abstract = "Unlike traditional database management systems which are organized around a single data model, a multi-model database (MMDB) utilizes a single, integrated back-end to support multiple data models, such as document, graph, relational, and key-value. As more and more platforms are proposed to deal with multi-model data, it becomes crucial to establish a benchmark for evaluating the performance and usability of MMDBs. Previous benchmarks, however, are inadequate for such scenario because they lack a comprehensive consideration for multiple models of data. In this paper, we present a benchmark, called UniBench, with the goal of facilitating a holistic and rigorous evaluation of MMDBs. UniBench consists of a mixed data model, a synthetic multi-model data generator, and a set of core workloads. Specifically, the data model simulates an emerging application: Social Commerce, a Web-based application combining E-commerce and social media. The data generator provides diverse data format including JSON, XML, key-value, tabular, and graph. The workloads are comprised of a set of multi-model queries and transactions, aiming to cover essential aspects of multi-model data management. We implemented all workloads on ArangoDB and OrientDB to illustrate the feasibility of our proposed benchmarking system and show the learned lessons through the evaluation of these two multi-model databases. The source code and data of this benchmark can be downloaded at http://udbms.cs.helsinki.fi/bench/.",
keywords = "113 Computer and information sciences",
author = "Chao Zhang and Jiaheng Lu and Pengfei Xu and Yuxing Chen",
year = "2019",
doi = "10.1007/978-3-030-11404-6_2",
language = "English",
isbn = "978-3-030-11403-9",
series = "Lecture Notes in Computer Science",
publisher = "Springer",
pages = "7--23",
editor = "Raghunath Nambiar and Meikel Poess",
booktitle = "Performance Evaluation and Benchmarking for the Era of Artificial Intelligence",
address = "United States",

}

Zhang, C, Lu, J, Xu, P & Chen, Y 2019, UniBench: A Benchmark for Multi-Model Database Management Systems. in R Nambiar & M Poess (eds), Performance Evaluation and Benchmarking for the Era of Artificial Intelligence: 10th TPC Technology Conference, TPCTC 2018, Rio de Janeiro, Brazil, August 27–31, 2018, Revised Selected Papers. Lecture Notes in Computer Science, vol. 11135, Springer, Cham, pp. 7-23, TPC Technology Conference on Performance Evaluation & Benchmarking, Rio De Janeiro, Brazil, 27/08/2017. https://doi.org/10.1007/978-3-030-11404-6_2

UniBench: A Benchmark for Multi-Model Database Management Systems. / Zhang, Chao; Lu, Jiaheng; Xu, Pengfei; Chen, Yuxing.

Performance Evaluation and Benchmarking for the Era of Artificial Intelligence: 10th TPC Technology Conference, TPCTC 2018, Rio de Janeiro, Brazil, August 27–31, 2018, Revised Selected Papers. ed. / Raghunath Nambiar; Meikel Poess. Cham : Springer, 2019. p. 7-23 (Lecture Notes in Computer Science; Vol. 11135).

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

TY - GEN

T1 - UniBench: A Benchmark for Multi-Model Database Management Systems

AU - Zhang, Chao

AU - Lu, Jiaheng

AU - Xu, Pengfei

AU - Chen, Yuxing

PY - 2019

Y1 - 2019

N2 - Unlike traditional database management systems which are organized around a single data model, a multi-model database (MMDB) utilizes a single, integrated back-end to support multiple data models, such as document, graph, relational, and key-value. As more and more platforms are proposed to deal with multi-model data, it becomes crucial to establish a benchmark for evaluating the performance and usability of MMDBs. Previous benchmarks, however, are inadequate for such scenario because they lack a comprehensive consideration for multiple models of data. In this paper, we present a benchmark, called UniBench, with the goal of facilitating a holistic and rigorous evaluation of MMDBs. UniBench consists of a mixed data model, a synthetic multi-model data generator, and a set of core workloads. Specifically, the data model simulates an emerging application: Social Commerce, a Web-based application combining E-commerce and social media. The data generator provides diverse data format including JSON, XML, key-value, tabular, and graph. The workloads are comprised of a set of multi-model queries and transactions, aiming to cover essential aspects of multi-model data management. We implemented all workloads on ArangoDB and OrientDB to illustrate the feasibility of our proposed benchmarking system and show the learned lessons through the evaluation of these two multi-model databases. The source code and data of this benchmark can be downloaded at http://udbms.cs.helsinki.fi/bench/.

AB - Unlike traditional database management systems which are organized around a single data model, a multi-model database (MMDB) utilizes a single, integrated back-end to support multiple data models, such as document, graph, relational, and key-value. As more and more platforms are proposed to deal with multi-model data, it becomes crucial to establish a benchmark for evaluating the performance and usability of MMDBs. Previous benchmarks, however, are inadequate for such scenario because they lack a comprehensive consideration for multiple models of data. In this paper, we present a benchmark, called UniBench, with the goal of facilitating a holistic and rigorous evaluation of MMDBs. UniBench consists of a mixed data model, a synthetic multi-model data generator, and a set of core workloads. Specifically, the data model simulates an emerging application: Social Commerce, a Web-based application combining E-commerce and social media. The data generator provides diverse data format including JSON, XML, key-value, tabular, and graph. The workloads are comprised of a set of multi-model queries and transactions, aiming to cover essential aspects of multi-model data management. We implemented all workloads on ArangoDB and OrientDB to illustrate the feasibility of our proposed benchmarking system and show the learned lessons through the evaluation of these two multi-model databases. The source code and data of this benchmark can be downloaded at http://udbms.cs.helsinki.fi/bench/.

KW - 113 Computer and information sciences

U2 - 10.1007/978-3-030-11404-6_2

DO - 10.1007/978-3-030-11404-6_2

M3 - Conference contribution

SN - 978-3-030-11403-9

T3 - Lecture Notes in Computer Science

SP - 7

EP - 23

BT - Performance Evaluation and Benchmarking for the Era of Artificial Intelligence

A2 - Nambiar, Raghunath

A2 - Poess, Meikel

PB - Springer

CY - Cham

ER -

Zhang C, Lu J, Xu P, Chen Y. UniBench: A Benchmark for Multi-Model Database Management Systems. In Nambiar R, Poess M, editors, Performance Evaluation and Benchmarking for the Era of Artificial Intelligence: 10th TPC Technology Conference, TPCTC 2018, Rio de Janeiro, Brazil, August 27–31, 2018, Revised Selected Papers. Cham: Springer. 2019. p. 7-23. (Lecture Notes in Computer Science). https://doi.org/10.1007/978-3-030-11404-6_2