A Multi-ontology Synthetic Benchmark for the Semantic Web

Abstract. One important use case for the Semantic Web is the integration of data across many heterogeneous ontologies. However, most Semantic Web Knowledge Bases are evaluated using the single ontology benchmark such as LUBM and UOBM. Therefore, there is a requirement to develop a benchmark system that is able to evaluate not only single but also federated ontology systems for different uses with different configurations of ontologies. To support such a need, based on our earlier work, we present a multi-ontology synthetic benchmark system that takes a two-level profile as input to generate user-customized ontologies together with related mappings and data sources. Meanwhile, a graph-based query generation algorithm and an owl:sameAs generation mechanism are also proposed. By using this benchmark, the Semantic Web systems can be evaluated against complex ontology configurations using the standard metrics of loading time, repository size, query response time and query completeness and soundness