Making an Embedded DBMS JIT-friendly

While database management systems (DBMSs) are highly optimized, interactions across the boundary between the programming language (PL) and the DBMS are costly, even for in-process embedded DBMSs. In this paper, we show that programs that interact with the popular embedded DBMS SQLite can be significantly optimized - by a factor of 3.4 in our benchmarks - by inlining across the PL / DBMS boundary. We achieved this speed-up by replacing parts of SQLite's C interpreter with RPython code and composing the resulting meta-tracing virtual machine (VM) - called SQPyte - with the PyPy VM. SQPyte does not compromise stand-alone SQL performance and is 2.2% faster than SQLite on the widely used TPC-H benchmark suite.Comment: 24 pages, 18 figure

Apache Calcite: A Foundational Framework for Optimized Query Processing Over Heterogeneous Data Sources

Apache Calcite is a foundational software framework that provides query processing, optimization, and query language support to many popular open-source data processing systems such as Apache Hive, Apache Storm, Apache Flink, Druid, and MapD. Calcite's architecture consists of a modular and extensible query optimizer with hundreds of built-in optimization rules, a query processor capable of processing a variety of query languages, an adapter architecture designed for extensibility, and support for heterogeneous data models and stores (relational, semi-structured, streaming, and geospatial). This flexible, embeddable, and extensible architecture is what makes Calcite an attractive choice for adoption in big-data frameworks. It is an active project that continues to introduce support for the new types of data sources, query languages, and approaches to query processing and optimization.Comment: SIGMOD'1

Duplicates and translation of nested SQL queries into XRA

The PRISMA/DB system contains a parser to translate the database language SQL into eXtended Relational Algebra (XRA). The early definition of XRA, which has a multi-set semantics, proves inadequate for translating SQL according to its nested-iteration semantics. The prime cause is that no distinction is made between original and generated duplicate tuples during nested query handling. To achieve a correct translation, tuple identifiers were introduced into XRA and the system

A performant XQuery to SQL translator

We describe a largely complete and efficient XQuery to SQL translation for XML publishing. Our translation supports the entire XQuery language, except for functions, if statements and upwards navigation axes. The system has three important properties. First, it preserves the correct XQuery semantics. This is accomplished by first translating XQuery into core-XQuery, using a complete XQuery implementation, Galax. Second, we optimize the resulting SQL queries. We develop a comprehensive framework for optimizing the XQuery to SQL translation, which is effective for a wide range of XQuery workloads. Third, our translation is platform independent. Our system achieves high degree of efficiency on a wide range of relational systems. This paper reports an extensive experimental validation on several XQuery workloads, using MySQL, PostgreSQL, and SQL Server, and compares this approach with five native XQuery engines: Galax (the newer, optimized version), Saxon, QizOpen, IMDB and Quexo

Flattening an object algebra to provide performance

Algebraic transformation and optimization techniques have been the method of choice in relational query execution, but applying them in object-oriented (OO) DBMSs is difficult due to the complexity of OO query languages. This paper demonstrates that the problem can be simplified by mapping an OO data model to the binary relational model implemented by Monet, a state-of-the-art database kernel. We present a generic mapping scheme to flatten data models and study the case of straightforward OO model. We show how flattening enabled us to implement a query algebra, using only a very limited set of simple operations. The required primitives and query execution strategies are discussed, and their performance is evaluated on the 1-GByte TPC-D (Transaction-processing Performance Council's Benchmark D), showing that our divide-and-conquer approach yields excellent result