Materialisierte views in verteilten key-value stores

Distributed key-value stores have become the solution of choice for warehousing large volumes of data. However, their architecture is not suitable for real-time analytics. To achieve the required velocity, materialized views can be used to provide summarized data for fast access. The main challenge then, is the incremental, consistent maintenance of views at large scale. Thus, we introduce our View Maintenance System (VMS) to maintain SQL queries in a data-intensive real-time scenario.Verteilte key-value stores sind ein Typ moderner Datenbanken um große Mengen an Daten zu verarbeiten. Trotzdem erlaubt ihre Architektur keine analytischen Abfragen in Echtzeit. Materialisierte Views können diesen Nachteil ausgleichen, indem sie schnellen Zuriff auf Ergebnisse ermöglichen. Die Herausforderung ist dann, das inkrementelle und konsistente Aktualisieren der Views. Daher präsentieren wir unser View Maintenance System (VMS), das datenintensive SQL Abfragen in Echtzeit berechnet

A unifying approach for queries and updates in deductive databases

This dissertation presents a unifying approach to process (recursive) queries and updates in a deductive database. To improve query performance, a combined top-down and bottom-up evaluation method is used to compile rules into iterative programs that contain relational algebra operators. This method is based on the lemma resolution that retains previous results to guarantee termination.Due to locality in database processing, it is desirable to materialize frequently used queries against views of the database. Unfortunately, if updates are allowed, maintaining materialized view tables becomes a major problem. We propose to materialize views incrementally, as queries are being answered. Hence views in our approach are only partially materialized. For such views, we design algorithms to perform updates only when the underlying view tables are actually affected.We compare our approach to two conventional methods for dealing with views: total materialization and query-modification. The first method materializes the entire view when it is defined while the second recomputes the view on the fly without maintaining any physical view tables. We demonstrate that our approach is a compromise between these two methods and performs better than either one in many situations.It is also desirable to be able to update views just like updating base tables. However, view updates are inherently ambiguous and the semantics of update propagation on recursively defined views were not well understood in the past. Using dynamic logic programming and lemma resolution, we are able to define the semantics of recursive view updates. These are expressed in the form of update translators specified by the database administrator when the view is defined. To guarantee completeness, we identify a subset of safe update translators. We prove that this subset of translators always terminate and are complete

How to Price Shared Optimizations in the Cloud

Data-management-as-a-service systems are increasingly being used in collaborative settings, where multiple users access common datasets. Cloud providers have the choice to implement various optimizations, such as indexing or materialized views, to accelerate queries over these datasets. Each optimization carries a cost and may benefit multiple users. This creates a major challenge: how to select which optimizations to perform and how to share their cost among users. The problem is especially challenging when users are selfish and will only report their true values for different optimizations if doing so maximizes their utility. In this paper, we present a new approach for selecting and pricing shared optimizations by using Mechanism Design. We first show how to apply the Shapley Value Mechanism to the simple case of selecting and pricing additive optimizations, assuming an offline game where all users access the service for the same time-period. Second, we extend the approach to online scenarios where users come and go. Finally, we consider the case of substitutive optimizations. We show analytically that our mechanisms induce truth- fulness and recover the optimization costs. We also show experimentally that our mechanisms yield higher utility than the state-of-the-art approach based on regret accumulation.Comment: VLDB201

Trade-offs in Static and Dynamic Evaluation of Hierarchical Queries

We investigate trade-offs in static and dynamic evaluation of hierarchical queries with arbitrary free variables. In the static setting, the trade-off is between the time to partially compute the query result and the delay needed to enumerate its tuples. In the dynamic setting, we additionally consider the time needed to update the query result in the presence of single-tuple inserts and deletes to the input database. Our approach observes the degree of values in the database and uses different computation and maintenance strategies for high-degree and low-degree values. For the latter it partially computes the result, while for the former it computes enough information to allow for on-the-fly enumeration. The main result of this work defines the preprocessing time, the update time, and the enumeration delay as functions of the light/heavy threshold and of the factorization width of the hierarchical query. By conveniently choosing this threshold, our approach can recover a number of prior results when restricted to hierarchical queries. For a restricted class of hierarchical queries, our approach can achieve worst-case optimal update time and enumeration delay conditioned on the Online Matrix-Vector Multiplication Conjecture.Comment: Technical Report; 52 pages. The updated version contains: new diagrams and plots summarizing known results and putting the results of the paper into context; introduction of delta_i-hieararchical queries, for any non-negative integer i; optimality results for delta_0- and delta_1-hieararchical querie