Incremental Maintenance of Recursive Views Using Relational Calculus/SQL

Views are a central component of both traditional database systems and new applications such as data warehouses. Very often the desired views (e.g. the transitive closure) cannot be defined in the standard language of the underlying database system. Fortunately, it is often possible to incrementally maintain these views using the standard language. For example, transitive closure of acyclic graphs, and of undirected graphs, can be maintained in relational calculus after both single edge insertions and deletions. Many such results have been published in the theoretical database community. The purpose of this survey is to make these useful results known to the wider database research and development community. There are many interesting issues involved in the maintenance of recursive views. A maintenance algorithm may be applicable to just one view, or to a class of views specified by a view definition language such as Datalog. The maintenance algorithm can be specified in a maintenance language of different expressiveness, such as the conjunctive queries, the relational calculus or SQL. Ideally, this maintenance language should be less expensive than the view def-inition language. The maintenance algorithm may allow updates of different kinds, such as just single tuple insertions, just single tuple deletions, special set-based insertions and/or deletions, or combinations thereof. The view maintenance algorithms may also need to maintain auxiliary relations to help maintain the views of interest. It is of interest to know the minimal arity necessary for these auxiliary relation

Inkrementelle Integritätsprüfung und Sichtenaktualisierung in SQL

Zentrales Problem der Implementierung von Integritätsbedingungen und materialisierten Sichten ist die effiziente Reaktion auf Basisfakten-änderungen. Ändert sich ein Basisfakt durch Einfügung, Löschung oder Modifikation, so müssen die Integritätsbedingungen geprüft und die abgeleiteten und gespeicherten Fakten materialisierter Sichten aktualisiert werden. Da die Dauer der Integritätsprüfung und Sichten-aktualisierung unmittelbar die Ausführungsdauer einer Transaktion beeinflußt, ist eine effiziente Durchführung von entscheidender Bedeutung. Effiziente Verfahren zur Integritätsprüfung und Aktualisierung materialisierter Sichten gehören seit mehr als 15 Jahren zu den wichtigsten Problemen der Forschungen zu deduktiven Datenbanken. Die im Kontext von Datalog und Relationaler Algebra entwickelten inkrementellen Verfahren haben jedoch bislang kaum Anwendung in SQL gefunden. Dies zeigt sich an den funktional sehr beschränkten Implementierungen beider Konzepte in SQL-basierten Datenbank-systemen kommerzieller Hersteller. Zentrale Idee inkrementeller Ansätze ist, dass Integritätsprüfung und Sichtenaktualisierung nur für die aktuell geänderten Fakten durchgeführt werden. Zu diesem Zweck werden im Rahmen eines Änderungspropagierungsprozesses die durch Basisfaktenänderungen induzierten Änderungen abgeleiteter Fakten ermittelt. Da bei den bisherigen inkrementellen Verfahren SQL-spezifische Konzepte wie das SQL-Transaktions- und Integritätskonzept nicht berücksichtigt wurden, ist eine Anwendung dieser Verfahren in SQL nicht direkt möglich. Aus diesem Grund werden im Rahmen der vorgelegten Dissertation inkrementelle Verfahren zur effizienten Integritätsprüfung und Sichtenaktualisierung im Kontext von SQL entwickelt. Bei den Verfahrensentwürfen werden SQL-spezifische Systemeigenschaften unverändert berücksichtigt. Die Implementierung dieser Verfahren in SQL-basierten, kommerziellen Datenbanksystemen würde diese Systeme funktional um leistungsfähige Komponenten zur Integritätsprüfung und zur Simulation und Aktualisierung materialisierter Sichten erweitern

Scalable Automated Incrementalization for Real-Time Static Analyses

This thesis proposes a framework for easy development of static analyses, whose results are incrementalized to provide instantaneous feedback in an integrated development environment (IDE). Today, IDEs feature many tools that have static analyses as their foundation to assess software quality and catch correctness problems. Yet, these tools often fail to provide instantaneous feedback and are thus restricted to nightly build processes. This precludes developers from fixing issues at their inception time, i.e., when the problem and the developed solution are both still fresh in mind. In order to provide instantaneous feedback, incrementalization is a well-known technique that utilizes the fact that developers make only small changes to the code and, hence, analysis results can be re-computed fast based on these changes. Yet, incrementalization requires carefully crafted static analyses. Thus, a manual approach to incrementalization is unattractive. Automated incrementalization can alleviate these problems and allows analyses writers to formulate their analyses as queries with the full data set in mind, without worrying over the semantics of incremental changes. Existing approaches to automated incrementalization utilize standard technologies, such as deductive databases, that provide declarative query languages, yet also require to materialize the full dataset in main-memory, i.e., the memory is permanently blocked by the data required for the analyses. Other standard technologies such as relational databases offer better scalability due to persistence, yet require large transaction times for data. Both technologies are not a perfect match for integrating static analyses into an IDE, since the underlying data, i.e., the code base, is already persisted and managed by the IDE. Hence, transitioning the data into a database is redundant work. In this thesis a novel approach is proposed that provides a declarative query language and automated incrementalization, yet retains in memory only a necessary minimum of data, i.e., only the data that is required for the incrementalization. The approach allows to declare static analyses as incrementally maintained views, where the underlying formalism for incrementalization is the relational algebra with extensions for object-orientation and recursion. The algebra allows to deduce which data is the necessary minimum for incremental maintenance and indeed shows that many views are self-maintainable, i.e., do not require to materialize memory at all. In addition an optimization for the algebra is proposed that allows to widen the range of self-maintainable views, based on domain knowledge of the underlying data. The optimization works similar to declaring primary keys for databases, i.e., the optimization is declared on the schema of the data, and defines which data is incrementally maintained in the same scope. The scope makes all analyses (views) that correlate only data within the boundaries of the scope self-maintainable. The approach is implemented as an embedded domain specific language in a general-purpose programming language. The implementation can be understood as a database-like engine with an SQL-style query language and the execution semantics of the relational algebra. As such the system is a general purpose database-like query engine and can be used to incrementalize other domains than static analyses. To evaluate the approach a large variety of static analyses were sampled from real-world tools and formulated as incrementally maintained views in the implemented engine