Building Efficient Query Engines in a High-Level Language

Abstraction without regret refers to the vision of using high-level programming languages for systems development without experiencing a negative impact on performance. A database system designed according to this vision offers both increased productivity and high performance, instead of sacrificing the former for the latter as is the case with existing, monolithic implementations that are hard to maintain and extend. In this article, we realize this vision in the domain of analytical query processing. We present LegoBase, a query engine written in the high-level language Scala. The key technique to regain efficiency is to apply generative programming: LegoBase performs source-to-source compilation and optimizes the entire query engine by converting the high-level Scala code to specialized, low-level C code. We show how generative programming allows to easily implement a wide spectrum of optimizations, such as introducing data partitioning or switching from a row to a column data layout, which are difficult to achieve with existing low-level query compilers that handle only queries. We demonstrate that sufficiently powerful abstractions are essential for dealing with the complexity of the optimization effort, shielding developers from compiler internals and decoupling individual optimizations from each other. We evaluate our approach with the TPC-H benchmark and show that: (a) With all optimizations enabled, LegoBase significantly outperforms a commercial database and an existing query compiler. (b) Programmers need to provide just a few hundred lines of high-level code for implementing the optimizations, instead of complicated low-level code that is required by existing query compilation approaches. (c) The compilation overhead is low compared to the overall execution time, thus making our approach usable in practice for compiling query engines

Lossless, Persisted Summarization of Static Callgraph, Points-To and Data-Flow Analysis

Static analysis is used to automatically detect bugs and security breaches, and aids compiler optimization. Whole-program analysis (WPA) can yield high precision, however causes long analysis times and thus does not match common software-development workflows, making it often impractical to use for large, real-world applications. This paper thus presents the design and implementation of ModAlyzer, a novel static-analysis approach that aims at accelerating whole-program analysis by making the analysis modular and compositional. It shows how to compute lossless, persisted summaries for callgraph, points-to and data-flow information, and it reports under which circumstances this function-level compositional analysis outperforms WPA. We implemented ModAlyzer as an extension to LLVM and PhASAR, and applied it to 12 real-world C and C++ applications. At analysis time, ModAlyzer modularly and losslessly summarizes the analysis effect of the library code those applications share, hence avoiding its repeated re-analysis. The experimental results show that the reuse of these summaries can save, on average, 72% of analysis time over WPA. Moreover, because it is lossless, the module-wise analysis fully retains precision and recall. Surprisingly, as our results show, it sometimes even yields precision superior to WPA. The initial summary generation, on average, takes about 3.67 times as long as WPA

Suitability of Unidata Metapps for Incorporation in Platform-Independent User-Customized Aviation Weather Products Generation Software

The Air Force Combat Climatology Center (AFCCC) is tasked to provide long-range seasonal forecasts for worldwide locations. Currently, the best long-range temperature forecasts the weather community has are the climatological standard normals. This study creates a stepping-stone into the solution of long-range forecasting by finding a process to predict temperatures better than those using climatological standard normals or simple frequency distributions of occurrences. Northern Hemispheric teleconnection indices and the standardized Southern Oscillation index are statistically compared to three-month summed Heating Degree Days (HDDs) and Cooling Degree Days (CDDs) at 14 U.S. locations. First, linear regression was accomplished. The results showed numerous valid models, however, the percent of variance resolved by the models was rarely over 30%. The HDDs and CDDs were then analyzed with Data-mining classification tree statistics, however, the results proved difficult to extract any predictive quantitative information. Finally a Data-mining regression tree analysis was performed. At each conditional outcome, a range of HDDs/CDDs is produced using the predicted standard deviations about the mean. Verification of independent teleconnection indices was used as predictors in the conditional model; 90% of the resulting HDDs/CDDs fell into the calculated range. An overall average reduction in the forecast range was 35.7% over climatolog

Hierarchical programming language for modal multi-rate real-time stream processing applications

Modal multi-rate stream processing applications with real-time constraints which are executed on multi-core embedded systems often cannot be conveniently specified using current programming languages. An important issue is that sequential programming languages do not allow for convenient programming of multi-rate behavior, whereas parallel programming languages are insufficiently analyzable such that deadlock-freedom and a sufficient throughput cannot be guaranteed.\ud \ud In this paper a programming language is proposed by which a sequential specification of the behavior of an application can be nested in a concurrent specification. Multi-rate behavior can be conveniently expressed using concurrent modules which have well-defined, but restricted interfaces. Complex control behavior can be expressed in the sequential specification of the body of a module. The language is not Turing complete such that a Compositional Temporal Analysis (CTA) model can be derived. It is shown that the CTA model can be used despite the presence of control statements and that the composition of black-box components is possible. Algorithms with a polynomial time complexity can be used to verify whether throughput and latency constraints are met and to determine sufficient buffer capacities.\ud \ud A Phase Alternating Line (PAL) video decoder application is used to demonstrate the applicability of the presented language and analysis approach

Configurable nD-visualization for complex Building Information Models

With the ongoing development of building information modelling (BIM) towards a comprehensive coverage of all construction project information in a semantically explicit way, visual representations became decoupled from the building information models. While traditional construction drawings implicitly contained the visual representation besides the information, nowadays they are generated on the fly, hard-coded in software applications dedicated to other tasks such as analysis, simulation, structural design or communication. Due to the abstract nature of information models and the increasing amount of digital information captured during construction projects, visual representations are essential for humans in order to access the information, to understand it, and to engage with it. At the same time digital media open up the new field of interactive visualizations. The full potential of BIM can only be unlocked with customized task-specific visualizations, with engineers and architects actively involved in the design and development process of these visualizations. The visualizations must be reusable and reliably reproducible during communication processes. Further, to support creative problem solving, it must be possible to modify and refine them. This thesis aims at reconnecting building information models and their visual representations: on a theoretic level, on the level of methods and in terms of tool support. First, the research seeks to improve the knowledge about visualization generation in conjunction with current BIM developments such as the multimodel. The approach is based on the reference model of the visualization pipeline and addresses structural as well as quantitative aspects of the visualization generation. Second, based on the theoretic foundation, a method is derived to construct visual representations from given visualization specifications. To this end, the idea of a domain-specific language (DSL) is employed. Finally, a software prototype proofs the concept. Using the visualization framework, visual representations can be generated from a specific building information model and a specific visualization description.Mit der fortschreitenden Entwicklung des Building Information Modelling (BIM) hin zu einer umfassenden Erfassung aller Bauprojektinformationen in einer semantisch expliziten Weise werden Visualisierungen von den Gebäudeinformationen entkoppelt. Während traditionelle Architektur- und Bauzeichnungen die visuellen Reprä̈sentationen implizit als Träger der Informationen enthalten, werden sie heute on-the-fly generiert. Die Details ihrer Generierung sind festgeschrieben in Softwareanwendungen, welche eigentlich für andere Aufgaben wie Analyse, Simulation, Entwurf oder Kommunikation ausgelegt sind. Angesichts der abstrakten Natur von Informationsmodellen und der steigenden Menge digitaler Informationen, die im Verlauf von Bauprojekten erfasst werden, sind visuelle Repräsentationen essentiell, um sich die Information erschließen, sie verstehen, durchdringen und mit ihnen arbeiten zu können. Gleichzeitig entwickelt sich durch die digitalen Medien eine neues Feld der interaktiven Visualisierungen. Das volle Potential von BIM kann nur mit angepassten aufgabenspezifischen Visualisierungen erschlossen werden, bei denen Ingenieur*innen und Architekt*innen aktiv in den Entwurf und die Entwicklung dieser Visualisierungen einbezogen werden. Die Visualisierungen müssen wiederverwendbar sein und in Kommunikationsprozessen zuverlässig reproduziert werden können. Außerdem muss es möglich sein, Visualisierungen zu modifizieren und neu zu definieren, um das kreative Problemlösen zu unterstützen. Die vorliegende Arbeit zielt darauf ab, Gebäudemodelle und ihre visuellen Repräsentationen wieder zu verbinden: auf der theoretischen Ebene, auf der Ebene der Methoden und hinsichtlich der unterstützenden Werkzeuge. Auf der theoretischen Ebene trägt die Arbeit zunächst dazu bei, das Wissen um die Erstellung von Visualisierungen im Kontext von Bauprojekten zu erweitern. Der verfolgte Ansatz basiert auf dem Referenzmodell der Visualisierungspipeline und geht dabei sowohl auf strukturelle als auch auf quantitative Aspekte des Visualisierungsprozesses ein. Zweitens wird eine Methode entwickelt, die visuelle Repräsentationen auf Basis gegebener Visualisierungsspezifikationen generieren kann. Schließlich belegt ein Softwareprototyp die Realisierbarkeit des Konzepts. Mit dem entwickelten Framework können visuelle Repräsentationen aus jeweils einem spezifischen Gebäudemodell und einer spezifischen Visualisierungsbeschreibung generiert werden

SoftBound: Highly Compatible and Complete Spatial Memory Safety for C

The serious bugs and security vulnerabilities facilitated by C/C++’s lack of bounds checking are well known. Yet, C and C++ remain in widespread use. Unfortunately, C’s arbitrary pointer arithmetic, conflation of pointers and arrays, and programmer-visible memory layout make retrofitting C/C++ with spatial safety guarantees extremely challenging. Existing approaches suffer from incompleteness, have high runtime overhead, or require non-trivial changes to the C source code. Thus far, these deficiencies have prevented widespread adoption of such techniques. This paper proposes SoftBound, a compile time transformation for enforcing complete spatial safety of C. SoftBound records base and bound information for every pointer as disjoint metadata. This decoupling enables SoftBound to provide complete spatial safety while requiring no changes to C source code. Moreover, SoftBound performs metadata manipulation only when loading or storing pointer values. A formal proof shows this is sufficient to provide complete spatial safety even in the presence of wild casts. SoftBound’s full checking mode provides complete spatial violation detection. To further reduce overheads, SoftBound has a store-only checking mode that successfully detects all the security vulnerabilities in a test suite while adding 15% or less overhead to half of the benchmarks

Inferences of Local Genetic Adaptation from Palaeolithic Hunter Gatherers

Ancient human genomes are an extremely powerful tool for exploring human demography and local genetic adaptation. Despite the thousands of genomes now available, there have been fewer amalgamated datasets focused on consistently processing samples created using different methods and filters. Therefore, I collated and processed a sample of 299 mostly publicly available ancient genomes sampling genetic diversity in Europe and recapitulating known clustering within ancient individuals. I made inferences of local genetic adaptation in 30 of the oldest Palaeolithic Hunter Gatherers, which were selected as they may reveal novel adaptations not detectable in younger individuals and or highlight earlier adaptations associated with the colonisation of Eurasia. I found many previously characterised candidates and one novel candidate in TCEA3. I also explored the advantage of incorporating ancient genomes in a test for Local Genetic Adaptation and observed a slight increase in candidates. Finally, I investigated functional annotations for the candidates of local genetic adaptation, highlighting some variants that may have influenced gene expression and disease risk and some which fall within Neanderthal- introgressed regions

Imfit: A Fast, Flexible New Program for Astronomical Image Fitting

I describe a new, open-source astronomical image-fitting program called Imfit, specialized for galaxies but potentially useful for other sources, which is fast, flexible, and highly extensible. A key characteristic of the program is an object-oriented design which allows new types of image components (2D surface-brightness functions) to be easily written and added to the program. Image functions provided with Imfit include the usual suspects for galaxy decompositions (Sersic, exponential, Gaussian), along with Core-Sersic and broken-exponential profiles, elliptical rings, and three components which perform line-of-sight integration through 3D luminosity-density models of disks and rings seen at arbitrary inclinations. Available minimization algorithms include Levenberg-Marquardt, Nelder-Mead simplex, and Differential Evolution, allowing trade-offs between speed and decreased sensitivity to local minima in the fit landscape. Minimization can be done using the standard chi^2 statistic (using either data or model values to estimate per-pixel Gaussian errors, or else user-supplied error images) or Poisson-based maximum-likelihood statistics; the latter approach is particularly appropriate for cases of Poisson data in the low-count regime. I show that fitting low-S/N galaxy images using chi^2 minimization and individual-pixel Gaussian uncertainties can lead to significant biases in fitted parameter values, which are avoided if a Poisson-based statistic is used; this is true even when Gaussian read noise is present.Comment: pdflatex, 27 pages, 19 figures. Revised version, accepted by ApJ. Programs, source code, and documentation available at: http://www.mpe.mpg.de/~erwin/code/imfit

Hydrodynamic study of a bow of a combatant hull

This thesis regards methods to study and improve the hydrodynamic performances of a ship. More specifically, the attention is focused on the ship's resistance and on modern methods used in the design process to reduce it and achieve the best design configuration. These are CFD analyses and optimization techniques. Each aspect related to this modern design process is described in detail. The original part of this thesis is the study and the optimization of the DTMB 5415 hydrodynamics