Cost-Based Optimization of Integration Flows

Integration flows are increasingly used to specify and execute data-intensive integration tasks between heterogeneous systems and applications. There are many different application areas such as real-time ETL and data synchronization between operational systems. For the reasons of an increasing amount of data, highly distributed IT infrastructures, and high requirements for data consistency and up-to-dateness of query results, many instances of integration flows are executed over time. Due to this high load and blocking synchronous source systems, the performance of the central integration platform is crucial for an IT infrastructure. To tackle these high performance requirements, we introduce the concept of cost-based optimization of imperative integration flows that relies on incremental statistics maintenance and inter-instance plan re-optimization. As a foundation, we introduce the concept of periodical re-optimization including novel cost-based optimization techniques that are tailor-made for integration flows. Furthermore, we refine the periodical re-optimization to on-demand re-optimization in order to overcome the problems of many unnecessary re-optimization steps and adaptation delays, where we miss optimization opportunities. This approach ensures low optimization overhead and fast workload adaptation

Interactive Model-Based Compilation: A Modeller-Driven Development Approach

There is a growing tendency for using domain-specific languages, which help domain experts to stay focussed on abstract problem solutions. It is important to carefully design these languages and tools, which fundamentally perform model-to-model transformations. The quality of both usually decides the effectiveness of the subsequent development and therefore the quality of the final applications. However, as the complexity and safety requirements of modern systems grow, it becomes increasingly burdensome to create highly customized languages and difficult to provide reasonable overviews within these tools. This thesis introduces a new interactive model-based compilation methodology. Compilations for arbitrary model-to-model transformations are themselves described as models. They can be instantiated for particular inputs, e. g. a program, to create concrete compilation runs, which return the result of that compilation. The compilation instance is interactively observable. Intermediate results serve as new inputs and as documentation. They can be used to create highly customized views and facilitate understandability. This methodology guides modellers from the start of the compilation to the final result so that they can interactively refine their models. The methodology has been implemented and validated as the KIELER Compiler (KiCo) and is available as part of the KIELER open-source project. It is used to implement the current reference compiler for the SCCharts language, a statecharts dialect designed for specifying safety-critical reactive systems based on a synchronous model of computation. The interactive model-based compilation approach was key to the rapid prototyping of three different compilation strategies, as well as new language extensions, variations and closely related languages. The results are verified with benchmarks, which are again modelled using the same approach and technology. The usability of the SCCharts language and the KiCo tooling is documented with long-term surveys and real-life industrial, academic and teaching examples

Physical Plan Instrumentation in Databases: Mechanisms and Applications

Database management systems (DBMSs) are designed with the goal set to compile SQL queries to physical plans that, when executed, provide results to the SQL queries. Building on this functionality, an ever-increasing number of application domains (e.g., provenance management, online query optimization, physical database design, interactive data profiling, monitoring, and interactive data visualization) seek to operate on how queries are executed by the DBMS for a wide variety of purposes ranging from debugging and data explanation to optimization and monitoring. Unfortunately, DBMSs provide little, if any, support to facilitate the development of this class of important application domains. The effect is such that database application developers and database system architects either rewrite the database internals in ad-hoc ways; work around the SQL interface, if possible, with inevitable performance penalties; or even build new databases from scratch only to express and optimize their domain-specific application logic over how queries are executed. To address this problem in a principled manner in this dissertation, we introduce a prototype DBMS, namely, Smoke, that exposes instrumentation mechanisms in the form of a framework to allow external applications to manipulate physical plans. Intuitively, a physical plan is the underlying representation that DBMSs use to encode how a SQL query will be executed, and providing instrumentation mechanisms at this representation level allows applications to express and optimize their logic on how queries are executed. Having such an instrumentation-enabled DBMS in-place, we then consider how to express and optimize applications that rely their logic on how queries are executed. To best demonstrate the expressive and optimization power of instrumentation-enabled DBMSs, we express and optimize applications across several important domains including provenance management, interactive data visualization, interactive data profiling, physical database design, online query optimization, and query discovery. Expressivity-wise, we show that Smoke can express known techniques, introduce novel semantics on known techniques, and introduce new techniques across domains. Performance-wise, we show case-by-case that Smoke is on par with or up-to several orders of magnitudes faster than state-of-the-art imperative and declarative implementations of important applications across domains. As such, we believe our contributions provide evidence and form the basis towards a class of instrumentation-enabled DBMSs with the goal set to express and optimize applications across important domains with core logic over how queries are executed by DBMSs

Grid-Forming Converter Control Method to Improve DC-Link Stability in Inverter-Based AC Grids

As renewable energy sources with power-electronic interfaces become functionally and economically viable alternatives to bulk synchronous generators, it becomes vital to understand the behavior of these inverter-interfaced sources in ac grids devoid of any synchronous generation, i.e. inverter-based grids. In these types of grids, the inverters need to operate in parallel in grid-forming mode to regulate and synchronize their output voltage while also delivering the power required by the loads. It is common practice, therefore, to mimic the parallel operation control of the very synchronous generators that these inverter-based sources are meant to replace. This practice, however, is based on impractical assumptions and completely disregards the key differences between synchronous machines and power electronic inverters, as well as the dynamics of the dc source connected to the inverter. This dissertation aims to highlight the shortcomings of conventional controllers and derive an improved grid-forming inverter controller that is effective in parallel ac operation without sacrificing dc-link stability. This dissertation begins with a basis for understanding the control concepts used by grid-forming inverters in ac grids and exploring where existing ideas and methods are lacking in terms of efficient and stable inverter control. The knowledge gained from the literature survey is used to derive the requirements for a grid-forming control method that is appropriate for inverter-based ac grids. This is followed by a review and comparative analysis of the performance of five commonly used control techniques for grid-forming inverters, which reveal that nested loop controllers can have a destabilizing effect under changing grid conditions. This observation is further explored through an impedance-based stability analysis of single-loop and nested-loop controllers in grid-forming inverters, followed by a review of impedance-based analysis methods that can be used to assess the control design for grid-forming inverters. An improved grid-forming inverter controller is proposed with a demonstrated ability to achieve both dc-link and ac output stability with proportional power-sharing. This dissertation ends with a summary of the efforts and contributions as well as ideas for future applications of the proposed controller

DBToaster: higher-order delta processing for dynamic, frequently fresh views

Applications ranging from algorithmic trading to scientific data analysis require real-time analytics based on views over databases receiving thousands of updates each second. Such views have to be kept fresh at millisecond latencies. At the same time, these views have to support classical SQL, rather than window semantics, to enable applications that combine current with aged or historical data. In this article, we present the DBToaster system, which keeps materialized views of standard SQL queries continuously fresh as data changes very rapidly. This is achieved by a combination of aggressive compilation techniques and DBToaster's original recursive finite differencing technique which materializes a query and a set of its higher-order deltas as views. These views support each other's incremental maintenance, leading to a reduced overall view maintenance cost. DBToaster supports tens of thousands of complete view refreshes per second for a wide range of queries

The Application of Ant Colony Optimization

The application of advanced analytics in science and technology is rapidly expanding, and developing optimization technics is critical to this expansion. Instead of relying on dated procedures, researchers can reap greater rewards by utilizing cutting-edge optimization techniques like population-based metaheuristic models, which can quickly generate a solution with acceptable quality. Ant Colony Optimization (ACO) is one the most critical and widely used models among heuristics and meta-heuristics. This book discusses ACO applications in Hybrid Electric Vehicles (HEVs), multi-robot systems, wireless multi-hop networks, and preventive, predictive maintenance

Continuously Providing Approximate Results under Limited Resources: Load Shedding and Spilling in XML Streams

Because of the high volume and unpredictable arrival rates, stream processing systems may not always be able to keep up with the input data streams, resulting in buffer overflow and uncontrolled loss of data. To continuously supply online results, two alternate solutions to tackle this problem of unpredictable failures of such overloaded systems can be identified. One technique, called load shedding, drops some fractions of data from the input stream to reduce the memory and CPU requirements of the workload. However, dropping some portions of the input data means that the accuracy of the output is reduced since some data is lost. To produce eventually complete results, the second technique, called data spilling, pushes some fractions of data to persistent storage temporarily when the processing speed cannot keep up with the arrival rate. The processing of the disk resident data is then postponed until a later time when system resources become available. This dissertation explores these load reduction technologies in the context of XML stream systems. Load shedding in the specific context of XML streams poses several unique opportunities and challenges. Since XML data is hierarchical, subelements, extracted from different positions of the XML tree structure, may vary in their importance. Further, dropping different subelements may vary in their savings of storage and computation. Hence, unlike prior work in the literature that drops data completely or not at all, in this dissertation we introduce the notion of structure-oriented load shedding, meaning selectively some XML subelements are shed from the possibly complex XML objects in the XML stream. First we develop a preference model that enables users to specify the relative importance of preserving different subelements within the XML result structure. This transforms shedding into the problem of rewriting the user query into shed queries that return approximate answers with their utility as measured by the user preference model. Our optimizer finds the appropriate shed queries to maximize the output utility driven by our structure-based preference model under the limitation of available computation resources. The experimental results demonstrate that our proposed XML-specific shedding solution consistently achieves higher utility results compared to the existing relational shedding techniques. Second, we introduces structure-based spilling, a spilling technique customized for XML streams by considering the spilling of partial substructures of possibly complex XML elements. Several new challenges caused by structure-based spilling are addressed. When a path is spilled, multiple other paths may be affected. We categorize varying types of spilling side effects on the query caused by spilling. How to execute the reduced query to produce the correct runtime output is also studied. Three optimization strategies are developed to select the reduced query that maximizes the output quality. We also examine the clean-up stage to guarantee that an entire result set is eventually generated by producing supplementary results to complement the partial results output earlier. The experimental study demonstrates that our proposed solutions consistently achieve higher quality results compared to the state-of-the-art techniques. Third, we design an integrated framework that combines both shedding and spilling policies into one comprehensive methodology. Decisions on the choice of whether to shed or spill data may be affected by the application needs and data arrival patterns. For some input data, it may be worth to flush it to disk if a delayed output of its result will be important, while other data would best directly dropped from the system given that a delayed delivery of these results would no longer be meaningful to the application. Therefore we need sophisticated technologies capable of deploying both shedding and spilling techniques within one integrated strategy with the ability to deliver the most appropriate decision customers need for each specific circumstance. We propose a novel flexible framework for structure-based shed and spill approaches, applicable in any XML stream system. We propose a solution space that represents all the shed and spill candidates. An age-based quality model is proposed for evaluating the output quality for different reduced query and supplementary query pairs. We also propose a family of four optimization strategies, OptF, OptSmart, HiX and Fex. OptF and OptSmart are both guaranteed to identify an optimal solution of reduced and supplementary query pair, with OptSmart exhibiting significantly less overhead than OptF. HiX and Fex use heuristic-based approaches that are much more efficient than OptF and OptSmart

Γ (Gamma): cloud-based analog circuit design system

Includes bibliographical references.2016 Summer.With ever increasing demand for lower power consumption, lower cost, and higher performance, designing analog circuits to meet design specifications has become an increasing challenging task, On one hand, analog circuit designers must have intimate knowledge about the underlining silicon process technology's capability to achieve the desired specifications. On the other hand, they must understand the impact of tweaking circuits to satisfy a given specification on all circuit performance parameters. Analog designers have traditionally learned to tackle design problems with numerous circuit simulations using accurate circuit simulators such as SPICE, and have increasingly relied on trial-and-error approaches to reach a converging point. However, the increased complexity with each generation of silicon technology and high dimensionality of searching for solutions, even for some simple analog circuits, have made trial-and-error approaches extremely inefficient, causing long design cycles and often missed market opportunities. Novel rapid and accurate circuit evaluation methods that are tightly integrated with circuit search and optimization methods are needed to aid design productivity. Furthermore, the current design environment with fully distributed licensing and supporting structures is cumbersome at best to allow efficient and up-to-date support for design engineers. With increasing support and licensing costs, fewer and fewer design centers can afford it. Cloud-based software as a service (SaaS) model provides new opportunities for CAD applications. It enables immediate software delivery and update to customers at very low cost. SaaS tools benefit from fast feedback and sharing channels between users and developers and run on hardware resources tailored and provided for them by software vendors. However, web-based tools must perform in a very short turn-around schedule and be always responsive. A new class of analog design tools is presented in this dissertation. The tools provide effective design aid to analog circuit designers with a dash-board control of many important circuit parameters. Fast and accurate circuit evaluations are achieved using a novel lookup-table transistor models (LUT) with novel built-in features tightly integrated with the search engine to achieve desired speed and accuracy. This enables circuit evaluation time several orders faster than SPICE simulations. The proposed architecture for analog design attempts to break the traditional analog design flow using SPICE based trial-and-error methods by providing designers with useful information about the effects of prior design decisions they have made and potential next steps they can take to meet specifications. Benefiting from the advantages offered by web-hosted architectures, the proposed architecture incorporates SaaS as its operating model. The application of the proposed architecture is illustrated by an analog circuit sizer and optimizer. The Γ (Gamma) sizer and optimizer show how web-based design-decision supporting tool can help analog circuit designers to reduce design time and achieve high quality circuit

Функционално и императивно реактивно програмирање употребом генерализације монаде наставка у програмском језику C++.

Постоји велики број проблема који захтевају писање програмских система који имају компоненте које се извршавају међусобно асинхроно једне од других...There is a big class of problems that require software systems with asynchronously executed components..