Adaptive indexing in modern database kernels

Physical design represents one of the hardest problems for database management systems. Without proper tuning, systems cannot achieve good performance. Offline indexing creates indexes a priori assuming good workload knowledge and idle time. More recently, online indexing monitors the workload trends and creates or drops indexes online. Adaptive indexing takes another step towards completely automating the tuning process of a database system, by enabling incremental and partial online indexing. The main idea is that physical design changes continuously, adaptively, partially, incrementally and on demand while processing queries as part of the execution operators. As such it brings a plethora of opportunities for rethinking and improving every single corner of database system design. We will analyze the indexing space between offline, online and adaptive indexing through several state of the art indexing techniques, e. g., what-if analysis and soft indexes. We will discuss in detail adaptive indexing techniques such as database cracking, adaptive merging, sideways cracking and various hybrids that try to balance the online tuning overhead with the convergence speed to optimal performance. In addition, we will discuss how various aspects of modern techniques for database architectures, such as vectorization, bulk processing, column-store execution and storage affect adaptive indexing. Finally, we will discuss several open research topics towards fully automomous database kernels

Benchmarking adaptive indexing

Ideally, realizing the best physical design for the current and all subsequent workloads would impact neither performance nor storage usage. In reality, workloads and datasets can change dramatically over time and index creation impacts the performance of concurrent user and system activity. We propose a framework that evaluates the key premise of adaptive indexing --- a new indexing paradigm where index creation and re-organization take place automatically and incrementally, as a side-effect of query execution. We focus on how the incremental costs and benefits of dynamic reorganization are distributed across the workload's lifetime. We believe measuring the costs and utility of the stages of adaptation are relevant metrics for evaluating new query processing paradigms and comparing them to traditional approaches

From big data to big information and big knowledge: The case of Earth observation data

Some particularly important rich sources of open and free big geospatial data are the Earth observation (EO) programs of various countries such as the Landsat program of the US and the Copernicus programme of the European Union. EO data is a paradigmatic case of big data and the same is true for the big information and big knowledge extracted from it. EO data (satellite images and in-situ data), and the information and knowledge extracted from it, can be utilized in many applications with financial and environmental impact in areas such as emergency management, climate change, agriculture and security

Quality predictors of abdominal fetal electrocardiography recording in antenatal ambulatory and bedside settings

Background: Fetal electrocardiography using an abdominal monitor (Monica AN24™) could increase the diagnostic use of fetal heart rate (fHR) variability measurements. However, signal quality may depend on factors such as maternal physical activity, posture, and bedside versus ambulatory setting. Methods: Sixty-three healthy women wore the monitor at home and 42 women during a hospital stay. All women underwent a posture experiment, and all home and 13 hospital participants wore the monitor during daytime and nighttime. The success rate (SR) of fHR detection was analyzed in relation to maternal physical activity, posture, daytime versus nighttime, and other maternal and fetal predictors. Results: Ambulatorily, the SR was 86.8% for nighttime and 40.2% for daytime. The low daytime SR was largely due to effects of maternal physical activity and posture. The in-hospital SR was lower during nighttime (71.1%) and similar during daytime (43.3%). SR was related to gestational age, but not affected by pre-pregnancy and current body mass index or fetal growth restriction. Conclusions: The success of beat-to-beat fHR detection strongly depends on the home/hospital setting and predictors such as time of recording, activity levels, and maternal posture. Its clinical utility may be limited in periods of unsupervised recording with physical activity or posture shifts

Transactional support for adaptive indexing

Adaptive indexing initializes and optimizes indexes incrementally, as a side effect of query processing. The goal is to achieve the benefits of indexes while hiding or minimizing the costs of index creation. However, index-optimizing side effects seem to turn read-only queries into update transactions that might, for example, create lock contention. This paper studies concurrency contr

Forecasting the cost of processing multi-join queries via hashing for main-memory databases (Extended version)

Database management systems (DBMSs) carefully optimize complex multi-join queries to avoid expensive disk I/O. As servers today feature tens or hundreds of gigabytes of RAM, a significant fraction of many analytic databases becomes memory-resident. Even after careful tuning for an in-memory environment, a linear disk I/O model such as the one implemented in PostgreSQL may make query response time predictions that are up to 2X slower than the optimal multi-join query plan over memory-resident data. This paper introduces a memory I/O cost model to identify good evaluation strategies for complex query plans with multiple hash-based equi-joins over memory-resident data. The proposed cost model is carefully validated for accuracy using three different systems, including an Amazon EC2 instance, to control for hardware-specific differences. Prior work in parallel query evaluation has advocated right-deep and bushy trees for multi-join queries due to their greater parallelization and pipelining potential. A surprising finding is that the conventional wisdom from shared-nothing disk-based systems does not directly apply to the modern shared-everything memory hierarchy. As corroborated by our model, the performance gap between the optimal left-deep and right-deep query plan can grow to about 10X as the number of joins in the query increases.Comment: 15 pages, 8 figures, extended version of the paper to appear in SoCC'1

Подбор механического метода борьбы с асфальтово-парафиновыми отложениями

В дипломной работе рассмотрены способы образования отложений парафина, механизмы образования АСПО, современные способы борьбы с отложениями, также проведен патентный поиск. Приведенный в дипломной работе расчет показывает, что для данной скважины применение скребков – центраторов целесообразно, условие прочности выполняется. Результатом проведенной дипломной работы является расчет на прочность штанг ШСНУ. В работе рассмотрено, как параметры скважины влияют на выпадение парафина из нефти.In the thesis work the methods of paraffin deposition, mechanisms of AFS, modern ways of dealing with deposits, also held a patent search. Powered by a research paper calculation shows that for this well use scrapers - centralizers appropriate strength condition is satisfied. The result of conducted research paper is to calculate the strength SHSNU rods. The paper considers how well the parameters affect the loss of paraffin oil

Thinking Big in a Small World — Efficient Query Execution on Small-Scale SMPs

Many techniques developed for parallel database systems were focused on large-scale, often prototypical, hardware platforms. Therefore, most results cannot easily be transfered to widely available workstation clusters such as multiprocessor workstations. In this paper we address exploitation of pipelining parallelism in query processing on small multiprocessor environments. We present DTE/R, a strategy for executing pipelining segments of arbitrary length by replicating the segment's operator. Therefore, DTE/R avoids static processor-to-operator assignment of conventional processing techniques. Consequently, DTE/R achieves automatic load-balancing and skew-handling. Furthermore, DTE/R outperforms conventional pipelining execution techniques substantially

Single-agent gemcitabine versus cisplatin-etoposide: Early results of a randomised phase II study in locally advanced or metastatic non-small-cell lung cancer

Background This randomised study was designed to determine the response rate, survival and toxicity of single-agent gemcitabine and cisplatin-etoposide in chemo-naïve patients with locally advanced or metastatic non-small-cell lung cancer. Patients and methods Gemcitabine 1,000 mg/m2 was given as a 30 min intravenous infusion on days 1, 8, 15 of a 28-day cycle, cisplatin 100 mg/m2 on day 1, and etoposide 100 mg/m2 on days 1 (following cisplatin), 2 and 3. Major eligibility criteria included histologically confirmed non-small-cell lung cancer, measurable disease, Zubrod PS 0-2; no prior chemotherapy, no prior radiation of the measured lesion, and no CNS metastases. Results 146 patients were enrolled, 71 patients on gemcitabine and 75 patients on cisplatin-etoposide. Patient characteristics were well matched across both arms. Sixty-six gemcitabine patients and 72 cisplatin-etoposide patients were evaluable. Partial responses were seen in 12 gemcitabine patients (18.2%; 95% CI: 9.8-30) and 11 cisplatin-etoposide patients (15.3%; 95% CI: 7.9-25.7). Early indications show no statistical differences between the two treatments with respect to time to disease progression or survival. Haematological and laboratory toxicity were moderate and manageable. However, hospitalisation because of neutropenic fever was required for 6 (8%) cisplatin-etoposide patients but not for any gemcitabine patients. Non-haematological toxicity was more pronounced with significant differences in nausea and vomiting (grade 3 and 4: 11% gemcitabine vs. 29% cisplatin-etoposide; despite the allowance for 5-HT3 antiemetics during the first cycle of cisplatin-etoposide), and alopecia (grade 3 and 4: 3% gemcitabine vs. 62% cisplatin-etoposide). Conclusions In this randomised study, single-agent gemcitabine was at least as active but better tolerated than the combination cisplatin-etoposid