85,843 research outputs found
Stochastic Database Cracking: Towards Robust Adaptive Indexing in Main-Memory Column-Stores
Modern business applications and scientific databases call for inherently
dynamic data storage environments. Such environments are characterized by two
challenging features: (a) they have little idle system time to devote on
physical design; and (b) there is little, if any, a priori workload knowledge,
while the query and data workload keeps changing dynamically. In such
environments, traditional approaches to index building and maintenance cannot
apply. Database cracking has been proposed as a solution that allows on-the-fly
physical data reorganization, as a collateral effect of query processing.
Cracking aims to continuously and automatically adapt indexes to the workload
at hand, without human intervention. Indexes are built incrementally,
adaptively, and on demand. Nevertheless, as we show, existing adaptive indexing
methods fail to deliver workload-robustness; they perform much better with
random workloads than with others. This frailty derives from the inelasticity
with which these approaches interpret each query as a hint on how data should
be stored. Current cracking schemes blindly reorganize the data within each
query's range, even if that results into successive expensive operations with
minimal indexing benefit. In this paper, we introduce stochastic cracking, a
significantly more resilient approach to adaptive indexing. Stochastic cracking
also uses each query as a hint on how to reorganize data, but not blindly so;
it gains resilience and avoids performance bottlenecks by deliberately applying
certain arbitrary choices in its decision-making. Thereby, we bring adaptive
indexing forward to a mature formulation that confers the workload-robustness
previous approaches lacked. Our extensive experimental study verifies that
stochastic cracking maintains the desired properties of original database
cracking while at the same time it performs well with diverse realistic
workloads.Comment: VLDB201
Adaptive Merging on Phase Change Memory
Indexing is a well-known database technique used to facilitate data access
and speed up query processing. Nevertheless, the construction and modification
of indexes are very expensive. In traditional approaches, all records in the
database table are equally covered by the index. It is not effective, since
some records may be queried very often and some never. To avoid this problem,
adaptive merging has been introduced. The key idea is to create index
adaptively and incrementally as a side-product of query processing. As a
result, the database table is indexed partially depending on the query
workload. This paper faces a problem of adaptive merging for phase change
memory (PCM). The most important features of this memory type are: limited
write endurance and high write latency. As a consequence, adaptive merging
should be investigated from the scratch. We solve this problem in two steps.
First, we apply several PCM optimization techniques to the traditional adaptive
merging approach. We prove that the proposed method (eAM) outperforms a
traditional approach by 60%. After that, we invent the framework for adaptive
merging (PAM) and a new PCM-optimized index. It further improves the system
performance by 20% for databases where search queries interleave with data
modifications
Distributed top-k aggregation queries at large
Top-k query processing is a fundamental building block for efficient ranking in a large number of applications. Efficiency is a central issue, especially for distributed settings, when the data is spread across different nodes in a network. This paper introduces novel optimization methods for top-k aggregation queries in such distributed environments. The optimizations can be applied to all algorithms that fall into the frameworks of the prior TPUT and KLEE methods. The optimizations address three degrees of freedom: 1) hierarchically grouping input lists into top-k operator trees and optimizing the tree structure, 2) computing data-adaptive scan depths for different input sources, and 3) data-adaptive sampling of a small subset of input sources in scenarios with hundreds or thousands of query-relevant network nodes. All optimizations are based on a statistical cost model that utilizes local synopses, e.g., in the form of histograms, efficiently computed convolutions, and estimators based on order statistics. The paper presents comprehensive experiments, with three different real-life datasets and using the ns-2 network simulator for a packet-level simulation of a large Internet-style network
Adaptive work placement for query processing on heterogeneous computing resources
The hardware landscape is currently changing from homogeneous multi-core systems towards heterogeneous systems with many di↵erent computing units, each with their own characteristics. This trend is a great opportunity for database systems to increase the overall performance if the heterogeneous resources can be utilized eciently. To achieve this, the main challenge is to place the right work on the right computing unit. Current approaches tackling this placement for query processing assume that data cardinalities of intermediate results can be correctly estimated. However, this assumption does not hold for complex queries. To overcome this problem, we propose an adaptive placement approach being independent of cardinality estimation of intermediate results. Our approach is incorporated in a novel adaptive placement sequence. Additionally, we implement our approach as an extensible virtualization layer, to demonstrate the broad applicability with multiple database systems. In our evaluation, we clearly show that our approach significantly improves OLAP query processing on heterogeneous hardware, while being adaptive enough to react to changing cardinalities of intermediate query results
Adaptive network protocols to support queries in dynamic networks
Recent technological advancements have led to the popularity of mobile devices, which can dynamically form wireless networks. In order to discover and obtain distributed information, queries are widely used by applications in opportunistically formed mobile networks. Given the popularity of this approach, application developers can choose from a number of implementations of query processing protocols to support the distributed execution of a query over the network. However, different inquiry strategies (i.e., the query processing protocol and associated parameters used to execute a query) have different tradeoffs between the quality of the query's result and the cost required for execution under different operating conditions. The application developer's choice of inquiry strategy is important to meet the application's needs while considering the limited resources of the mobile devices that form the network. We propose adaptive approaches to choose the most appropriate inquiry strategy in dynamic mobile environments. We introduce an architecture for adaptive queries which employs knowledge about the current state of the dynamic mobile network and the history of previous query results to learn the most appropriate inquiry strategy to balance quality and cost tradeoffs in a given setting, and use this information to dynamically adapt the continuous query's execution
- …