279 research outputs found

    Statistical structures for internet-scale data management

    Get PDF
    Efficient query processing in traditional database management systems relies on statistics on base data. For centralized systems, there is a rich body of research results on such statistics, from simple aggregates to more elaborate synopses such as sketches and histograms. For Internet-scale distributed systems, on the other hand, statistics management still poses major challenges. With the work in this paper we aim to endow peer-to-peer data management over structured overlays with the power associated with such statistical information, with emphasis on meeting the scalability challenge. To this end, we first contribute efficient, accurate, and decentralized algorithms that can compute key aggregates such as Count, CountDistinct, Sum, and Average. We show how to construct several types of histograms, such as simple Equi-Width, Average-Shifted Equi-Width, and Equi-Depth histograms. We present a full-fledged open-source implementation of these tools for distributed statistical synopses, and report on a comprehensive experimental performance evaluation, evaluating our contributions in terms of efficiency, accuracy, and scalability

    Selected problems in cardinality estimation

    Full text link
    Cardinality estimation remains a critical task in query processing. Query optimizers rely on the accuracy of cardinality estimates when generating execution plans, and, in approximate query answering, estimated cardinalities affect the quality of query results. In this thesis, we present multiple new cardinality estimation techniques. The techniques differ vastly by the query under consideration. For single relation queries, we use the principle of maximum entropy to combine information extracted from samples and histograms. For join size estimation, we rely on a model that requires one to find estimates for the intersection size of join attributes. For queries with multiple joins, sketches serve as compact representations of join results that are combined via a data structure that approximates the joint frequency distribution of join attributes. In addition, we present a technique to transform selection predicates into a representation that allows estimators based on machine learning to effectively learn query result cardinalities. For each cardinality estimator presented in this thesis, we precisely define its problem scope, the construction process, and how to obtain estimates. Then, we compare to state-of-the-art cardinality estimators and run a thorough evaluation with queries over multiple data sets. Based on our observations, we analyze the strengths and limitations of each of our cardinality estimators and identify its preferred use case

    Rank-aware, Approximate Query Processing on the Semantic Web

    Get PDF
    Search over the Semantic Web corpus frequently leads to queries having large result sets. So, in order to discover relevant data elements, users must rely on ranking techniques to sort results according to their relevance. At the same time, applications oftentimes deal with information needs, which do not require complete and exact results. In this thesis, we face the problem of how to process queries over Web data in an approximate and rank-aware fashion
    corecore