47,790 research outputs found
Efficient & Effective Selective Query Rewriting with Efficiency Predictions
To enhance effectiveness, a user's query can be rewritten internally by the search engine in many ways, for example by applying proximity, or by expanding the query with related terms. However, approaches that benefit effectiveness often have a negative impact on efficiency, which has impacts upon the user satisfaction, if the query is excessively slow. In this paper, we propose a novel framework for using the predicted execution time of various query rewritings to select between alternatives on a per-query basis, in a manner that ensures both effectiveness and efficiency. In particular, we propose the prediction of the execution time of ephemeral (e.g., proximity) posting lists generated from uni-gram inverted index posting lists, which are used in establishing the permissible query rewriting alternatives that may execute in the allowed time. Experiments examining both the effectiveness and efficiency of the proposed approach demonstrate that a 49% decrease in mean response time (and 62% decrease in 95th-percentile response time) can be attained without significantly hindering the effectiveness of the search engine
Optimizing Federated Queries Based on the Physical Design of a Data Lake
The optimization of query execution plans is known to be crucial
for reducing the query execution time. In particular, query optimization
has been studied thoroughly for relational databases
over the past decades. Recently, the Resource Description Framework
(RDF) became popular for publishing data on the Web. As
a consequence, federations composed of different data models
like RDF and relational databases evolved. One type of these
federations are Semantic Data Lakes where every data source is
kept in its original data model and semantically annotated with
ontologies or controlled vocabularies. However, state-of-the-art
query engines for federated query processing over Semantic Data
Lakes often rely on optimization techniques tailored for RDF. In
this paper, we present query optimization techniques guided
by heuristics that take the physical design of a Data Lake into
account. The heuristics are implemented on top of Ontario, a
SPARQL query engine for Semantic Data Lakes. Using sourcespecific
heuristics, the query engine is able to generate more efficient
query execution plans by exploiting the knowledge about
indexes and normalization in relational databases. We show that
heuristics which take the physical design of the Data Lake into
account are able to speed up query processing
Knowledge-infused and Consistent Complex Event Processing over Real-time and Persistent Streams
Emerging applications in Internet of Things (IoT) and Cyber-Physical Systems
(CPS) present novel challenges to Big Data platforms for performing online
analytics. Ubiquitous sensors from IoT deployments are able to generate data
streams at high velocity, that include information from a variety of domains,
and accumulate to large volumes on disk. Complex Event Processing (CEP) is
recognized as an important real-time computing paradigm for analyzing
continuous data streams. However, existing work on CEP is largely limited to
relational query processing, exposing two distinctive gaps for query
specification and execution: (1) infusing the relational query model with
higher level knowledge semantics, and (2) seamless query evaluation across
temporal spaces that span past, present and future events. These allow
accessible analytics over data streams having properties from different
disciplines, and help span the velocity (real-time) and volume (persistent)
dimensions. In this article, we introduce a Knowledge-infused CEP (X-CEP)
framework that provides domain-aware knowledge query constructs along with
temporal operators that allow end-to-end queries to span across real-time and
persistent streams. We translate this query model to efficient query execution
over online and offline data streams, proposing several optimizations to
mitigate the overheads introduced by evaluating semantic predicates and in
accessing high-volume historic data streams. The proposed X-CEP query model and
execution approaches are implemented in our prototype semantic CEP engine,
SCEPter. We validate our query model using domain-aware CEP queries from a
real-world Smart Power Grid application, and experimentally analyze the
benefits of our optimizations for executing these queries, using event streams
from a campus-microgrid IoT deployment.Comment: 34 pages, 16 figures, accepted in Future Generation Computer Systems,
October 27, 201
Measuring and Managing Answer Quality for Online Data-Intensive Services
Online data-intensive services parallelize query execution across distributed
software components. Interactive response time is a priority, so online query
executions return answers without waiting for slow running components to
finish. However, data from these slow components could lead to better answers.
We propose Ubora, an approach to measure the effect of slow running components
on the quality of answers. Ubora randomly samples online queries and executes
them twice. The first execution elides data from slow components and provides
fast online answers; the second execution waits for all components to complete.
Ubora uses memoization to speed up mature executions by replaying network
messages exchanged between components. Our systems-level implementation works
for a wide range of platforms, including Hadoop/Yarn, Apache Lucene, the
EasyRec Recommendation Engine, and the OpenEphyra question answering system.
Ubora computes answer quality much faster than competing approaches that do not
use memoization. With Ubora, we show that answer quality can and should be used
to guide online admission control. Our adaptive controller processed 37% more
queries than a competing controller guided by the rate of timeouts.Comment: Technical Repor
A Delta Debugger for ILP Query Execution
Because query execution is the most crucial part of Inductive Logic
Programming (ILP) algorithms, a lot of effort is invested in developing faster
execution mechanisms. These execution mechanisms typically have a low-level
implementation, making them hard to debug. Moreover, other factors such as the
complexity of the problems handled by ILP algorithms and size of the code base
of ILP data mining systems make debugging at this level a very difficult job.
In this work, we present the trace-based debugging approach currently used in
the development of new execution mechanisms in hipP, the engine underlying the
ACE Data Mining system. This debugger uses the delta debugging algorithm to
automatically reduce the total time needed to expose bugs in ILP execution,
thus making manual debugging step much lighter.Comment: Paper presented at the 16th Workshop on Logic-based Methods in
Programming Environments (WLPE2006
Pattern based processing of XPath queries
As the popularity of areas including document storage and
distributed systems continues to grow, the demand for high
performance XML databases is increasingly evident. This
has led to a number of research eorts aimed at exploiting
the maturity of relational database systems in order to in-
crease XML query performance. In our approach, we use an
index structure based on a metamodel for XML databases
combined with relational database technology to facilitate
fast access to XML document elements. The query process
involves transforming XPath expressions to SQL which can
be executed over our optimised query engine. As there are
many dierent types of XPath queries, varying processing
logic may be applied to boost performance not only to indi-
vidual XPath axes, but across multiple axes simultaneously.
This paper describes a pattern based approach to XPath
query processing, which permits the execution of a group of
XPath location steps in parallel
- …