SCALABLE PROCESSING OF MULTIPLE AGGREGATE CONTINUOUS QUERIES

Data Stream Management Systems (DSMSs) were developed to be at the heart of every monitor- ing application. Monitoring applications typically register hundreds of Continuous Queries (CQs) in DSMSs in order to continuously process unbounded data streams to detect events of interest. DSMSs must be designed to efficiently handle unbounded streams with large volumes of data and large numbers of CQs, i.e., exhibit scalability. This need for scalability means that the underlying processing techniques a DSMS adopts should be optimized for high throughput (i.e., tuple output rate). Towards this, two main approaches were proposed in the literature: (1) Multiple Query Opti- mization (MQO) and (2) Scheduling. In this dissertation we focus on optimizing the processing of multiple Aggregate Continuous Queries (ACQs), given their high processing cost and popularity in all monitoring applications. Specifically, in this dissertation, we explore shared processing of ACQs and introduce the con- cept of ’Weaveability’ as an indicator of the potential gains of sharing the processing of ACQs. We develop Weave Share, a multiple ACQs optimizer that considers the different uncorrelated factors of the processing cost, such as the input rate and ACQs’ specifications. In order to fully reap the benefits of the new weave-based optimization techniques, we conceptualize a new underlying ag- gregate operator implementation and realize it in the TriOps framework. TriOps enables adaptive sharing of multiple ACQs that have different window specification, predicates and group-by at- tributes. The properties of the proposed techniques are studied analytically and their performance advantages are experimentally evaluated using simulation and in the context of the AQSIOS DSMS prototype

Phenomenon-Aware Stream Query Processing

Spatio-temporal data streams that are generated from mobile stream sources (e.g., mobile sensors) experience similar environmental conditions that result in distinct phenomena. Several research efforts are dedicated to detect and track various phenomena inside a data stream management system (DSMS). In this paper, we use the detected phenomena to reduce the demand on the DSMS resources. The main idea is to let the query processor observe the input data streams at the phenomena level. Then, each incoming continuous query is directed only to those phenomena that participate in the query answer. Two levels of indexing are employed, a phenomenon index and a query index. The phenomenon index provides a fine resolution view of the input streams that participate in a particular phenomenon. The query index utilizes the phenomenon index to maintain a query deployment map in which each input stream is aware of the set of continuous queries that the stream contributes to their answers. Both indices are updated dynamically in response to the evolving nature of phenomena and to the mobility of the stream sources. Experimental results show the efficiency of this approach with respect to the accuracy of the query result and the resource utilization of the DSMS

Multi-dimensional phenomenon-aware stream query processing

Geographically co-located sensors tend to participate in the same environmental phenomena. Phenomenon-aware stream query processing improves scalability by subscribing each query only to a subset of sensors that participate in the phenomena of interest to that query. In the case of sensors that generate readings with a multi-attribute schema, phenomena may develop across the values of one or more attributes. However tracking and detecting phenomena across all attributes does not scale well as the dimensions increase. As the size of sensor network increases, and as the number of attributes being tracked by a sensor increases this becomes a major bottleneck. In this paper, we present a novel n-dimensional Phenomenon Detection and Tracking mechanism (termed as nd-PDT) over n-ary sensor readings. We reduce the number of dimensions to be tracked by first dropping dimensions without any meaningful phenomena, and then we further reduce the dimensionality by continuously detecting and updating various forms of functional dependencies amongst the phenomenon dimensions