Distributed top-k aggregation queries at large

Top-k query processing is a fundamental building block for efficient ranking in a large number of applications. Efficiency is a central issue, especially for distributed settings, when the data is spread across different nodes in a network. This paper introduces novel optimization methods for top-k aggregation queries in such distributed environments. The optimizations can be applied to all algorithms that fall into the frameworks of the prior TPUT and KLEE methods. The optimizations address three degrees of freedom: 1) hierarchically grouping input lists into top-k operator trees and optimizing the tree structure, 2) computing data-adaptive scan depths for different input sources, and 3) data-adaptive sampling of a small subset of input sources in scenarios with hundreds or thousands of query-relevant network nodes. All optimizations are based on a statistical cost model that utilizes local synopses, e.g., in the form of histograms, efficiently computed convolutions, and estimators based on order statistics. The paper presents comprehensive experiments, with three different real-life datasets and using the ns-2 network simulator for a packet-level simulation of a large Internet-style network

Pheromone-based In-Network Processing for wireless sensor network monitoring systems

Monitoring spatio-temporal continuous fields using wireless sensor networks (WSNs) has emerged as a novel solution. An efficient data-driven routing mechanism for sensor querying and information gathering in large-scale WSNs is a challenging problem. In particular, we consider the case of how to query the sensor network information with the minimum energy cost in scenarios where a small subset of sensor nodes has relevant readings. In order to deal with this problem, we propose a Pheromone-based In-Network Processing (PhINP) mechanism. The proposal takes advantages of both a pheromone-based iterative strategy to direct queries towards nodes with relevant information and query- and response-based in-network filtering to reduce the number of active nodes. Additionally, we apply reinforcement learning to improve the performance. The main contribution of this work is the proposal of a simple and efficient mechanism for information discovery and gathering. It can reduce the messages exchanged in the network, by allowing some error, in order to maximize the network lifetime. We demonstrate by extensive simulations that using PhINP mechanism the query dissemination cost can be reduced by approximately 60% over flooding, with an error below 1%, applying the same in-network filtering strategy.Fil: Riva, Guillermo Gaston. Universidad Nacional de Córdoba. Facultad de Ciencias Exactas, Físicas y Naturales; Argentina. Universidad Tecnológica Nacional; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; ArgentinaFil: Finochietto, Jorge Manuel. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Estudios Avanzados en Ingeniería y Tecnología. Universidad Nacional de Córdoba. Facultad de Ciencias Exactas Físicas y Naturales. Instituto de Estudios Avanzados en Ingeniería y Tecnología; Argentin