EFFICIENT SKYLINE SYSTEM DEVELOPMENT FOR NORMAL AND HIDDEN DATABASES: APPLICATION FOR GOOGLE FLIGHTS

Deep web databases provide strict search interface and limited web access with top-k results based on a pre-defined ranking function. However, top-k results may not be suitable for multi-criteria decision making because of the variety in preferences. To make the results more relevant to such a decision maker, skyline records were introduced, and as per definition these records are not dominated by any other record such that a record dominates another if it is better or as good as other for all attributes and better in at least one attribute. In this report, we introduce an algorithm for discovering skyline records from hidden databases using different multi-objective attributes on a real-world database. We predicted a new lower bound for the minimum issued number of queries to extract the skyline. This was supported by our algorithm which accomplished the above task in an efficient manner including the worst-case scenario hence proving our theory via running rigorous experiments on a hidden database given the limitations on hand.This contribution was made possible by NPRP grant #07- 794-1-145 from the Qatar National Research Fund (a member of Qatar Foundation). The statements made herein are solely the responsibility of the authors

Scalable parallelization of skyline computation for multi-core processors

The skyline is an important query operator for multi-criteria decision making. It reduces a dataset to only those points that offer optimal trade-offs of dimensions. In general, it is very expensive to compute. Recently, multicore CPU algorithms have been proposed to accelerate the computation of the skyline. However, they do not sufficiently minimize dominance tests and so are not competitive with state-of-the-art sequential algorithms. In this paper, we introduce a novel multicore skyline algorithm, Hybrid, which processes points in blocks. It maintains a shared, global skyline among all threads, which is used to minimize dominance tests while maintaining high throughput. The algorithm uses an efficiently-updatable data structure over the shared, global skyline, based on point-based partitioning. Also, we release a large benchmark of optimized skyline algorithms, with which we demonstrate on challenging workloads a 100-fold speedup over state-of-the-art multicore algorithms and a 10-fold speedup with 16 cores over state-of-the-art sequential algorithms