A New Efficient Privacy For A Multi-Skyline Queries With Mapreduce

The skyline query technology has pulled in much consideration as of late. This is for the most part because of the significance of horizon brings about numerous applications, for example, multi-criteria basic leadership, information mining, and data suggested frameworks. The horizon administrator has pulled in impressive consideration as of late because of its wide applications. Be that as it may, registering a horizon is testing today since we need to manage enormous information. For information escalated applications, the MapReduce system has been broadly utilized as of late. In this paper, what's more, we apply the strength control sifting technique to viably prune non-horizon focuses ahead of time. We next segment information in light of the districts separated by the quad tree and figure hopeful horizon focuses for each segment utilizing MapReduce. At long last, we propose a productive technique for preparing multi-horizon questions with MapReduce with no change of the Hadoop internals. Through different analyses, we demonstrate that our approach beats past investigations by requests of exten

A Review On Hadoop: Privacy For A Multi-Skyline Queries With Map Reduce

The significance of skyline  brings about numerous applications, for example, multi-criteria basic leadership, information mining, and data prescribed frameworks. Horizon inquiries are valuable for finding intriguing tuples from an extensive informational collection as indicated by different criteria. The sizes of informational collections are always expanding and the design of back-closes are changing from single-hub situations to non-traditional ideal models like MapReduce The horizon administrator has pulled in impressive consideration as of late because of its wide applications. In any case, processing a horizon is testing today since we need to manage huge information. For information concentrated applications, the MapReduce structure has been broadly utilized as of late. In this paper, also, we apply the strength control sifting technique to adequately prune non-horizon focuses ahead of time. We next parcel information in light of the areas separated by the quad tree and process competitor horizon focuses for each segment utilizing MapReduce.   At long last, MapReduce Grid Partitioning based Single-Reducer Skyline Computation (MR-GPSRS) utilizes a solitary reducer to amass the neighborhood horizons properly to figure the worldwide horizon. Conversely, MapReduce Grid Partitioning based Multiple Reducer Skyline Computation (MR-GPMRS) additionally separates neighborhood horizons and disperses them to different reducers that process the worldwide horizon in a free and parallel way. The proposed calculations are assessed through broad analyses, and the outcomes demonstrate that MR-GPMRS fundamentally beats the choices in different settings. we propose an effective technique for preparing multi-horizon inquiries with MapReduce with no alteration of the Hadoop internals. Through different analyses, we demonstrate that our approach beats past examinations by requests of extent

Reverse Skyline Computation over Sliding Windows

Reverse skyline queries have been used in many real-world applications such as business planning, market analysis, and environmental monitoring. In this paper, we investigated how to efficiently evaluate continuous reverse skyline queries over sliding windows. We first theoretically analyzed the inherent properties of reverse skyline on data streams and proposed a novel pruning technique to reduce the number of data points preserved for processing continuous reverse skyline queries. Then, an efficient approach, called Semidominance Based Reverse Skyline (SDRS), was proposed to process continuous reverse skyline queries. Moreover, an extension was also proposed to handle n-of-N and (n1,n2)-of-N reverse skyline queries. Our extensive experimental studies have demonstrated the efficiency as well as effectiveness of the proposed approach with various experimental settings

An Energy-Efficient Skyline Query for Massively Multidimensional Sensing Data

Cyber physical systems (CPS) sense the environment based on wireless sensor networks. The sensing data of such systems present the characteristics of massiveness and multi-dimensionality. As one of the major monitoring methods used in in safe production monitoring and disaster early-warning applications, skyline query algorithms are extensively adopted for multiple-objective decision analysis of these sensing data. With the expansion of network sizes, the amount of sensing data increases sharply. Then, how to improve the query efficiency of skyline query algorithms and reduce the transmission energy consumption become pressing and difficult to accomplish issues. Therefore, this paper proposes a new energy-efficient skyline query method for massively multidimensional sensing data. First, the method uses a node cut strategy to dynamically generate filtering tuples with little computational overhead when collecting query results instead of issuing queries with filters. It can judge the domination relationship among different nodes, remove the detected data sets of dominated nodes that are irrelevant to the query, modify the query path dynamically, and reduce the data comparison and computational overhead. The efficient dynamic filter generated by this strategy uses little non-skyline data transmission in the network, and the transmission distance is very short. Second, our method also employs the tuple-cutting strategy inside the node and generates the local cutting tuples by the sub-tree with the node itself as the root node, which will be used to cut the detected data within the nodes of the sub-tree. Therefore, it can further control the non-skyline data uploading. A large number of experimental results show that our method can quickly return an overview of the monitored area and reduce the communication overhead. Additionally, it can shorten the response time and improve the efficiency of the query

Energy-Efficient Reverse Skyline Query Processing over Wireless Sensor Networks

Reverse skyline query plays an important role in many sensing applications, such as environmental monitoring, habitat monitoring, and battlefield monitoring. Due to the limited power supplies of wireless sensor nodes, the existing centralized approaches, which do not consider energy efficiency, cannot be directly applied to the distributed sensor environment. In this paper, we investigate how to process reverse skyline queries energy efficiently in wireless sensor networks. Initially, we theoretically analyzed the properties of reverse skyline query and proposed a skyband-based approach to tackle the problem of reverse skyline query answering over wireless sensor networks. Then, an energy-efficient approach is proposed to minimize the communication cost among sensor nodes of evaluating range reverse skyline query. Moreover, optimization mechanisms to improve the performance of multiple reverse skylines are also discussed. Extensive experiments on both real-world data and synthetic data have demonstrated the efficiency and effectiveness of our proposed approaches with various experimental settings

빅데이터의 효율적인 스카이라인 질의 처리를 위한 병렬처리 알고리즘

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2017. 2. 심규석.스카이라인 질의와 스카이라인에서 파생된 동적 스카이라인, 역 스카이라인 그리고 확률적 스카이라인 질의들은 다양한 응용이 가능하기 때문에 최근에 많은 연구가 진행되어 왔다. 스카이라인 질의들은 큰 데이터를 처리해야하는 경우가 많기 때문에 효율적인 스카이라인 질의 처리는 중요한 문제이다. 큰 데이터를 처리해야하는 경우를 위해 맵리듀스 프레임워크가 제안되었고, 따라서 본 논문에서는 스카이라인, 동적 스카이라인, 역 스카이라인, 확률적 스카이라인 질의 처리를 위한 효율적인 맵리듀스 알고리즘을 개발한다. 스카이라인, 동적 스카이라인, 역 스카이라인에 대해서는 질의 결과에 포함될 수 없는 데이터를 빠르게 제거하기 위해서 쿼드트리에 기반한 히스토그램을 생성한다. 그리고 히스토그램에 따라 데이터를 여러 파티션으로 나누고 각 파티션에 있는 데이터만을 이용하여 스카이라인이 될 수 있는 후보 데이터를 맵리듀스를 이용하여 병렬적으로 뽑아낸다. 그 후에 다시 맵리듀스를 사용하여 병렬적으로 후보 데이터중 실제 스카이라인을 찾아낸다. 확률적 스카이라인의 효율적인 처리를 위해 먼저 세가지 필터링 기법을 제안하였다. 이 필터링 기법을 활용할 수 있도록 쿼드트리에 기반한 히스토그램을 생성한다. 쿼드트리의 영역에 따라 데이터를 파티션하고 각 파티션마다 확률적 스카이라인 점들을 찾아낸다. 각 컴퓨터의 수행시간을 비슷하게 맞추기 위해서 부하균형 기법도 제안하였다. 다양한 실험을 통해 제안한 알고리즘의 성능들이 최신 관련 연구 보다 좋음을 확인하였고, 사용하는 컴퓨터의 수를 늘림에 따라 성능이 확장성을 갖고 있음을 확인하였다.The skyline operator and its variants such as dynamic skyline, reverse skyline and probabilistic skyline operators have attracted considerable attention recently due to its broad applications. However, computing a skyline is challenging today since we have to deal with big data. For data-intensive applications, the MapReduce framework has been widely used recently. In this dissertation, we propose the efficient parallel algorithms for processing skyline, dynamic skyline, reverse skyline and probabilistic skyline queries using MapReduce. For the skyline, dynamic skyline and reverse skyline queries, we first build quadtree-based histograms to prune out non-skyline points. We next partition data based on the regions divided by the histograms and compute candidate skyline points for each partition using MapReduce. Finally, in every partition, we check whether each skyline candidate point is actually a skyline point or not using MapReduce. For the probabilistic skyline query, we first introduce three filtering techniques to prune out points that are not probabilistic skyline points. Then, we build a quadtree-based histogram and split data into partitions according to the regions divided by the quadtree. We finally compute the probabilistic skyline points for each partition using MapReduce. We also develop the workload balancing methods to make the estimated execution times of all available machines to be similar. We did experiments to compare our algorithms with the state-of-the-art algorithms using MapReduce and confirmed the effectiveness as well as the scalability of our proposed skyline algorithms.1 INTRODUCTION 1 1.1 Motivation 1 1.2 Contributions of This Dissertation 6 1.3 Dissertation Overview 8 2 Related Work 10 2.1 Skyline Queries 10 2.2 Reverse Skyline Queries 13 2.3 Probabilistic Skyline Queries 14 3 Background 17 3.1 Skyline and Its Variants 17 3.2 MapReduce Framework 22 4 Parallel Skyline Query Processing 24 4.1 SKY-MR: Our Skyline Computation Algorithm 24 4.1.1 SKY-QTREE: The Sky-Quadtree Building Algorithm 25 4.1.2 L-SKY-MR: The Local Skyline Computation Algorithm 29 4.1.3 G-SKY-MR: The Global Skyline Computation Algorithm 32 4.2 Experiment 34 4.2.1 Performance Results for Skylines 36 4.2.2 Performance Results in Other Environments 41 5 Parallel Reverse Skyline Query Processing 45 5.1 RSKY-MR: Our Reverse Skyline Computation Algorithm 45 5.1.1 RSKY-QTREE: The Rsky-Quadtree Building Algorithm 47 5.1.2 Computations of Reverse Skylines using Rsky-Quadtrees 50 5.1.3 L-RSKY-MR: The Local Reverse Skyline Computation Algorithm 53 5.1.4 G-RSKY-MR: The Global Reverse Skyline Computation Algorithm 57 5.2 Experiment 59 5.2.1 Performance Results for Reverse Skylines 59 6 Parallel Probabilistic Skyline Query Processing 63 6.1 Early Pruning Techniques 63 6.1.1 Upper-bound Filtering 63 6.1.2 Zero-probability Filtering 67 6.1.3 Dominance-Power Filtering 68 6.2 Utilization of a PS-QTREE for Pruning 69 6.2.1 Generating a PS-QTREE 70 6.2.2 Exploiting a PS-QTREE for Filtering 70 6.2.3 Partitioning Objects by a PS-QTREE 71 6.3 PS-QPF-MR: Our Algorithm with Quadtree Partitiong and Filtering 73 6.3.1 Optimizations of PS-QPF-MR 79 6.3.2 Sample Size and Split Threshold of a PSQtree 83 6.4 PS-BRF-MR: Our Algorithm with Random Partitioning and Filtering 84 6.5 Experiments 87 6.5.1 Performance Results for Probabilistic Skylines 89 7 Conclusion 97 Bibliography 99 Abstract (In Korean) 105Docto

Data centric storage framework for an intelligent wireless sensor network

In the last decade research into Wireless Sensor Networks (WSN) has triggered extensive growth in flexible and previously difficult to achieve scientific activities carried out in the most demanding and often remote areas of the world. This success has provoked research into new WSN related challenges including finding techniques for data management, analysis, and how to gather information from large, diverse, distributed and heterogeneous data sets. The shift in focus to research into a scalable, accessible and sustainable intelligent sensor networks reflects the ongoing improvements made in the design, development, deployment and operation of WSNs. However, one of the key and prime pre-requisites of an intelligent network is to have the ability of in-network data storage and processing which is referred to as Data Centric Storage (DCS). This research project has successfully proposed, developed and implemented a comprehensive DCS framework for WSN. Range query mechanism, similarity search, load balancing, multi-dimensional data search, as well as limited and constrained resources have driven the research focus. The architecture of the deployed network, referred to as Disk Based Data Centric Storage (DBDCS), was inspired by the magnetic disk storage platter consisting of tracks and sectors. The core contributions made in this research can be summarized as: a) An optimally synchronized routing algorithm, referred to Sector Based Distance (SBD) routing for the DBDCS architecture; b) DCS Metric based Similarity Searching (DCSMSS) with the realization of three exemplar queries – Range query, K-nearest neighbor query (KNN) and Skyline query; and c) A Decentralized Distributed Erasure Coding (DDEC) algorithm that achieves a similar level of reliability with less redundancy. SBD achieves high power efficiency whilst reducing updates and query traffic, end-to-end delay, and collisions. In order to guarantee reliability and minimizing end-to-end latency, a simple Grid Coloring Algorithm (GCA) is used to derive the time division multiple access (TDMA) schedules. The GCA uses a slot reuse concept to minimize the TDMA frame length. A performance evaluation was conducted with simulation results showing that SBD achieves a throughput enhancement by a factor of two, extension of network life time by 30%, and reduced end-to-end latency. DCSMSS takes advantage of a vector distance index, called iDistance, transforming the issue of similarity searching into the problem of an interval search in one dimension. DCSMSS balances the load across the network and provides efficient similarity searching in terms of three types of queries – range query, k-query and skyline query. Extensive simulation results reveal that DCSMSS is highly efficient and significantly outperforms previous approaches in processing similarity search queries. DDEC encoded the acquired information into n fragments and disseminated across n nodes inside a sector so that the original source packets can be recovered from any k surviving nodes. A lost fragment can also be regenerated from any d helper nodes. DDEC was evaluated against 3-Way Replication using different performance matrices. The results have highlighted that the use of erasure encoding in network storage can provide the desired level of data availability at a smaller memory overhead when compared to replication