How to evaluate multiple range-sum queries progressively

Decision support system users typically submit batches of range-sum queries simultaneously rather than issuing individual, unrelated queries. We propose a wavelet based technique that exploits I/O sharing across a query batch to evaluate the set of queries progressively and efficiently. The challenge is that now controlling the structure of errors across query results becomes more critical than minimizing error per individual query. Consequently, we define a class of structural error penalty functions and show how they are controlled by our technique. Experiments demonstrate that our technique is efficient as an exact algorithm, and the progressive estimates are accurate, even after less than one I/O per query

Highly-efficient SPT arithmetic of fast freecube calculation

分析了目前国内外数据立方体计算的研究现状,首先在free-set的概念上,挖掘free-set的性质,建立了FreeCube的概念结构。然后基于BUC算法,充分考虑到free-set的性质,在对维划分的选择和free-set判断上去掉了不必要的划分和判断,从而提出了一种计算FreeCube的高效算法SPT,最后从多个角度进行了实验,并与相关工作做了对比,证明该算法具有一定的优越性。The current domestic and international research situation of data cube calculation are analyzed.Its merits and demerits are pointed out.The free-set property is excavated and the concept construction of freecube on the free-set conception is established.With regard to freecube calculation,fully considering the free-set characteristics while combining the characteristics of BUC's cal-culation,an efficient calculation way SPT is put forward.After the related work is compared,the result show the superiority of the algorithm.浙江理工大学科学基金项目(111251A4Y04002

Verifying Completeness of Relational Query Answers from Online Servers

10.1145/1330332.1330337ACM Transactions on Information and System Security11

Consistent data aggregate retrieval for sensor network systems.

Lee Lok Hang.Thesis (M.Phil.)--Chinese University of Hong Kong, 2005.Includes bibliographical references (leaves 87-93).Abstracts in English and Chinese.Abstract --- p.iAcknowledgement --- p.ivChapter 1 --- Introduction --- p.1Chapter 1.1 --- Sensors and Sensor Networks --- p.3Chapter 1.2 --- Sensor Network Deployment --- p.7Chapter 1.3 --- Motivations --- p.7Chapter 1.4 --- Contributions --- p.9Chapter 1.5 --- Thesis Organization --- p.10Chapter 2 --- Literature Review --- p.11Chapter 2.1 --- Data Cube --- p.11Chapter 2.2 --- Data Aggregation in Sensor Networks --- p.12Chapter 2.2.1 --- Hierarchical Data Aggregation --- p.13Chapter 2.2.2 --- Gossip-based Aggregation --- p.13Chapter 2.2.3 --- Hierarchical Gossip Aggregation --- p.13Chapter 2.3 --- GAF Algorithm --- p.14Chapter 2.4 --- Concurrency Control --- p.17Chapter 2.4.1 --- Two-phase Locking --- p.17Chapter 2.4.2 --- Timestamp Ordering --- p.18Chapter 3 --- Building Distributed Data Cubes in Sensor Network --- p.20Chapter 3.1 --- Aggregation Operators --- p.21Chapter 3.2 --- Distributed Prefix (PS) Sum Data Cube --- p.22Chapter 3.2.1 --- Prefix Sum (PS) Data Cube --- p.22Chapter 3.2.2 --- Notations --- p.24Chapter 3.2.3 --- Querying a PS Data Cube --- p.25Chapter 3.2.4 --- Building Distributed PS Data Cube --- p.27Chapter 3.2.5 --- Time Bounds --- p.32Chapter 3.2.6 --- Fast Aggregate Queries on Multiple Regions --- p.37Chapter 3.2.7 --- Simulation Results --- p.43Chapter 3.3 --- Distributed Local Prefix Sum (LPS) Data Cube --- p.50Chapter 3.3.1 --- Local Prefix Sum Data Cube --- p.52Chapter 3.3.2 --- Notations --- p.55Chapter 3.3.3 --- Querying an LPS Data Cube --- p.56Chapter 3.3.4 --- Building Distributed LPS Data Cube --- p.61Chapter 3.3.5 --- Time Bounds --- p.63Chapter 3.3.6 --- Fast Aggregate Queries on Multiple Regions --- p.67Chapter 3.3.7 --- Simulation Results --- p.68Chapter 3.3.8 --- Distributed PS Data Cube Vs Distributed LPS Data Cube --- p.74Chapter 4 --- Concurrency Control and Consistency in Sensor Networks --- p.76Chapter 4.1 --- Data Inconsistency in Sensor Networks --- p.76Chapter 4.2 --- Traditional Concurrency Control Protocols and Sensor Networks --- p.80Chapter 4.3 --- The Consistent Retrieval of Data from Distributed Data Cubes --- p.81Chapter 5 --- Conclusions --- p.85References --- p.87Appendix --- p.94A Publications --- p.9

Reusing dynamic data marts for query management in an on-demand ETL architecture

Data analysts working often have a requirement to integrate an in-house data warehouse with external datasets, especially web-based datasets. Doing so can give them important insights into their performance when compared with competitors, their industry in general on a global scale, and make predictions as to sales, providing important decision support services. The quality of these insights depends on the quality of the data imported into the analysis dataset. There is a wealth of data freely available from government sources online but little unity between data sources, leading to a requirement for a data processing layer wherein various types of quality issues and heterogeneities can be resolved. Traditionally, this is achieved with an Extract-Transform-Load (ETL) series of processes which are performed on all of the available data, in advance, in a batch process typically run outside of business hours. While this is recognized as a powerful knowledge-based support, it is very expensive to build and maintain, and is very costly to update, in the event that new data sources become available. On-demand ETL offers a solution in that data is only acquired when needed and new sources can be added as they come online. However, this form of dynamic ETL is very difficult to deliver. In this research dissertation, we explore the possibilities of creating dynamic data marts which can be created using non-warehouse data to support the inclusion of new sources. We then examine how these dynamic structures can be used for query fulfillment andhow they can support an overall on-demand query mechanism. At each step of the research and development, we employ a robust validation using a real-world data warehouse from the agricultural domain with selected Agri web sources to test the dynamic elements of the proposed architecture