Optimized cost effective approach for selection of materialized views in data warehousing

A data warehouse efficiently processes a given set of queries by utilizing the multiple materialized views. Owing to the constraint on space and maintenance cost, the materialization of all views is unfeasible. One of the critical decisions involved in the process of designing a data warehouse for optimal efficiency, is the materialized views selection. The primary goal of data warehousing is to select a suitable set of views that minimizes the total cost associated with the materialized views. In this paper, we have presented a framework, an optimized version of our previous work, for the selection of views to materialize, for a given storage space constraints, which intends to achieve the best combination of good query response, low query processing cost and low view maintenance cost. All the cost metrics associated with the materialized views selection that comprise the query execution frequencies, base-relation update frequencies, query access costs, view maintenance costs and the system's storage space constraints are considered by this framework. This framework optimizes the maintenance, storage and query processing cost as it selects the most cost effective views to materialize. Thus, an efficient data warehousing system is the outcome.Facultad de Informátic

Materializing views in data warehouse: an efficient approach to OLAP.

Gou Gang.Thesis (M.Phil.)--Chinese University of Hong Kong, 2003.Includes bibliographical references (leaves 83-87).Abstracts in English and Chinese.Acknowledgement --- p.iiiChapter 1 --- Introduction --- p.1Chapter 1.1 --- Data Warehouse and OLAP --- p.4Chapter 1.2 --- Computational Model: Dependent Lattice --- p.10Chapter 1.3 --- Materialized View Selection --- p.12Chapter 1.3.1 --- Materialized View Selection under a Disk-Space Constraint --- p.13Chapter 1.3.2 --- Materialized View Selection under a Maintenance-Time Con- straint --- p.16Chapter 1.4 --- Main Contributions --- p.21Chapter 2 --- A* Search: View Selection under a Disk-Space Constraint --- p.24Chapter 2.1 --- The Weakness of Greedy Algorithms --- p.25Chapter 2.2 --- A*-algorithm --- p.29Chapter 2.2.1 --- An Estimation Function --- p.36Chapter 2.2.2 --- Pruning Feasible Subtrees --- p.38Chapter 2.2.3 --- Approaching the Optimal Solution from Two Directions --- p.41Chapter 2.2.4 --- NIBS Order: Accelerating Convergence --- p.43Chapter 2.2.5 --- Sliding Techniques: Eliminating Redundant H-Computation --- p.45Chapter 2.2.6 --- Examples --- p.50Chapter 2.3 --- Experiment Results --- p.54Chapter 2.3.1 --- Analysis of Experiment Results --- p.55Chapter 2.3.2 --- Computing for a Series of S Constraints --- p.60Chapter 2.4 --- Conclusions --- p.62Chapter 3 --- Randomized Search: View Selection under a Maintenance-Time Constraint --- p.64Chapter 3.1 --- Non-monotonic Property --- p.65Chapter 3.2 --- A Stochastic-Ranking-Based Evolutionary Algorithm --- p.67Chapter 3.2.1 --- A Basic Evolutionary Algorithm --- p.68Chapter 3.2.2 --- The Weakness of the rg-Method --- p.69Chapter 3.2.3 --- Stochastic Ranking: a Novel Constraint Handling Technique --- p.70Chapter 3.2.4 --- View Selection Using the Stochastic-Ranking-Based Evolu- tionary Algorithm --- p.72Chapter 3.3 --- Conclusions --- p.74Chapter 4 --- Conclusions --- p.75Chapter 4.1 --- Thesis Review --- p.76Chapter 4.2 --- Future Work --- p.78Chapter A --- My Publications for This Thesis --- p.81Bibliography --- p.8

Maintenance-cost view-selection in large data warehouse systems: algorithms, implementations and evaluations.

Choi Chi Hon.Thesis (M.Phil.)--Chinese University of Hong Kong, 2003.Includes bibliographical references (leaves 120-126).Abstracts in English and Chinese.Abstract --- p.iAbstract (Chinese) --- p.iiAcknowledgement --- p.iiiContents --- p.ivList of Figures --- p.viiiList of Tables --- p.xChapter 1 --- Introduction --- p.1Chapter 1.1 --- Maintenance Cost View Selection Problem --- p.2Chapter 1.2 --- Previous Research Works --- p.3Chapter 1.3 --- Major Contributions --- p.4Chapter 1.4 --- Thesis Organization --- p.6Chapter 2 --- Literature Review --- p.7Chapter 2.1 --- Data Warehouse and OLAP Systems --- p.8Chapter 2.1.1 --- What Is Data Warehouse? --- p.8Chapter 2.1.2 --- What Is OLAP? --- p.10Chapter 2.1.3 --- Difference Between Operational Database Systems and OLAP --- p.10Chapter 2.1.4 --- Data Warehouse Architecture --- p.12Chapter 2.1.5 --- Multidimensional Data Model --- p.13Chapter 2.1.6 --- Star Schema and Snowflake Schema --- p.15Chapter 2.1.7 --- Data Cube --- p.17Chapter 2.1.8 --- ROLAP and MOLAP --- p.19Chapter 2.1.9 --- Query Optimization --- p.20Chapter 2.2 --- Materialized View --- p.22Chapter 2.2.1 --- What Is A Materialized View --- p.23Chapter 2.2.2 --- The Role of Materialized View in OLAP --- p.23Chapter 2.2.3 --- The Challenges in Exploiting Materialized View --- p.24Chapter 2.2.4 --- What Is View Maintenance --- p.25Chapter 2.3 --- View Selection --- p.27Chapter 2.3.1 --- Selection Strategy --- p.27Chapter 2.4 --- Summary --- p.32Chapter 3 --- Problem Definition --- p.33Chapter 3.1 --- View Selection Under Constraint --- p.33Chapter 3.2 --- The Lattice Framework for Maintenance Cost View Selection Prob- lem --- p.35Chapter 3.3 --- The Difficulties of Maintenance Cost View Selection Problem --- p.39Chapter 3.4 --- Summary --- p.41Chapter 4 --- What Difference Heuristics Make --- p.43Chapter 4.1 --- Motivation --- p.44Chapter 4.2 --- Example --- p.46Chapter 4.3 --- Existing Algorithms --- p.49Chapter 4.3.1 --- A*-Heuristic --- p.51Chapter 4.3.2 --- Inverted-Tree Greedy --- p.52Chapter 4.3.3 --- Two-Phase Greedy --- p.54Chapter 4.3.4 --- Integrated Greedy --- p.57Chapter 4.4 --- A Performance Study --- p.60Chapter 4.5 --- Summary --- p.68Chapter 5 --- Materialized View Selection as Constrained Evolutionary Opti- mization --- p.71Chapter 5.1 --- Motivation --- p.72Chapter 5.2 --- Evolutionary Algorithms --- p.73Chapter 5.2.1 --- Constraint Handling: Penalty v.s. Stochastic Ranking --- p.74Chapter 5.2.2 --- The New Stochastic Ranking Evolutionary Algorithm --- p.78Chapter 5.3 --- Experimental Studies --- p.81Chapter 5.3.1 --- Experimental Setup --- p.82Chapter 5.3.2 --- Experimental Results --- p.82Chapter 5.4 --- Summary --- p.89Chapter 6 --- Dynamic Materialized View Management Based On Predicates --- p.90Chapter 6.1 --- Motivation --- p.91Chapter 6.2 --- Examples --- p.93Chapter 6.3 --- Related Work: Static Prepartitioning-Based Materialized View Management --- p.96Chapter 6.4 --- A New Dynamic Predicate-based Partitioning Approach --- p.99Chapter 6.4.1 --- System Overview --- p.102Chapter 6.4.2 --- Partition Advisor --- p.103Chapter 6.4.3 --- View Manager --- p.104Chapter 6.5 --- A Performance Study --- p.108Chapter 6.5.1 --- Performance Metrics --- p.110Chapter 6.5.2 --- Feasibility Studies --- p.110Chapter 6.5.3 --- Query Locality --- p.112Chapter 6.5.4 --- The Effectiveness of Disk Size --- p.115Chapter 6.5.5 --- Scalability --- p.115Chapter 6.6 --- Summary --- p.116Chapter 7 --- Conclusions and Future Work --- p.118Bibliography --- p.12

Repetitive querying of large random heterogeneous datasets in RDBMS using materialized views

A methodology has been developed to increase time efficiency of querying large heterogeneous datasets repetitively by applying materialized views on repetitive complex queries. Additionally, a simple user interface is provided to demonstrate the utility of this research methodology. The programs demonstrate sufficiently that the core design can be used to deploy a complete system which could be used in different domains. The methodology as developed in this research is presented as an experimental proof-of-concept prototype based on an abstract design

Інформаційна технологія побудови розподілених сховищ даних гібридного типу

У дисертаційній роботі вирішено актуальне науково-практичне завдання створення інформаційної технології побудови розподілених сховищ даних гібридного типу з врахуванням властивостей даних і статистики виконання запитів до сховища. Здійснено аналіз проблеми побудови сховищ даних з врахуванням властивостей даних і виконуваних запитів, обґрунтовано актуальність вирішення цієї проблеми. Визначено вимоги до інформаційної технології побудови розподілених сховищ гібридного типу. Введено поняття мультибазових сховищ даних, розроблено концептуальну, логічну та фізичну моделі таких сховищ і процедури міжрівневих переходів. Описано інтеграцію даних у сховище за допомогою процедур перетворення елементів даних і операцій, вибору моделей представлення даних. Розташування даних по вузлах, маршрути реплікації даних визначаються за критерієм мінімальної сукупної вартості збереження та обробки даних з використанням модифікованого генетичного алгоритму. На основі запропонованих моделей і методів створено інформаційну технологію побудови розподілених сховищ гібридного типу, яка вирішує поставлене наукове завдання. Зазначена технологія застосована при розробленні інформаційних та інформаційно-аналітичних систем Міністерства фінансів України. Результати впровадження підтвердили її відповідність поставленим вимогам

Materialized View Selection as Constrained Evolutionary Optimization

One of the important issues in data warehouse development is the selection of a set of views to materialize in order to accelerate a large number of on-line analytical processing (OLAP) queries. The maintenance-cost view-selection problem is to select a set of materialized views under certain resource constraints for the purpose of minimizing the total query processing cost. However, the search space for possible materialized views may be exponentially large. A heuristic algorithm often has to be used to find a near optimal solution. In this paper, for the maintenance-cost view-selection problem, we propose a new constrained evolutionary algorithm. Constraints are incorporated into the algorithm through a stochastic ranking procedure. No penalty functions are used. Our experimental results show that the constraint handling technique, i.e., stochastic ranking, can deal with constraints effectively. Our algorithm is able to find a near-optimal feasible solution and scales with the problem size well

Modélisation des bases de données multidimensionnelles : analyse par fonctions d'agrégation multiples

Le résumé en français n'a pas été communiqué par l'auteur.Le résumé en anglais n'a pas été communiqué par l'auteur