OLEMAR: An Online Environment for Mining Association Rules in Multidimensional Data

Data warehouses and OLAP (online analytical processing) provide tools to explore and navigate through data cubes in order to extract interesting information under different perspectives and levels of granularity. Nevertheless, OLAP techniques do not allow the identification of relationships, groupings, or exceptions that could hold in a data cube. To that end, we propose to enrich OLAP techniques with data mining facilities to benefit from the capabilities they offer. In this chapter, we propose an online environment for mining association rules in data cubes. Our environment called OLEMAR (online environment for mining association rules), is designed to extract associations from multidimensional data. It allows the extraction of inter-dimensional association rules from data cubes according to a sum-based aggregate measure, a more general indicator than aggregate values provided by the traditional COUNT measure. In our approach, OLAP users are able to drive a mining process guided by a meta-rule, which meets their analysis objectives. In addition, the environment is based on a formalization, which exploits aggregate measures to revisit the definition of the support and the confidence of discovered rules. This formalization also helps evaluate the interestingness of association rules according to two additional quality measures: lift and loevinger. Furthermore, in order to focus on the discovered associations and validate them, we provide a visual representation based on the graphic semiology principles. Such a representation consists in a graphic encoding of frequent patterns and association rules in the same multidimensional space as the one associated with the mined data cube. We have developed our approach as a component in a general online analysis platform called Miningcubes according to an Apriori-like algorithm, which helps extract inter-dimensional association rules directly from materialized multidimensional structures of data. In order to illustrate the effectiveness and the efficiency of our proposal, we analyze a real-life case study about breast cancer data and conduct performance experimentation of the mining process

Mineração de dados em sistemas OLAP

Dissertação de mestrado em Engenharia InformáticaAs diversas vantagens que os data warehouses têm proporcionado no que toca ao armazenamento e processamento de informação levaram a uma subida substancial na aquisição deste tipo de estruturas por parte das organizações. De facto, os data warehouses são caracterizados por um modelo de dados que permite, entre várias opções, realizar pesquisas complexas, selecionar conjuntos de dados de maior interesse, executar operações de sintetização, fazer comparações de dados e proporcionar diferentes visualizações dos dados. No entanto, a sua complexidade acarreta diversos custos, nomeadamente custos de computação e de materialização. Por um lado, a pré-computação de um cubo a partir de um data warehouse proporciona tempos de resposta reduzidos às pesquisas realizadas, mas, por outro lado, isso causa problemas no que toca à quantidade de espaço de armazenamento necessário. As técnicas de mineração de dados, nomeadamente aquelas que consideram os algoritmos de mineração de regras de associação, permitem encontrar conjuntos de itens frequentes entre os dados, permitindo, consequentemente, definir um conjunto de preferências de exploração ou de utilização. O estudo de preferências OLAP apresentado nesta dissertação visa identificar os dados mais acedidos por parte dos utilizadores, de forma a ser possível chegar a um consenso sobre quais as partes de um cubo que não são necessárias materializar, uma vez que não são utilizadas em processos de análise, mantendo tempos de resposta das pesquisas aceitáveis e reduzindo significativamente a quantidade de memória utilizada.The many benefits provided by data warehouses, in particular regarding to storage and data processing, have led to a substantial growth of the data warehousing market and in the number of organizations who adopted these systems. In fact, the data model of this type of structures allows the user to perform a large number of different operations: complex queries, find the most interesting information, aggregate and compare different values, and to provide an interactive data visualization. However, its complexity brings some computation and materialization costs. The pre-computation of the all data cube can provide a precise and fast response to analytical queries, but it requires an enormous quantity of space to storage all materialized views. The application of data mining techniques, such as algorithms for mining association rules, allows the discovery of frequent items among data and, consequently, the definition of OLAP preferences. The study of OLAP preferences presented in this dissertation aims to identify the most accessed parts in a data cube and to define which parts should be materialized. With the identification and materialization only of the important parts for the analysis, it is possible to preserve a satisfactory query response time, achieving a significant reduction of memory costs

Analyse multidimensionnelle interactive de résultats de simulation (aide à la décision dans le domaine de l'agroécologie)

Dans cette thèse, nous nous sommes intéressés à l'analyse des données de simulation issues du modèle agro-hydrologique TNT. Les objectifs consistaient à élaborer des méthodes d'analyse des résultats de simulation qui replacent l'utilisateur au coeur du processus décisionnel, et qui permettent d'analyser et d'interpréter de gros volumes de données de manière efficace. La démarche développée consiste à utiliser des méthodes d'analyse multidimensionnelle interactive. Tout d'abord, nous avons proposé une méthode d'archivage des résultats de simulation dans une base de données décisionnelle (i.e. entrepôt de données), adaptée au caractère spatio-temporel des données de simulation produites. Ensuite, nous avons suggéré d'analyser ces données de simulations avec des méthodes d'analyse en ligne (OLAP) afin de fournir aux acteurs des informations stratégiques pour améliorer le processus d'aide à la prise de décision. Enfin, nous avons proposé deux méthodes d'extraction de skyline dans le contexte des entrepôts de données afin de permettre aux acteurs de formuler de nouvelles questions en combinant des critères environnementaux contradictoires, et de trouver les solutions compromis associées à leurs attentes, puis d'exploiter les préférences des acteurs pour détecter et faire ressortir les données susceptibles de les intéresser. La première méthode EC2Sky, permet un calcul incrémental et efficace des skyline en présence de préférences utilisateurs dynamiques, et ce malgré de gros volumes de données. La deuxième méthode HSky, étend la recherche des points skyline aux dimensions hiérarchiques. Elle permet aux utilisateurs de naviguer le long des axes des dimensions hiérarchiques (i.e. spécialisation / généralisation) tout en assurant un calcul en ligne des points skyline correspondants. Ces contributions ont été motivées et expérimentées par l'application de gestion des pratiques agricoles pour l'amélioration de la qualité des eaux des bassins versants agricoles, et nous avons proposé un couplage entre le modèle d'entrepôt de données agro-hydrologiques construit et les méthodes d'extraction de skyline proposées.This thesis concerns the analysis of simulation data generated by the agrohydrological model TNT. Our objective is to develop analytical methods for massive simulation results. We want to place the user at the heart of the decision-making process, while letting him handle and analyze large amounts of data in a very efficient way. Our first contribution is an original approach N-Catch, relying on interactive multidimensional analysis methods for archiving simulation results in a decisional database (i.e. data warehouse) adapted to the spatio-temporal nature of the simulation data. In addition, we suggest to analyze the simulation data with online analytical methods (OLAP) to provide strategic information for stakeholders to improve the decision making process. Our second contribution concern two methods for computing skyline queries in the context of data warehouses. These methods enable stakeholders to formulate new questions by combining conflicting environmental criteria, to find compromise solutions associated with their expectations, and to exploit the stakeholder preferences to identify and highlight the data of potential interest. The first method EC2Sky, focuses on how to answer efficiently and progressively skyline queries in the presence of several dynamic user preferences despite of large volume of data. The second method HSky, extends the skyline computation to hierarchical dimensions. It allows the user to navigate along the dimensions hierarchies (i.e. specialize / generalize) while ensuring the online computation of associated skylines. Finally, we present the application of our proposals for managing agricultural practices to improve water quality in agricultural watersheds. We propose a coupling between the agro-hydrological data warehouse model N-Catch and the proposed skyline computation methods.RENNES1-Bibl. électronique (352382106) / SudocSudocFranceF

Analyse multidimensionnelle interactive de résultats de simulation (aide à la décision dans le domaine de l'agroécologie)

Dans cette thèse, nous nous sommes intéressés à l'analyse des données de simulation issues du modèle agro-hydrologique TNT. Les objectifs consistaient à élaborer des méthodes d'analyse des résultats de simulation qui replacent l'utilisateur au coeur du processus décisionnel, et qui permettent d'analyser et d'interpréter de gros volumes de données de manière efficace. La démarche développée consiste à utiliser des méthodes d'analyse multidimensionnelle interactive. Tout d'abord, nous avons proposé une méthode d'archivage des résultats de simulation dans une base de données décisionnelle (i.e. entrepôt de données), adaptée au caractère spatio-temporel des données de simulation produites. Ensuite, nous avons suggéré d'analyser ces données de simulations avec des méthodes d'analyse en ligne (OLAP) afin de fournir aux acteurs des informations stratégiques pour améliorer le processus d'aide à la prise de décision. Enfin, nous avons proposé deux méthodes d'extraction de skyline dans le contexte des entrepôts de données afin de permettre aux acteurs de formuler de nouvelles questions en combinant des critères environnementaux contradictoires, et de trouver les solutions compromis associées à leurs attentes, puis d'exploiter les préférences des acteurs pour détecter et faire ressortir les données susceptibles de les intéresser. La première méthode EC2Sky, permet un calcul incrémental et efficace des skyline en présence de préférences utilisateurs dynamiques, et ce malgré de gros volumes de données. La deuxième méthode HSky, étend la recherche des points skyline aux dimensions hiérarchiques. Elle permet aux utilisateurs de naviguer le long des axes des dimensions hiérarchiques (i.e. spécialisation / généralisation) tout en assurant un calcul en ligne des points skyline correspondants. Ces contributions ont été motivées et expérimentées par l'application de gestion des pratiques agricoles pour l'amélioration de la qualité des eaux des bassins versants agricoles, et nous avons proposé un couplage entre le modèle d'entrepôt de données agro-hydrologiques construit et les méthodes d'extraction de skyline proposées.This thesis concerns the analysis of simulation data generated by the agrohydrological model TNT. Our objective is to develop analytical methods for massive simulation results. We want to place the user at the heart of the decision-making process, while letting him handle and analyze large amounts of data in a very efficient way. Our first contribution is an original approach N-Catch, relying on interactive multidimensional analysis methods for archiving simulation results in a decisional database (i.e. data warehouse) adapted to the spatio-temporal nature of the simulation data. In addition, we suggest to analyze the simulation data with online analytical methods (OLAP) to provide strategic information for stakeholders to improve the decision making process. Our second contribution concern two methods for computing skyline queries in the context of data warehouses. These methods enable stakeholders to formulate new questions by combining conflicting environmental criteria, to find compromise solutions associated with their expectations, and to exploit the stakeholder preferences to identify and highlight the data of potential interest. The first method EC2Sky, focuses on how to answer efficiently and progressively skyline queries in the presence of several dynamic user preferences despite of large volume of data. The second method HSky, extends the skyline computation to hierarchical dimensions. It allows the user to navigate along the dimensions hierarchies (i.e. specialize / generalize) while ensuring the online computation of associated skylines. Finally, we present the application of our proposals for managing agricultural practices to improve water quality in agricultural watersheds. We propose a coupling between the agro-hydrological data warehouse model N-Catch and the proposed skyline computation methods.RENNES1-Bibl. électronique (352382106) / SudocSudocFranceF

High Performance Multidimensional Analysis and Data Mining

Summary information from data in large databases is used to auswer queries in On-Line Analytical Processing (OLAP) systems and to build decision support systems over them. The Data Cube is used to calculate and store summary in-formation on a variety of dimensions, which is computed only partially if the number of dimensions is large. Queries posed on such systems are quite complex and require dif-ferent views of data. These may either be auswered from a materialized cube in the data cube or calculated on the fly. Further, data mining for associations can be performed on the data cube. Analytical models need to capture the multi-dimensionality of the underlying data, a task for which mul-tidimensional databases are well suited. Multidimensional databases tore data in multidimensional structure on which analytical operations are performed. A challenge for these systems is how to handle large data sets in a large number of dimensions. This paper presents q parallel OLAP h&structure for multidimensional databases integrated with association rule mining. Scheduling optimisations for parallel computation of wmplete data cubes are presented. We propose left aud right schedules for partial data cubes for m-way mining of association rules. Our implementation on the IBM SP-2, a shared-nothing parallel machine, cau handle large data sets and a large number of dimensions by using disk I/O in our algorithms.