Emergent intertransaction association rules for abnormality detection in intelligent environments

This paper is concerned with identifying anomalous behaviour of people in smart environments. We propose the use of emergent transaction mining and the use of the extended frequent pattern tree as a basis. Our experiments on two data sets demonstrate that emergent intertransaction associations are able to detect abnormality present in real world data and that both short and long term behavioural changes can be discovered. The use of intertransaction associations is shown to be advantageous in the detection of temporal associationanomalies otherwise not readily detectable by traditional "market basket" intratransaction mining

Curious Negotiator

n negotiation the exchange of information is as important as the exchange of offers. The curious negotiator is a multiagent system with three types of agents. Two negotiation agents, each representing an individual, develop consecutive offers, supported by information, whilst requesting information from its opponent. A mediator agent, with experience of prior negotiations, suggests how the negotiation may develop. A failed negotiation is a missed opportunity. An observer agent analyses failures looking for new opportunities. The integration of negotiation theory and data mining enables the curious negotiator to discover and exploit negotiation opportunities. Trials will be conducted in electronic business

Unexpected rules using a conceptual distance based on fuzzy ontology

AbstractOne of the major drawbacks of data mining methods is that they generate a notably large number of rules that are often obvious or useless or, occasionally, out of the user’s interest. To address such drawbacks, we propose in this paper an approach that detects a set of unexpected rules in a discovered association rule set. Generally speaking, the proposed approach investigates the discovered association rules using the user’s domain knowledge, which is represented by a fuzzy domain ontology. Next, we rank the discovered rules according to the conceptual distances of the rules

Relational Patterns

Information Systems Working Papers Serie

Investigation of discovering rules from data.

by Ng, King Kwok.Thesis submitted in: December 1999.Thesis (M.Phil.)--Chinese University of Hong Kong, 2000.Includes bibliographical references (leaves 99-104).Abstracts in English and Chinese.Acknowledgments --- p.iiAbstract --- p.iiiChapter 1 --- Introduction --- p.1Chapter 1.1 --- Data Mining and Rule Discovery --- p.1Chapter 1.1.1 --- Association Rule --- p.3Chapter 1.1.2 --- Sequential Pattern --- p.4Chapter 1.1.3 --- Dependence Rule --- p.6Chapter 1.2 --- Association Rule Mining --- p.8Chapter 1.3 --- Contributions --- p.9Chapter 1.4 --- Outline of the Thesis --- p.10Chapter 2 --- Related Work on Association Rule Mining --- p.11Chapter 2.1 --- Batch Algorithms --- p.11Chapter 2.1.1 --- The Apriori Algorithm --- p.11Chapter 2.1.2 --- The DIC Algorithm --- p.13Chapter 2.1.3 --- The Partition Algorithm --- p.15Chapter 2.1.4 --- The Sampling Algorithm --- p.15Chapter 2.2 --- Incremental Association Rule Mining --- p.16Chapter 2.2.1 --- The FUP Algorithm --- p.17Chapter 2.2.2 --- The FUP2 Algorithm --- p.18Chapter 2.2.3 --- The FUP* Algorithm --- p.19Chapter 2.2.4 --- The Negative Border Method --- p.20Chapter 2.2.5 --- Limitations of Existing Incremental Association Rule Mining Algorithms --- p.21Chapter 3 --- A New Incremental Association Rule Mining Approach --- p.23Chapter 3.1 --- Outline for the Proposed Approach --- p.23Chapter 3.2 --- Our New Approach --- p.26Chapter 3.2.1 --- The IDIC_M Algorithm --- p.26Chapter 3.2.2 --- A Variant Algorithm: The IDIC_S Algorithm --- p.29Chapter 3.3 --- Performance Evaluation of Our Approach --- p.30Chapter 3.3.1 --- Experimental Results for Algorithm IDIC_M --- p.30Chapter 3.3.2 --- Experimental Results for Algorithm IDIC_S --- p.35Chapter 3.4 --- Discussion --- p.39Chapter 4 --- Related Work on Multiple_Level AR and Belief-Driven Mining --- p.41Chapter 4.1 --- Background on Multiple_Level Association Rules --- p.41Chapter 4.2 --- Related Work on Multiple-Level Association Rules --- p.42Chapter 4.2.1 --- The Basic Algorithm --- p.42Chapter 4.2.2 --- The Cumulate Algorithm --- p.44Chapter 4.2.3 --- The EstMerge Algorithm --- p.44Chapter 4.2.4 --- Using Hierarchy-Information Encoded Transaction Table --- p.45Chapter 4.3 --- Background on Rule Mining in the Presence of User Belief --- p.46Chapter 4.4 --- Related Work on Rule Mining in the Presence of User Belief --- p.47Chapter 4.4.1 --- Post-Analysis of Learned Rules --- p.47Chapter 4.4.2 --- Using General Impressions to Analyze Discovered Classification Rules --- p.49Chapter 4.4.3 --- A Belief-Driven Method for Discovering Unexpected Patterns --- p.50Chapter 4.4.4 --- Constraint-Based Rule Mining --- p.51Chapter 4.5 --- Limitations of Existing Approaches --- p.52Chapter 5 --- Multiple-Level Association Rules Mining in the Presence of User Belief --- p.54Chapter 5.1 --- User Belief Under Taxonomy --- p.55Chapter 5.2 --- Formal Definition of Rule Interestingness --- p.57Chapter 5.3 --- The MARUB_E Mining Algorithm --- p.61Chapter 6 --- Experiments on MARUB_E --- p.64Chapter 6.1 --- Preliminary Experiments --- p.64Chapter 6.2 --- Experiments on Synthetic Data --- p.68Chapter 6.3 --- Experiments on Real Data --- p.71Chapter 7 --- Dealing with Vague Belief of User --- p.76Chapter 7.1 --- User Belief Under Taxonomy --- p.76Chapter 7.2 --- Relationship with Constraint-Based Rule Mining --- p.79Chapter 7.3 --- Formal Definition of Rule Interestingness --- p.79Chapter 7.4 --- The MARUB_V Mining Algorithm --- p.81Chapter 8 --- Experiments on MARUB_V --- p.84Chapter 8.1 --- Preliminary Experiments --- p.84Chapter 8.1.1 --- Experiments on Synthetic Data --- p.87Chapter 8.1.2 --- Experiments on Real Data --- p.93Chapter 9 --- Conclusions and Future Work --- p.96Chapter 9.1 --- Conclusions --- p.95Chapter 9.2 --- Future Work --- p.9

TEXTUAL DATA MINING FOR NEXT GENERATION INTELLIGENT DECISION MAKING IN INDUSTRIAL ENVIRONMENT: A SURVEY

This paper proposes textual data mining as a next generation intelligent decision making technology for sustainable knowledge management solutions in any industrial environment. A detailed survey of applications of Data Mining techniques for exploiting information from different data formats and transforming this information into knowledge is presented in the literature survey. The focus of the survey is to show the power of different data mining techniques for exploiting information from data. The literature surveyed in this paper shows that intelligent decision making is of great importance in many contexts within manufacturing, construction and business generally. Business intelligence tools, which can be interpreted as decision support tools, are of increasing importance to companies for their success within competitive global markets. However, these tools are dependent on the relevancy, accuracy and overall quality of the knowledge on which they are based and which they use. Thus the research work presented in the paper uncover the importance and power of different data mining techniques supported by text mining methods used to exploit information from semi-structured or un-structured data formats. A great source of information is available in these formats and when exploited by combined efforts of data and text mining tools help the decision maker to take effective decision for the enhancement of business of industry and discovery of useful knowledge is made for next generation of intelligent decision making. Thus the survey shows the power of textual data mining as the next generation technology for intelligent decision making in the industrial environment

The EDAM Project: Mining Atmospheric Aerosol Datasets

Data mining has been a very active area of research in the database, machine learning, and mathematical programming communities in recent years. EDAM (Exploratory Data Analysis and Management) is a joint project between researchers in Atmospheric Chemistry and Computer Science at Carleton College and the University of Wisconsin-Madison that aims to develop data mining techniques for advancing the state of the art in analyzing atmospheric aerosol datasets. There is a great need to better understand the sources, dynamics, and compositions of atmospheric aerosols. The traditional approach for particle measurement, which is the collection of bulk samples of particulates on filters, is not adequate for studying particle dynamics and real-time correlations. This has led to the development of a new generation of real-time instruments that provide continuous or semi-continuous streams of data about certain aerosol properties. However, these instruments have added a significant level of complexity to atmospheric aerosol data, and dramatically increased the amounts of data to be collected, managed, and analyzed. Our abilit y to integrate the data from all of these new and complex instruments now lags far behind our data-collection capabilities, and severely limits our ability to understand the data and act upon it in a timely manner. In this paper, we present an overview of the EDAM project. The goal of the project, which is in its early stages, is to develop novel data mining algorithms and approaches to managing and monitoring multiple complex data streams. An important objective is data quality assurance, and real-time data mining offers great potential. The approach that we take should also provide good techniques to deal with gas-phase and semi-volatile data. While atmospheric aerosol analysis is an important and challenging domain that motivates us with real problems and serves as a concrete test of our results, our objective is to develop techniques that have broader applicability, and to explore some fundamental challenges in data mining that are not specific to any given application domain

On Unexpectedness in Recommender Systems: Or How to Better Expect the Unexpected

Although the broad social and business success of recommender systems has been achieved across several domains, there is still a long way to go in terms of user satisfaction. One of the key dimensions for significant improvement is the concept of unexpectedness. In this paper, we propose a method to improve user satisfaction by generating unexpected recommendations based on the utility theory of economics. In particular, we propose a new concept of unexpectedness as recommending to users those items that depart from what they expect from the system. We define and formalize the concept of unexpectedness and discuss how it differs from the related notions of novelty, serendipity, and diversity. Besides, we suggest several mechanisms for specifying the users’ expectations and propose specific performance metrics to measure the unexpectedness of recommendation lists.We also take into consideration the quality of recommendations using certain utility functions and present an algorithm for providing the users with unexpected recommendations of high quality that are hard to discover but fairly match their interests. Finally, we conduct several experiments on “real-world” data sets to compare our recommendation results with some other standard baseline methods. The proposed approach outperforms these baseline methods in terms of unexpectedness and other important metrics, such as coverage and aggregate diversity, while avoiding any accuracy loss