3,994 research outputs found
Efficient Discovery of Association Rules and Frequent Itemsets through Sampling with Tight Performance Guarantees
The tasks of extracting (top-) Frequent Itemsets (FI's) and Association
Rules (AR's) are fundamental primitives in data mining and database
applications. Exact algorithms for these problems exist and are widely used,
but their running time is hindered by the need of scanning the entire dataset,
possibly multiple times. High quality approximations of FI's and AR's are
sufficient for most practical uses, and a number of recent works explored the
application of sampling for fast discovery of approximate solutions to the
problems. However, these works do not provide satisfactory performance
guarantees on the quality of the approximation, due to the difficulty of
bounding the probability of under- or over-sampling any one of an unknown
number of frequent itemsets. In this work we circumvent this issue by applying
the statistical concept of \emph{Vapnik-Chervonenkis (VC) dimension} to develop
a novel technique for providing tight bounds on the sample size that guarantees
approximation within user-specified parameters. Our technique applies both to
absolute and to relative approximations of (top-) FI's and AR's. The
resulting sample size is linearly dependent on the VC-dimension of a range
space associated with the dataset to be mined. The main theoretical
contribution of this work is a proof that the VC-dimension of this range space
is upper bounded by an easy-to-compute characteristic quantity of the dataset
which we call \emph{d-index}, and is the maximum integer such that the
dataset contains at least transactions of length at least such that no
one of them is a superset of or equal to another. We show that this bound is
strict for a large class of datasets.Comment: 19 pages, 7 figures. A shorter version of this paper appeared in the
proceedings of ECML PKDD 201
Pruning based Distance Sketches with Provable Guarantees on Random Graphs
Measuring the distances between vertices on graphs is one of the most
fundamental components in network analysis. Since finding shortest paths
requires traversing the graph, it is challenging to obtain distance information
on large graphs very quickly. In this work, we present a preprocessing
algorithm that is able to create landmark based distance sketches efficiently,
with strong theoretical guarantees. When evaluated on a diverse set of social
and information networks, our algorithm significantly improves over existing
approaches by reducing the number of landmarks stored, preprocessing time, or
stretch of the estimated distances.
On Erd\"{o}s-R\'{e}nyi graphs and random power law graphs with degree
distribution exponent , our algorithm outputs an exact distance
data structure with space between and
depending on the value of , where is the number of vertices. We
complement the algorithm with tight lower bounds for Erdos-Renyi graphs and the
case when is close to two.Comment: Full version for the conference paper to appear in The Web
Conference'1
An intelligent recommendation system framework for student relationship management
In order to enhance student satisfaction, many services have been provided in order to meet student needs. A recommendation system is a significant service which can be used to assist students in several ways. This paper proposes a conceptual framework of an Intelligent Recommendation System in order to support Student Relationship Management (SRM) for a Thai private university. This article proposed the system architecture of an Intelligent Recommendation System (IRS) which aims to assist students to choose an appropriate course for their studies. Moreover, this study intends to compare different data mining techniques in various recommendation systems and to determine appropriate algorithms for the proposed electronic Intelligent Recommendation System (IRS). The IRS also aims to support Student Relationship Management (SRM) in the university. The IRS has been designed using data mining and artificial intelligent techniques such as clustering, association rule and classification
SOTXTSTREAM: Density-based self-organizing clustering of text streams
A streaming data clustering algorithm is presented building upon the density-based selforganizing stream clustering algorithm SOSTREAM. Many density-based clustering algorithms are limited by their inability to identify clusters with heterogeneous density. SOSTREAM addresses this limitation through the use of local (nearest neighbor-based) density determinations. Additionally, many stream clustering algorithms use a two-phase clustering approach. In the first phase, a micro-clustering solution is maintained online, while in the second phase, the micro-clustering solution is clustered offline to produce a macro solution. By performing self-organization techniques on micro-clusters in the online phase, SOSTREAM is able to maintain a macro clustering solution in a single phase. Leveraging concepts from SOSTREAM, a new density-based self-organizing text stream clustering algorithm, SOTXTSTREAM, is presented that addresses several shortcomings of SOSTREAM. Gains in clustering performance of this new algorithm are demonstrated on several real-world text stream datasets
- …