4,067 research outputs found
Finding and tracking multi-density clusters in an online dynamic data stream
The file attached to this record is the author's final peer reviewed version.Change is one of the biggest challenges in dynamic stream mining. From a data-mining perspective, adapting and tracking change is desirable in order to understand how and why change has occurred. Clustering, a form of unsupervised learning, can be used to identify the underlying patterns in a stream. Density-based clustering identifies clusters as areas of high density separated by areas of low density. This paper proposes a Multi-Density Stream Clustering (MDSC) algorithm to address these two problems; the multi-density problem and the problem of discovering and tracking changes in a dynamic stream. MDSC consists of two on-line components; discovered, labelled clusters and an outlier buffer. Incoming points are assigned to a live cluster or passed to the outlier buffer. New clusters are discovered in the buffer using an ant-inspired swarm intelligence approach. The newly discovered cluster is uniquely labelled and added to the set of live clusters. Processed data is subject to an ageing function and will disappear when it is no longer relevant. MDSC is shown to perform favourably to state-of-the-art peer stream-clustering algorithms on a range of real and synthetic data-streams. Experimental results suggest that MDSC can discover qualitatively useful patterns while being scalable and robust to noise
Recommended from our members
A Clustering System for Dynamic Data Streams Based on Metaheuristic Optimisation
open access articleThis article presents the Optimised Stream clustering algorithm (OpStream), a novel approach to cluster dynamic data streams. The proposed system displays desirable features, such as a low number of parameters and good scalability capabilities to both high-dimensional data and numbers of clusters in the dataset, and it is based on a hybrid structure using deterministic clustering methods and stochastic optimisation approaches to optimally centre the clusters. Similar to other state-of-the-art methods available in the literature, it uses “microclusters” and other established techniques, such as density based clustering. Unlike other methods, it makes use of metaheuristic optimisation to maximise performances during the initialisation phase, which precedes the classic online phase. Experimental results show that OpStream outperforms the state-of-the-art methods in several cases, and it is always competitive against other comparison algorithms regardless of the chosen optimisation method. Three variants of OpStream, each coming with a different optimisation algorithm, are presented in this study. A thorough sensitive analysis is performed by using the best variant to point out OpStream’s robustness to noise and resiliency to parameter changes
Adaptive firefly algorithm for hierarchical text clustering
Text clustering is essentially used by search engines to increase the recall and precision in information retrieval. As search engine operates on Internet content that is constantly being updated, there is a need for a clustering algorithm that offers automatic grouping of items without prior knowledge on the collection. Existing clustering methods have problems in determining optimal number of clusters and producing compact clusters. In this research, an adaptive hierarchical text clustering
algorithm is proposed based on Firefly Algorithm. The proposed Adaptive Firefly Algorithm (AFA) consists of three components: document clustering, cluster refining, and cluster merging. The first component introduces Weight-based Firefly Algorithm (WFA) that automatically identifies initial centers and their clusters for any given text collection. In order to refine the obtained clusters, a second algorithm, termed as Weight-based Firefly Algorithm with Relocate (WFAR), is proposed. Such an approach allows the relocation of a pre-assigned document into a newly created cluster. The third component, Weight-based Firefly Algorithm with Relocate and
Merging (WFARM), aims to reduce the number of produced clusters by merging nonpure clusters into the pure ones. Experiments were conducted to compare the proposed algorithms against seven existing methods. The percentage of success in
obtaining optimal number of clusters by AFA is 100% with purity and f-measure of 83% higher than the benchmarked methods. As for entropy measure, the AFA produced the lowest value (0.78) when compared to existing methods. The result indicates that Adaptive Firefly Algorithm can produce compact clusters. This research contributes to the text mining domain as hierarchical text clustering
facilitates the indexing of documents and information retrieval processes
Literature Review on Big Data Analytics Methods
Companies and industries are faced with a huge amount of raw data, which have information and knowledge in their hidden layer. Also, the format, size, variety, and velocity of generated data bring complexity for industries to apply them in an efficient and effective way. So, complexity in data analysis and interpretation incline organizations to deploy advanced tools and techniques to overcome the difficulties of managing raw data. Big data analytics is the advanced method that has the capability for managing data. It deploys machine learning techniques and deep learning methods to benefit from gathered data. In this research, the methods of both ML and DL have been discussed, and an ML/DL deployment model for IOT data has been proposed
Detecting synchronization clusters in multivariate time series via coarse-graining of Markov chains
Synchronization cluster analysis is an approach to the detection of
underlying structures in data sets of multivariate time series, starting from a
matrix R of bivariate synchronization indices. A previous method utilized the
eigenvectors of R for cluster identification, analogous to several recent
attempts at group identification using eigenvectors of the correlation matrix.
All of these approaches assumed a one-to-one correspondence of dominant
eigenvectors and clusters, which has however been shown to be wrong in
important cases. We clarify the usefulness of eigenvalue decomposition for
synchronization cluster analysis by translating the problem into the language
of stochastic processes, and derive an enhanced clustering method harnessing
recent insights from the coarse-graining of finite-state Markov processes. We
illustrate the operation of our method using a simulated system of coupled
Lorenz oscillators, and we demonstrate its superior performance over the
previous approach. Finally we investigate the question of robustness of the
algorithm against small sample size, which is important with regard to field
applications.Comment: Follow-up to arXiv:0706.3375. Journal submission 9 Jul 2007.
Published 19 Dec 200
Likelihood-based inference of B-cell clonal families
The human immune system depends on a highly diverse collection of
antibody-making B cells. B cell receptor sequence diversity is generated by a
random recombination process called "rearrangement" forming progenitor B cells,
then a Darwinian process of lineage diversification and selection called
"affinity maturation." The resulting receptors can be sequenced in high
throughput for research and diagnostics. Such a collection of sequences
contains a mixture of various lineages, each of which may be quite numerous, or
may consist of only a single member. As a step to understanding the process and
result of this diversification, one may wish to reconstruct lineage membership,
i.e. to cluster sampled sequences according to which came from the same
rearrangement events. We call this clustering problem "clonal family
inference." In this paper we describe and validate a likelihood-based framework
for clonal family inference based on a multi-hidden Markov Model (multi-HMM)
framework for B cell receptor sequences. We describe an agglomerative algorithm
to find a maximum likelihood clustering, two approximate algorithms with
various trade-offs of speed versus accuracy, and a third, fast algorithm for
finding specific lineages. We show that under simulation these algorithms
greatly improve upon existing clonal family inference methods, and that they
also give significantly different clusters than previous methods when applied
to two real data sets
- …