2 research outputs found
Clustering multivariate climate data streams using fractal dimension
A data stream is a flow of data produced continuously along the time. Storing and analyzing such information become challenging due to exponential growth of the data volume collected. In this context, some methods were proposed to cluster data streams with similar behavior along the time. However, those methods have failed on clustering data flows with more than one attribute, i.e., multivariate flows. This paper introduces a new method to cluster multivariate data streams, based on fractal dimension, reading the data only once. We evaluated our method over real multivariate data streams generated by climate sensors. Not only was our method able to cluster the flows of data, but also identified sensors with similar behavior during the analyzed period.CAPESCNPQFAPES
Recommended from our members
Real-time pre-processing technique for drift detection, feature tracking, and feature selection using adaptive micro-clusters for data stream classification
Data streams are unbounded, sequential data instances that are generated with high Velocity.
Data streams arrive online (i.e., instance by instance) and there is no control over the order
in which data instances arrive either within a data stream or across data streams. Classifying
sequential data instances is a challenging problem in machine learning with applications in
network intrusion detection, financial markets and sensor networks. The automatic labelling
of unseen instances from the stream in real-time is the main challenge that data stream classification
faces. For this, the classifier needs to adapt to concept drifts and can only have a
single-pass through the data with a limited amount of memory if the stream is generating data
instances at a high Velocity. Nowadays the focus of Data Stream Mining (DSM) lies in the
development of data mining algorithms rather than on pre-processing techniques. To the best
of the author knowledge, at present, there are no developments for truly real-time feature selection
in a streaming setting. This research work presents a real-time pre-processing technique,
in particular, feature tracking in combination with concept drift detection. The feature tracking
is designed to improve DSM classification algorithms by enabling real-time feature selection.
The pre-processing technique is based on tracking adaptive statistical summaries of the data
and class label distributions, known as Micro-Clusters. Thus the three objectives of this research
were to develop a real-time pre-processing technique that can (1) detect a concept drift,
(2) identify features that were involved in concept drift and thus potentially change their relevance
and (3) build a real-time feature selection method based on the developments mentioned
above. The evaluation of the developed technique is based on artificial data streams with known
ground truth and real datasets with and without artificially induced concept drift (i.e., controlled
and uncontrolled real datasets). It was observed that the developed method for concept drift
detection did detect induced concept drifts very well compared with alternative concept drift
detection methods. Overall the research represents a first attempt to resolve real-time feature
selection for DSM tasks. It has been shown that the technique can indeed identify concept drift,
track features, and identify features that may have changed their relevance for the DSM task in
real-time. It has also been shown that the developed method for real-time feature selection can
improve the accuracy of data stream classification tasks