6 research outputs found
The Design of Pre-Processing Multidimensional Data Based on Component Analysis
Increased implementation of new databases related to multidimensional data involving techniques to support
efficient query process, create opportunities for more extensive research. Pre-processing is required because of
lack of data attribute values, noisy data, errors, inconsistencies or outliers and differences in coding. Several types of pre-processing based on component analysis will be carried out for cleaning, data integration and transformation,
as well as to reduce the dimensions. Component analysis can be done by statistical methods, with the aim to separate the various sources of data into a statistical pattern independent. This paper aims to improve the quality of pre-processed data based on component analysis. RapidMiner is used for data pre-processing using FastICA algorithm. Kernel K-mean is used to cluster the pre-processed data and Expectation Maximization (EM) is used to model. The model was tested using wisconsin breast cancer datasets, lung cancer datasets and prostate cancer datasets. The result shows that the performance of the cluster vector value is higher and the processing time is shorter
FINE: Fisher Information Non-parametric Embedding
We consider the problems of clustering, classification, and visualization of
high-dimensional data when no straightforward Euclidean representation exists.
Typically, these tasks are performed by first reducing the high-dimensional
data to some lower dimensional Euclidean space, as many manifold learning
methods have been developed for this task. In many practical problems however,
the assumption of a Euclidean manifold cannot be justified. In these cases, a
more appropriate assumption would be that the data lies on a statistical
manifold, or a manifold of probability density functions (PDFs). In this paper
we propose using the properties of information geometry in order to define
similarities between data sets using the Fisher information metric. We will
show this metric can be approximated using entirely non-parametric methods, as
the parameterization of the manifold is generally unknown. Furthermore, by
using multi-dimensional scaling methods, we are able to embed the corresponding
PDFs into a low-dimensional Euclidean space. This not only allows for
classification of the data, but also visualization of the manifold. As a whole,
we refer to our framework as Fisher Information Non-parametric Embedding
(FINE), and illustrate its uses on a variety of practical problems, including
bio-medical applications and document classification.Comment: 30 pages, 21 figure