19,680 research outputs found
Feature Selection and Feature Extraction in Pattern Analysis: A Literature Review
Pattern analysis often requires a pre-processing stage for extracting or
selecting features in order to help the classification, prediction, or
clustering stage discriminate or represent the data in a better way. The reason
for this requirement is that the raw data are complex and difficult to process
without extracting or selecting appropriate features beforehand. This paper
reviews theory and motivation of different common methods of feature selection
and extraction and introduces some of their applications. Some numerical
implementations are also shown for these methods. Finally, the methods in
feature selection and extraction are compared.Comment: 14 pages, 1 figure, 2 tables, survey (literature review) pape
Neural Component Analysis for Fault Detection
Principal component analysis (PCA) is largely adopted for chemical process
monitoring and numerous PCA-based systems have been developed to solve various
fault detection and diagnosis problems. Since PCA-based methods assume that the
monitored process is linear, nonlinear PCA models, such as autoencoder models
and kernel principal component analysis (KPCA), has been proposed and applied
to nonlinear process monitoring. However, KPCA-based methods need to perform
eigen-decomposition (ED) on the kernel Gram matrix whose dimensions depend on
the number of training data. Moreover, prefixed kernel parameters cannot be
most effective for different faults which may need different parameters to
maximize their respective detection performances. Autoencoder models lack the
consideration of orthogonal constraints which is crucial for PCA-based
algorithms. To address these problems, this paper proposes a novel nonlinear
method, called neural component analysis (NCA), which intends to train a
feedforward neural work with orthogonal constraints such as those used in PCA.
NCA can adaptively learn its parameters through backpropagation and the
dimensionality of the nonlinear features has no relationship with the number of
training samples. Extensive experimental results on the Tennessee Eastman (TE)
benchmark process show the superiority of NCA in terms of missed detection rate
(MDR) and false alarm rate (FAR). The source code of NCA can be found in
https://github.com/haitaozhao/Neural-Component-Analysis.git.Comment: 10 pages,11 figure
Dimensionality Reduction via Regression in Hyperspectral Imagery
This paper introduces a new unsupervised method for dimensionality reduction
via regression (DRR). The algorithm belongs to the family of invertible
transforms that generalize Principal Component Analysis (PCA) by using
curvilinear instead of linear features. DRR identifies the nonlinear features
through multivariate regression to ensure the reduction in redundancy between
he PCA coefficients, the reduction of the variance of the scores, and the
reduction in the reconstruction error. More importantly, unlike other nonlinear
dimensionality reduction methods, the invertibility, volume-preservation, and
straightforward out-of-sample extension, makes DRR interpretable and easy to
apply. The properties of DRR enable learning a more broader class of data
manifolds than the recently proposed Non-linear Principal Components Analysis
(NLPCA) and Principal Polynomial Analysis (PPA). We illustrate the performance
of the representation in reducing the dimensionality of remote sensing data. In
particular, we tackle two common problems: processing very high dimensional
spectral information such as in hyperspectral image sounding data, and dealing
with spatial-spectral image patches of multispectral images. Both settings pose
collinearity and ill-determination problems. Evaluation of the expressive power
of the features is assessed in terms of truncation error, estimating
atmospheric variables, and surface land cover classification error. Results
show that DRR outperforms linear PCA and recently proposed invertible
extensions based on neural networks (NLPCA) and univariate regressions (PPA).Comment: 12 pages, 6 figures, 62 reference
A Face Recognition approach based on entropy estimate of the nonlinear DCT features in the Logarithm Domain together with Kernel Entropy Component Analysis
This paper exploits the feature extraction capabilities of the discrete
cosine transform (DCT) together with an illumination normalization approach in
the logarithm domain that increase its robustness to variations in facial
geometry and illumination. Secondly in the same domain the entropy measures are
applied on the DCT coefficients so that maximum entropy preserving pixels can
be extracted as the feature vector. Thus the informative features of a face can
be extracted in a low dimensional space. Finally, the kernel entropy component
analysis (KECA) with an extension of arc cosine kernels is applied on the
extracted DCT coefficients that contribute most to the entropy estimate to
obtain only those real kernel ECA eigenvectors that are associated with
eigenvalues having high positive entropy contribution. The resulting system was
successfully tested on real image sequences and is robust to significant
partial occlusion and illumination changes, validated with the experiments on
the FERET, AR, FRAV2D and ORL face databases. Experimental comparison is
demonstrated to prove the superiority of the proposed approach in respect to
recognition accuracy. Using specificity and sensitivity we find that the best
is achieved when Renyi entropy is applied on the DCT coefficients. Extensive
experimental comparison is demonstrated to prove the superiority of the
proposed approach in respect to recognition accuracy. Moreover, the proposed
approach is very simple, computationally fast and can be implemented in any
real-time face recognition system.Comment: 9 pages,Published Online August 2013 in MECS. International Journal
of Information Technology and Computer Science, 2013. arXiv admin note: text
overlap with arXiv:1112.3712 by other author
Evaluating software-based fingerprint liveness detection using Convolutional Networks and Local Binary Patterns
With the growing use of biometric authentication systems in the past years,
spoof fingerprint detection has become increasingly important. In this work, we
implement and evaluate two different feature extraction techniques for
software-based fingerprint liveness detection: Convolutional Networks with
random weights and Local Binary Patterns. Both techniques were used in
conjunction with a Support Vector Machine (SVM) classifier. Dataset
Augmentation was used to increase classifier's performance and a variety of
preprocessing operations were tested, such as frequency filtering, contrast
equalization, and region of interest filtering. The experiments were made on
the datasets used in The Liveness Detection Competition of years 2009, 2011 and
2013, which comprise almost 50,000 real and fake fingerprints' images. Our best
method achieves an overall rate of 95.2% of correctly classified samples - an
improvement of 35% in test error when compared with the best previously
published results.Comment: arXiv admin note: text overlap with arXiv:1301.3557 by other author
Unsupervised Deep Feature Extraction for Remote Sensing Image Classification
This paper introduces the use of single layer and deep convolutional networks
for remote sensing data analysis. Direct application to multi- and
hyper-spectral imagery of supervised (shallow or deep) convolutional networks
is very challenging given the high input data dimensionality and the relatively
small amount of available labeled data. Therefore, we propose the use of greedy
layer-wise unsupervised pre-training coupled with a highly efficient algorithm
for unsupervised learning of sparse features. The algorithm is rooted on sparse
representations and enforces both population and lifetime sparsity of the
extracted features, simultaneously. We successfully illustrate the expressive
power of the extracted representations in several scenarios: classification of
aerial scenes, as well as land-use classification in very high resolution
(VHR), or land-cover classification from multi- and hyper-spectral images. The
proposed algorithm clearly outperforms standard Principal Component Analysis
(PCA) and its kernel counterpart (kPCA), as well as current state-of-the-art
algorithms of aerial classification, while being extremely computationally
efficient at learning representations of data. Results show that single layer
convolutional networks can extract powerful discriminative features only when
the receptive field accounts for neighboring pixels, and are preferred when the
classification requires high resolution and detailed results. However, deep
architectures significantly outperform single layers variants, capturing
increasing levels of abstraction and complexity throughout the feature
hierarchy
Applying Deep Belief Networks to Word Sense Disambiguation
In this paper, we applied a novel learning algorithm, namely, Deep Belief
Networks (DBN) to word sense disambiguation (WSD). DBN is a probabilistic
generative model composed of multiple layers of hidden units. DBN uses
Restricted Boltzmann Machine (RBM) to greedily train layer by layer as a
pretraining. Then, a separate fine tuning step is employed to improve the
discriminative power. We compared DBN with various state-of-the-art supervised
learning algorithms in WSD such as Support Vector Machine (SVM), Maximum
Entropy model (MaxEnt), Naive Bayes classifier (NB) and Kernel Principal
Component Analysis (KPCA). We used all words in the given paragraph,
surrounding context words and part-of-speech of surrounding words as our
knowledge sources. We conducted our experiment on the SENSEVAL-2 data set. We
observed that DBN outperformed all other learning algorithms
Multi-View Kernels for Low-Dimensional Modeling of Seismic Events
The problem of learning from seismic recordings has been studied for years.
There is a growing interest in developing automatic mechanisms for identifying
the properties of a seismic event. One main motivation is the ability have a
reliable identification of man-made explosions. The availability of multiple
high-dimensional observations has increased the use of machine learning
techniques in a variety of fields. In this work, we propose to use a
kernel-fusion based dimensionality reduction framework for generating
meaningful seismic representations from raw data. The proposed method is tested
on 2023 events that were recorded in Israel and in Jordan. The method achieves
promising results in classification of event type as well as in estimating the
location of the event. The proposed fusion and dimensionality reduction tools
may be applied to other types of geophysical data
A Simple and Fast Algorithm for L1-norm Kernel PCA
We present an algorithm for L1-norm kernel PCA and provide a convergence
analysis for it. While an optimal solution of L2-norm kernel PCA can be
obtained through matrix decomposition, finding that of L1-norm kernel PCA is
not trivial due to its non-convexity and non-smoothness. We provide a novel
reformulation through which an equivalent, geometrically interpretable problem
is obtained. Based on the geometric interpretation of the reformulated problem,
we present a fixed-point type algorithm that iteratively computes a binary
weight for each observation. As the algorithm requires only inner products of
data vectors, it is computationally efficient and the kernel trick is
applicable. In the convergence analysis, we show that the algorithm converges
to a local optimal solution in a finite number of steps. Moreover, we provide a
rate of convergence analysis, which has been never done for any L1-norm PCA
algorithm, proving that the sequence of objective values converges at a linear
rate. In numerical experiments, we show that the algorithm is robust in the
presence of entry-wise perturbations and computationally scalable, especially
in a large-scale setting. Lastly, we introduce an application to outlier
detection where the model based on the proposed algorithm outperforms the
benchmark algorithms.Comment: 14 pages, 7 figure
Face Recognition: A Novel Multi-Level Taxonomy based Survey
In a world where security issues have been gaining growing importance, face
recognition systems have attracted increasing attention in multiple application
areas, ranging from forensics and surveillance to commerce and entertainment.
To help understanding the landscape and abstraction levels relevant for face
recognition systems, face recognition taxonomies allow a deeper dissection and
comparison of the existing solutions. This paper proposes a new, more
encompassing and richer multi-level face recognition taxonomy, facilitating the
organization and categorization of available and emerging face recognition
solutions; this taxonomy may also guide researchers in the development of more
efficient face recognition solutions. The proposed multi-level taxonomy
considers levels related to the face structure, feature support and feature
extraction approach. Following the proposed taxonomy, a comprehensive survey of
representative face recognition solutions is presented. The paper concludes
with a discussion on current algorithmic and application related challenges
which may define future research directions for face recognition.Comment: This paper is a preprint of a paper submitted to IET Biometrics. If
accepted, the copy of record will be available at the IET Digital Librar
- …