840 research outputs found
Positive Semidefinite Metric Learning Using Boosting-like Algorithms
The success of many machine learning and pattern recognition methods relies
heavily upon the identification of an appropriate distance metric on the input
data. It is often beneficial to learn such a metric from the input training
data, instead of using a default one such as the Euclidean distance. In this
work, we propose a boosting-based technique, termed BoostMetric, for learning a
quadratic Mahalanobis distance metric. Learning a valid Mahalanobis distance
metric requires enforcing the constraint that the matrix parameter to the
metric remains positive definite. Semidefinite programming is often used to
enforce this constraint, but does not scale well and easy to implement.
BoostMetric is instead based on the observation that any positive semidefinite
matrix can be decomposed into a linear combination of trace-one rank-one
matrices. BoostMetric thus uses rank-one positive semidefinite matrices as weak
learners within an efficient and scalable boosting-based learning process. The
resulting methods are easy to implement, efficient, and can accommodate various
types of constraints. We extend traditional boosting algorithms in that its
weak learner is a positive semidefinite matrix with trace and rank being one
rather than a classifier or regressor. Experiments on various datasets
demonstrate that the proposed algorithms compare favorably to those
state-of-the-art methods in terms of classification accuracy and running time.Comment: 30 pages, appearing in Journal of Machine Learning Researc
2D Face Recognition System Based on Selected Gabor Filters and Linear Discriminant Analysis LDA
We present a new approach for face recognition system. The method is based on
2D face image features using subset of non-correlated and Orthogonal Gabor
Filters instead of using the whole Gabor Filter Bank, then compressing the
output feature vector using Linear Discriminant Analysis (LDA). The face image
has been enhanced using multi stage image processing technique to normalize it
and compensate for illumination variation. Experimental results show that the
proposed system is effective for both dimension reduction and good recognition
performance when compared to the complete Gabor filter bank. The system has
been tested using CASIA, ORL and Cropped YaleB 2D face images Databases and
achieved average recognition rate of 98.9 %
Accurately Estimating Rigid Transformations in Registration using a Boosting-Inspired Mechanism
Feature extraction and matching provide the basis of many methods for object registration, modeling, retrieval, and recognition. However, this approach typically introduces false matches, due to lack of features, noise, occlusion, and cluttered backgrounds. In registration, these false matches lead to inaccurate estimation of the underlying transformation that brings the overlapping shapes into best possible alignment. In this paper, we propose a novel boosting-inspired method to tackle this challenging task. It includes three key steps: (i) underlying transformation estimation in the weighted least squares sense, (ii) boosting parameter estimation and regularization via Tsallis entropy, and (iii) weight re-estimation and regularization via Shannon entropy and update with a maximum fusion rule. The process is iterated. The final optimal underlying transformation is estimated as a weighted average of the transformations estimated from the latest iterations, with weights given by the boosting parameters. A comparative study based on real shape data shows that the proposed method outperforms four other state-of-the-art methods for evaluating the established point matches, enabling more accurate and stable estimation of the underlying transformation
Adaptive Boosting for Domain Adaptation: Towards Robust Predictions in Scene Segmentation
Domain adaptation is to transfer the shared knowledge learned from the source
domain to a new environment, i.e., target domain. One common practice is to
train the model on both labeled source-domain data and unlabeled target-domain
data. Yet the learned models are usually biased due to the strong supervision
of the source domain. Most researchers adopt the early-stopping strategy to
prevent over-fitting, but when to stop training remains a challenging problem
since the lack of the target-domain validation set. In this paper, we propose
one efficient bootstrapping method, called Adaboost Student, explicitly
learning complementary models during training and liberating users from
empirical early stopping. Adaboost Student combines the deep model learning
with the conventional training strategy, i.e., adaptive boosting, and enables
interactions between learned models and the data sampler. We adopt one adaptive
data sampler to progressively facilitate learning on hard samples and aggregate
"weak" models to prevent over-fitting. Extensive experiments show that (1)
Without the need to worry about the stopping time, AdaBoost Student provides
one robust solution by efficient complementary model learning during training.
(2) AdaBoost Student is orthogonal to most domain adaptation methods, which can
be combined with existing approaches to further improve the state-of-the-art
performance. We have achieved competitive results on three widely-used scene
segmentation domain adaptation benchmarks.Comment: 10 pages, 7 tables, 5 figure
Deep Neural Networks based Meta-Learning for Network Intrusion Detection
The digitization of different components of industry and inter-connectivity
among indigenous networks have increased the risk of network attacks. Designing
an intrusion detection system to ensure security of the industrial ecosystem is
difficult as network traffic encompasses various attack types, including new
and evolving ones with minor changes. The data used to construct a predictive
model for computer networks has a skewed class distribution and limited
representation of attack types, which differ from real network traffic. These
limitations result in dataset shift, negatively impacting the machine learning
models' predictive abilities and reducing the detection rate against novel
attacks. To address the challenges, we propose a novel deep neural network
based Meta-Learning framework; INformation FUsion and Stacking Ensemble
(INFUSE) for network intrusion detection. First, a hybrid feature space is
created by integrating decision and feature spaces. Five different classifiers
are utilized to generate a pool of decision spaces. The feature space is then
enriched through a deep sparse autoencoder that learns the semantic
relationships between attacks. Finally, the deep Meta-Learner acts as an
ensemble combiner to analyze the hybrid feature space and make a final
decision. Our evaluation on stringent benchmark datasets and comparison to
existing techniques showed the effectiveness of INFUSE with an F-Score of 0.91,
Accuracy of 91.6%, and Recall of 0.94 on the Test+ dataset, and an F-Score of
0.91, Accuracy of 85.6%, and Recall of 0.87 on the stringent Test-21 dataset.
These promising results indicate the strong generalization capability and the
potential to detect network attacks.Comment: Pages: 15, Figures: 10 and Tables:
- …