Manifold Elastic Net: A Unified Framework for Sparse Dimension Reduction

It is difficult to find the optimal sparse solution of a manifold learning based dimensionality reduction algorithm. The lasso or the elastic net penalized manifold learning based dimensionality reduction is not directly a lasso penalized least square problem and thus the least angle regression (LARS) (Efron et al. \cite{LARS}), one of the most popular algorithms in sparse learning, cannot be applied. Therefore, most current approaches take indirect ways or have strict settings, which can be inconvenient for applications. In this paper, we proposed the manifold elastic net or MEN for short. MEN incorporates the merits of both the manifold learning based dimensionality reduction and the sparse learning based dimensionality reduction. By using a series of equivalent transformations, we show MEN is equivalent to the lasso penalized least square problem and thus LARS is adopted to obtain the optimal sparse solution of MEN. In particular, MEN has the following advantages for subsequent classification: 1) the local geometry of samples is well preserved for low dimensional data representation, 2) both the margin maximization and the classification error minimization are considered for sparse projection calculation, 3) the projection matrix of MEN improves the parsimony in computation, 4) the elastic net penalty reduces the over-fitting problem, and 5) the projection matrix of MEN can be interpreted psychologically and physiologically. Experimental evidence on face recognition over various popular datasets suggests that MEN is superior to top level dimensionality reduction algorithms.Comment: 33 pages, 12 figure

Deep Adaptive Feature Embedding with Local Sample Distributions for Person Re-identification

Person re-identification (re-id) aims to match pedestrians observed by disjoint camera views. It attracts increasing attention in computer vision due to its importance to surveillance system. To combat the major challenge of cross-view visual variations, deep embedding approaches are proposed by learning a compact feature space from images such that the Euclidean distances correspond to their cross-view similarity metric. However, the global Euclidean distance cannot faithfully characterize the ideal similarity in a complex visual feature space because features of pedestrian images exhibit unknown distributions due to large variations in poses, illumination and occlusion. Moreover, intra-personal training samples within a local range are robust to guide deep embedding against uncontrolled variations, which however, cannot be captured by a global Euclidean distance. In this paper, we study the problem of person re-id by proposing a novel sampling to mine suitable \textit{positives} (i.e. intra-class) within a local range to improve the deep embedding in the context of large intra-class variations. Our method is capable of learning a deep similarity metric adaptive to local sample structure by minimizing each sample's local distances while propagating through the relationship between samples to attain the whole intra-class minimization. To this end, a novel objective function is proposed to jointly optimize similarity metric learning, local positive mining and robust deep embedding. This yields local discriminations by selecting local-ranged positive samples, and the learned features are robust to dramatic intra-class variations. Experiments on benchmarks show state-of-the-art results achieved by our method.Comment: Published on Pattern Recognitio

Study on multi-SVM systems and their applications to pattern recognition

制度:新 ; 報告番号:甲3136号 ; 学位の種類:博士(工学) ; 授与年月日:2010/7/12 ; 早大学位記番号:新541

BM3D Image Denoising using Learning-Based Adaptive Hard Thresholding

Image denoising is an important pre-processing step in most imaging applications. Block Matching and 3D Filtering (BM3D) is considered to be the current state-of-art algorithm for additive image denoising. But this algorithm uses a fixed hard thresholding scheme to attenuate noise from a 3D block. Experiments show that this fixed hard thresholding deteriorates the performance of BM3D because it does not consider the context of corresponding blocks. In this thesis, we propose a learning based adaptive hard thresholding method to solve this issue. Also, BM3D algorithm requires as an input the value of the noise level in the input image. But in real life it is not practical to pass as an input such noise level. In this thesis, we also attempt to automatically estimate the level of the noise in the input image. Experimental results demonstrate that our proposed algorithm outperforms BM3D in both objective and subjective fidelity criteria

Classification task-driven efficient feature extraction from tensor data

Automatic classification of complex data is an area of great interest as it allows to make efficient use of the increasingly data intensive environment that characterizes our modern world. This thesis presents to two contributions to this research area. Firstly, the problem of discriminative feature extraction for data organized in multidimensional arrays. In machine learning, Linear Discriminant Analysis (LDA) is a popular discriminative feature extraction method based on optimizing a Fisher type criterion to find the most discriminative data projection. Various extension of LDA to high-order tensor data have been developed. The method proposed is called the Efficient Greedy Feature Extraction method (EGFE). This method avoids solving optimization problems of very high dimension. Also, it can be stopped when the extracted features are deemed to be sufficient for a proper discrimination of the classes. Secondly, an application of EGFE methods to early detection of dementia disease. For the early detection task, four cognitive scores are used as the original data while we employ our greedy feature extraction method to derive discriminative privileged information feature from fMRI data. The results from the experiments presented in this thesis demonstrate the advantage of using privileged information for the early detection task

Integration of blcm and flbp in low resolution face recognition

Face recognition from face image has been a fast-growing topic in biometrics research community and a sizeable number of face recognition techniques based on texture analysis have been developed in the past few years. These techniques work well on grayscale and colour images with very few techniques deal with binary and low resolution image. With binary image becoming the preferred format for low face resolution analysis, there is need for further studies to provide a complete solution for image-based face recognition system with higher accuracy. To overcome the limitation of the existing techniques in extracting distinctive features in low resolution images due to the contrast between the face and background, we proposed a statistical feature analysis technique to fill in the gaps. To achieve this, the proposed technique integrates Binary Level Occurrence Matrix (BLCM) and Fuzzy Local Binary Pattern (FLBP) named BLCM-FLBP to extract global and local features of face from face low resolution images. The purpose of BLCM-FLBP is to distinctively improve performance of edge sharpness between black and white pixels in the binary image and to extract significant data relating to the features of face pattern. Experimental results on Yale and FEI datasets validates the superiority of the proposed technique over the other top-performing feature analysis techniques methods by utilizing different classifier which is Neural network (NN) and Random Forest (RF). The proposed technique achieved performance accuracy of 93.16% (RF), 95.27% (NN) when FEI dataset used, and the accuracy of 94.54% (RF), 93.61% (NN) when Yale.B used. Hence, the proposed technique outperforming other technique such as Gray Level Co-Occurrence Matrix (GLCM), Bag of Word (BOW), Fuzzy Local Binary Pattern (FLBP) respectively and Binary Level Occurrence Matrix (BLCM)