Latent Fisher Discriminant Analysis

Linear Discriminant Analysis (LDA) is a well-known method for dimensionality reduction and classification. Previous studies have also extended the binary-class case into multi-classes. However, many applications, such as object detection and keyframe extraction cannot provide consistent instance-label pairs, while LDA requires labels on instance level for training. Thus it cannot be directly applied for semi-supervised classification problem. In this paper, we overcome this limitation and propose a latent variable Fisher discriminant analysis model. We relax the instance-level labeling into bag-level, is a kind of semi-supervised (video-level labels of event type are required for semantic frame extraction) and incorporates a data-driven prior over the latent variables. Hence, our method combines the latent variable inference and dimension reduction in an unified bayesian framework. We test our method on MUSK and Corel data sets and yield competitive results compared to the baseline approach. We also demonstrate its capacity on the challenging TRECVID MED11 dataset for semantic keyframe extraction and conduct a human-factors ranking-based experimental evaluation, which clearly demonstrates our proposed method consistently extracts more semantically meaningful keyframes than challenging baselines.Comment: 12 page

Manifold Learning in MR spectroscopy using nonlinear dimensionality reduction and unsupervised clustering

Purpose To investigate whether nonlinear dimensionality reduction improves unsupervised classification of 1H MRS brain tumor data compared with a linear method. Methods In vivo single-voxel 1H magnetic resonance spectroscopy (55 patients) and 1H magnetic resonance spectroscopy imaging (MRSI) (29 patients) data were acquired from histopathologically diagnosed gliomas. Data reduction using Laplacian eigenmaps (LE) or independent component analysis (ICA) was followed by k-means clustering or agglomerative hierarchical clustering (AHC) for unsupervised learning to assess tumor grade and for tissue type segmentation of MRSI data. Results An accuracy of 93% in classification of glioma grade II and grade IV, with 100% accuracy in distinguishing tumor and normal spectra, was obtained by LE with unsupervised clustering, but not with the combination of k-means and ICA. With 1H MRSI data, LE provided a more linear distribution of data for cluster analysis and better cluster stability than ICA. LE combined with k-means or AHC provided 91% accuracy for classifying tumor grade and 100% accuracy for identifying normal tissue voxels. Color-coded visualization of normal brain, tumor core, and infiltration regions was achieved with LE combined with AHC. Conclusion Purpose To investigate whether nonlinear dimensionality reduction improves unsupervised classification of 1H MRS brain tumor data compared with a linear method. Methods In vivo single-voxel 1H magnetic resonance spectroscopy (55 patients) and 1H magnetic resonance spectroscopy imaging (MRSI) (29 patients) data were acquired from histopathologically diagnosed gliomas. Data reduction using Laplacian eigenmaps (LE) or independent component analysis (ICA) was followed by k-means clustering or agglomerative hierarchical clustering (AHC) for unsupervised learning to assess tumor grade and for tissue type segmentation of MRSI data. Results An accuracy of 93% in classification of glioma grade II and grade IV, with 100% accuracy in distinguishing tumor and normal spectra, was obtained by LE with unsupervised clustering, but not with the combination of k-means and ICA. With 1H MRSI data, LE provided a more linear distribution of data for cluster analysis and better cluster stability than ICA. LE combined with k-means or AHC provided 91% accuracy for classifying tumor grade and 100% accuracy for identifying normal tissue voxels. Color-coded visualization of normal brain, tumor core, and infiltration regions was achieved with LE combined with AHC. Conclusion The LE method is promising for unsupervised clustering to separate brain and tumor tissue with automated color-coding for visualization of 1H MRSI data after cluster analysis

Classification of Carpiodes Using Fourier Descriptors: A Content Based Image Retrieval Approach

Taxonomic classification has always been important to the study of any biological system. Many biological species will go unclassified and become lost forever at the current rate of classification. The current state of computer technology makes image storage and retrieval possible on a global level. As a result, computer-aided taxonomy is now possible. Content based image retrieval techniques utilize visual features of the image for classification. By utilizing image content and computer technology, the gap between taxonomic classification and species destruction is shrinking. This content based study utilizes the Fourier Descriptors of fifteen known landmark features on three Carpiodes species: C.carpio, C.velifer, and C.cyprinus. Classification analysis involves both unsupervised and supervised machine learning algorithms. Fourier Descriptors of the fifteen known landmarks provide for strong classification power on image data. Feature reduction analysis indicates feature reduction is possible. This proves useful for increasing generalization power of classification

A Novel Hybrid Dimensionality Reduction Method using Support Vector Machines and Independent Component Analysis

Due to the increasing demand for high dimensional data analysis from various applications such as electrocardiogram signal analysis and gene expression analysis for cancer detection, dimensionality reduction becomes a viable process to extracts essential information from data such that the high-dimensional data can be represented in a more condensed form with much lower dimensionality to both improve classification accuracy and reduce computational complexity. Conventional dimensionality reduction methods can be categorized into stand-alone and hybrid approaches. The stand-alone method utilizes a single criterion from either supervised or unsupervised perspective. On the other hand, the hybrid method integrates both criteria. Compared with a variety of stand-alone dimensionality reduction methods, the hybrid approach is promising as it takes advantage of both the supervised criterion for better classification accuracy and the unsupervised criterion for better data representation, simultaneously. However, several issues always exist that challenge the efficiency of the hybrid approach, including (1) the difficulty in finding a subspace that seamlessly integrates both criteria in a single hybrid framework, (2) the robustness of the performance regarding noisy data, and (3) nonlinear data representation capability. This dissertation presents a new hybrid dimensionality reduction method to seek projection through optimization of both structural risk (supervised criterion) from Support Vector Machine (SVM) and data independence (unsupervised criterion) from Independent Component Analysis (ICA). The projection from SVM directly contributes to classification performance improvement in a supervised perspective whereas maximum independence among features by ICA construct projection indirectly achieving classification accuracy improvement due to better intrinsic data representation in an unsupervised perspective. For linear dimensionality reduction model, I introduce orthogonality to interrelate both projections from SVM and ICA while redundancy removal process eliminates a part of the projection vectors from SVM, leading to more effective dimensionality reduction. The orthogonality-based linear hybrid dimensionality reduction method is extended to uncorrelatedness-based algorithm with nonlinear data representation capability. In the proposed approach, SVM and ICA are integrated into a single framework by the uncorrelated subspace based on kernel implementation. Experimental results show that the proposed approaches give higher classification performance with better robustness in relatively lower dimensions than conventional methods for high-dimensional datasets

Incorporating Multiresolution Analysis With Multiclassifiers And Decision Fusion For Hyperspectral Remote Sensing

The ongoing development and increased affordability of hyperspectral sensors are increasing their utilization in a variety of applications, such as agricultural monitoring and decision making. Hyperspectral Automated Target Recognition (ATR) systems typically rely heavily on dimensionality reduction methods, and particularly intelligent reduction methods referred to as feature extraction techniques. This dissertation reports on the development, implementation, and testing of new hyperspectral analysis techniques for ATR systems, including their use in agricultural applications where ground truthed observations available for training the ATR system are typically very limited. This dissertation reports the design of effective methods for grouping and down-selecting Discrete Wavelet Transform (DWT) coefficients and the design of automated Wavelet Packet Decomposition (WPD) filter tree pruning methods for use within the framework of a Multiclassifiers and Decision Fusion (MCDF) ATR system. The efficacy of the DWT MCDF and WPD MCDF systems are compared to existing ATR methods commonly used in hyperspectral remote sensing applications. The newly developed methods’ sensitivity to operating conditions, such as mother wavelet selection, decomposition level, and quantity and quality of available training data are also investigated. The newly developed ATR systems are applied to the problem of hyperspectral remote sensing of agricultural food crop contaminations either by airborne chemical application, specifically Glufosinate herbicide at varying concentrations applied to corn crops, or by biological infestation, specifically soybean rust disease in soybean crops. The DWT MCDF and WPD MCDF methods significantly outperform conventional hyperspectral ATR methods. For example, when detecting and classifying varying levels of soybean rust infestation, stepwise linear discriminant analysis, results in accuracies of approximately 30%-40%, but WPD MCDF methods result in accuracies of approximately 70%-80%

Tensor Canonical Correlation Analysis for Multi-View Dimension Reduction

© 2015 IEEE. Canonical correlation analysis (CCA) has proven an effective tool for two-view dimension reduction due to its profound theoretical foundation and success in practical applications. In respect of multi-view learning, however, it is limited by its capability of only handling data represented by two-view features, while in many real-world applications, the number of views is frequently many more. Although the ad hoc way of simultaneously exploring all possible pairs of features can numerically deal with multi-view data, it ignores the high order statistics (correlation information) which can only be discovered by simultaneously exploring all features. Therefore, in this work, we develop tensor CCA (TCCA) which straightforwardly yet naturally generalizes CCA to handle the data of an arbitrary number of views by analyzing the covariance tensor of the different views. TCCA aims to directly maximize the canonical correlation of multiple (more than two) views. Crucially, we prove that the main problem of multi-view canonical correlation maximization is equivalent to finding the best rank-1 approximation of the data covariance tensor, which can be solved efficiently using the well-known alternating least squares (ALS) algorithm. As a consequence, the high order correlation information contained in the different views is explored and thus a more reliable common subspace shared by all features can be obtained. In addition, a non-linear extension of TCCA is presented. Experiments on various challenge tasks, including large scale biometric structure prediction, internet advertisement classification, and web image annotation, demonstrate the effectiveness of the proposed method

Incorporating Multiresolution Analysis With Multiclassifiers And Decision Fusion For Hyperspectral Remote Sensing

The ongoing development and increased affordability of hyperspectral sensors are increasing their utilization in a variety of applications, such as agricultural monitoring and decision making. Hyperspectral Automated Target Recognition (ATR) systems typically rely heavily on dimensionality reduction methods, and particularly intelligent reduction methods referred to as feature extraction techniques. This dissertation reports on the development, implementation, and testing of new hyperspectral analysis techniques for ATR systems, including their use in agricultural applications where ground truthed observations available for training the ATR system are typically very limited. This dissertation reports the design of effective methods for grouping and down-selecting Discrete Wavelet Transform (DWT) coefficients and the design of automated Wavelet Packet Decomposition (WPD) filter tree pruning methods for use within the framework of a Multiclassifiers and Decision Fusion (MCDF) ATR system. The efficacy of the DWT MCDF and WPD MCDF systems are compared to existing ATR methods commonly used in hyperspectral remote sensing applications. The newly developed methods’ sensitivity to operating conditions, such as mother wavelet selection, decomposition level, and quantity and quality of available training data are also investigated. The newly developed ATR systems are applied to the problem of hyperspectral remote sensing of agricultural food crop contaminations either by airborne chemical application, specifically Glufosinate herbicide at varying concentrations applied to corn crops, or by biological infestation, specifically soybean rust disease in soybean crops. The DWT MCDF and WPD MCDF methods significantly outperform conventional hyperspectral ATR methods. For example, when detecting and classifying varying levels of soybean rust infestation, stepwise linear discriminant analysis, results in accuracies of approximately 30%-40%, but WPD MCDF methods result in accuracies of approximately 70%-80%