4,277 research outputs found
Alternating direction method of multipliers for penalized zero-variance discriminant analysis
We consider the task of classification in the high dimensional setting where
the number of features of the given data is significantly greater than the
number of observations. To accomplish this task, we propose a heuristic, called
sparse zero-variance discriminant analysis (SZVD), for simultaneously
performing linear discriminant analysis and feature selection on high
dimensional data. This method combines classical zero-variance discriminant
analysis, where discriminant vectors are identified in the null space of the
sample within-class covariance matrix, with penalization applied to induce
sparse structures in the resulting vectors. To approximately solve the
resulting nonconvex problem, we develop a simple algorithm based on the
alternating direction method of multipliers. Further, we show that this
algorithm is applicable to a larger class of penalized generalized eigenvalue
problems, including a particular relaxation of the sparse principal component
analysis problem. Finally, we establish theoretical guarantees for convergence
of our algorithm to stationary points of the original nonconvex problem, and
empirically demonstrate the effectiveness of our heuristic for classifying
simulated data and data drawn from applications in time-series classification
Sparsity and adaptivity for the blind separation of partially correlated sources
Blind source separation (BSS) is a very popular technique to analyze
multichannel data. In this context, the data are modeled as the linear
combination of sources to be retrieved. For that purpose, standard BSS methods
all rely on some discrimination principle, whether it is statistical
independence or morphological diversity, to distinguish between the sources.
However, dealing with real-world data reveals that such assumptions are rarely
valid in practice: the signals of interest are more likely partially
correlated, which generally hampers the performances of standard BSS methods.
In this article, we introduce a novel sparsity-enforcing BSS method coined
Adaptive Morphological Component Analysis (AMCA), which is designed to retrieve
sparse and partially correlated sources. More precisely, it makes profit of an
adaptive re-weighting scheme to favor/penalize samples based on their level of
correlation. Extensive numerical experiments have been carried out which show
that the proposed method is robust to the partial correlation of sources while
standard BSS techniques fail. The AMCA algorithm is evaluated in the field of
astrophysics for the separation of physical components from microwave data.Comment: submitted to IEEE Transactions on signal processin
Enhancement of ELDA Tracker Based on CNN Features and Adaptive Model Update
Appearance representation and the observation model are the most important components in designing a robust visual tracking algorithm for video-based sensors. Additionally, the exemplar-based linear discriminant analysis (ELDA) model has shown good performance in object tracking. Based on that, we improve the ELDA tracking algorithm by deep convolutional neural network (CNN) features and adaptive model update. Deep CNN features have been successfully used in various computer vision tasks. Extracting CNN features on all of the candidate windows is time consuming. To address this problem, a two-step CNN feature extraction method is proposed by separately computing convolutional layers and fully-connected layers. Due to the strong discriminative ability of CNN features and the exemplar-based model, we update both object and background models to improve their adaptivity and to deal with the tradeoff between discriminative ability and adaptivity. An object updating method is proposed to select the “good” models (detectors), which are quite discriminative and uncorrelated to other selected models. Meanwhile, we build the background model as a Gaussian mixture model (GMM) to adapt to complex scenes, which is initialized offline and updated online. The proposed tracker is evaluated on a benchmark dataset of 50 video sequences with various challenges. It achieves the best overall performance among the compared state-of-the-art trackers, which demonstrates the effectiveness and robustness of our tracking algorithm
Sparse canonical correlation analysis from a predictive point of view
Canonical correlation analysis (CCA) describes the associations between two
sets of variables by maximizing the correlation between linear combinations of
the variables in each data set. However, in high-dimensional settings where the
number of variables exceeds the sample size or when the variables are highly
correlated, traditional CCA is no longer appropriate. This paper proposes a
method for sparse CCA. Sparse estimation produces linear combinations of only a
subset of variables from each data set, thereby increasing the interpretability
of the canonical variates. We consider the CCA problem from a predictive point
of view and recast it into a regression framework. By combining an alternating
regression approach together with a lasso penalty, we induce sparsity in the
canonical vectors. We compare the performance with other sparse CCA techniques
in different simulation settings and illustrate its usefulness on a genomic
data set
- …