48,207 research outputs found
Feature Grouping and Sparse Principal Component Analysis
Sparse Principal Component Analysis (SPCA) is widely used in data processing
and dimension reduction; it uses the lasso to produce modified principal
components with sparse loadings for better interpretability. However, sparse
PCA never considers an additional grouping structure where the loadings share
similar coefficients (i.e., feature grouping), besides a special group with all
coefficients being zero (i.e., feature selection). In this paper, we propose a
novel method called Feature Grouping and Sparse Principal Component Analysis
(FGSPCA) which allows the loadings to belong to disjoint homogeneous groups,
with sparsity as a special case. The proposed FGSPCA is a subspace learning
method designed to simultaneously perform grouping pursuit and feature
selection, by imposing a non-convex regularization with naturally adjustable
sparsity and grouping effect. To solve the resulting non-convex optimization
problem, we propose an alternating algorithm that incorporates the
difference-of-convex programming, augmented Lagrange and coordinate descent
methods. Additionally, the experimental results on real data sets show that the
proposed FGSPCA benefits from the grouping effect compared with methods without
grouping effect.Comment: 21 pages, 5 figures, 2 table
Robust PCA as Bilinear Decomposition with Outlier-Sparsity Regularization
Principal component analysis (PCA) is widely used for dimensionality
reduction, with well-documented merits in various applications involving
high-dimensional data, including computer vision, preference measurement, and
bioinformatics. In this context, the fresh look advocated here permeates
benefits from variable selection and compressive sampling, to robustify PCA
against outliers. A least-trimmed squares estimator of a low-rank bilinear
factor analysis model is shown closely related to that obtained from an
-(pseudo)norm-regularized criterion encouraging sparsity in a matrix
explicitly modeling the outliers. This connection suggests robust PCA schemes
based on convex relaxation, which lead naturally to a family of robust
estimators encompassing Huber's optimal M-class as a special case. Outliers are
identified by tuning a regularization parameter, which amounts to controlling
sparsity of the outlier matrix along the whole robustification path of (group)
least-absolute shrinkage and selection operator (Lasso) solutions. Beyond its
neat ties to robust statistics, the developed outlier-aware PCA framework is
versatile to accommodate novel and scalable algorithms to: i) track the
low-rank signal subspace robustly, as new data are acquired in real time; and
ii) determine principal components robustly in (possibly) infinite-dimensional
feature spaces. Synthetic and real data tests corroborate the effectiveness of
the proposed robust PCA schemes, when used to identify aberrant responses in
personality assessment surveys, as well as unveil communities in social
networks, and intruders from video surveillance data.Comment: 30 pages, submitted to IEEE Transactions on Signal Processin
A D.C. Programming Approach to the Sparse Generalized Eigenvalue Problem
In this paper, we consider the sparse eigenvalue problem wherein the goal is
to obtain a sparse solution to the generalized eigenvalue problem. We achieve
this by constraining the cardinality of the solution to the generalized
eigenvalue problem and obtain sparse principal component analysis (PCA), sparse
canonical correlation analysis (CCA) and sparse Fisher discriminant analysis
(FDA) as special cases. Unlike the -norm approximation to the
cardinality constraint, which previous methods have used in the context of
sparse PCA, we propose a tighter approximation that is related to the negative
log-likelihood of a Student's t-distribution. The problem is then framed as a
d.c. (difference of convex functions) program and is solved as a sequence of
convex programs by invoking the majorization-minimization method. The resulting
algorithm is proved to exhibit \emph{global convergence} behavior, i.e., for
any random initialization, the sequence (subsequence) of iterates generated by
the algorithm converges to a stationary point of the d.c. program. The
performance of the algorithm is empirically demonstrated on both sparse PCA
(finding few relevant genes that explain as much variance as possible in a
high-dimensional gene dataset) and sparse CCA (cross-language document
retrieval and vocabulary selection for music retrieval) applications.Comment: 40 page
Implementation of a local principal curves algorithm for neutrino interaction reconstruction in a liquid argon volume
A local principal curve algorithm has been implemented in three dimensions
for automated track and shower reconstruction of neutrino interactions in a
liquid argon time projection chamber. We present details of the algorithm and
characterise its performance on simulated data sets.Comment: 14 pages, 17 figures; typing correction to Eq 5, the definition of
the local covariance matri
Stochastic Parallel Block Coordinate Descent for Large-scale Saddle Point Problems
We consider convex-concave saddle point problems with a separable structure
and non-strongly convex functions. We propose an efficient stochastic block
coordinate descent method using adaptive primal-dual updates, which enables
flexible parallel optimization for large-scale problems. Our method shares the
efficiency and flexibility of block coordinate descent methods with the
simplicity of primal-dual methods and utilizing the structure of the separable
convex-concave saddle point problem. It is capable of solving a wide range of
machine learning applications, including robust principal component analysis,
Lasso, and feature selection by group Lasso, etc. Theoretically and
empirically, we demonstrate significantly better performance than
state-of-the-art methods in all these applications.Comment: Accepted by AAAI 201
- …