4,279 research outputs found
Robust PCA as Bilinear Decomposition with Outlier-Sparsity Regularization
Principal component analysis (PCA) is widely used for dimensionality
reduction, with well-documented merits in various applications involving
high-dimensional data, including computer vision, preference measurement, and
bioinformatics. In this context, the fresh look advocated here permeates
benefits from variable selection and compressive sampling, to robustify PCA
against outliers. A least-trimmed squares estimator of a low-rank bilinear
factor analysis model is shown closely related to that obtained from an
-(pseudo)norm-regularized criterion encouraging sparsity in a matrix
explicitly modeling the outliers. This connection suggests robust PCA schemes
based on convex relaxation, which lead naturally to a family of robust
estimators encompassing Huber's optimal M-class as a special case. Outliers are
identified by tuning a regularization parameter, which amounts to controlling
sparsity of the outlier matrix along the whole robustification path of (group)
least-absolute shrinkage and selection operator (Lasso) solutions. Beyond its
neat ties to robust statistics, the developed outlier-aware PCA framework is
versatile to accommodate novel and scalable algorithms to: i) track the
low-rank signal subspace robustly, as new data are acquired in real time; and
ii) determine principal components robustly in (possibly) infinite-dimensional
feature spaces. Synthetic and real data tests corroborate the effectiveness of
the proposed robust PCA schemes, when used to identify aberrant responses in
personality assessment surveys, as well as unveil communities in social
networks, and intruders from video surveillance data.Comment: 30 pages, submitted to IEEE Transactions on Signal Processin
Manifold Elastic Net: A Unified Framework for Sparse Dimension Reduction
It is difficult to find the optimal sparse solution of a manifold learning
based dimensionality reduction algorithm. The lasso or the elastic net
penalized manifold learning based dimensionality reduction is not directly a
lasso penalized least square problem and thus the least angle regression (LARS)
(Efron et al. \cite{LARS}), one of the most popular algorithms in sparse
learning, cannot be applied. Therefore, most current approaches take indirect
ways or have strict settings, which can be inconvenient for applications. In
this paper, we proposed the manifold elastic net or MEN for short. MEN
incorporates the merits of both the manifold learning based dimensionality
reduction and the sparse learning based dimensionality reduction. By using a
series of equivalent transformations, we show MEN is equivalent to the lasso
penalized least square problem and thus LARS is adopted to obtain the optimal
sparse solution of MEN. In particular, MEN has the following advantages for
subsequent classification: 1) the local geometry of samples is well preserved
for low dimensional data representation, 2) both the margin maximization and
the classification error minimization are considered for sparse projection
calculation, 3) the projection matrix of MEN improves the parsimony in
computation, 4) the elastic net penalty reduces the over-fitting problem, and
5) the projection matrix of MEN can be interpreted psychologically and
physiologically. Experimental evidence on face recognition over various popular
datasets suggests that MEN is superior to top level dimensionality reduction
algorithms.Comment: 33 pages, 12 figure
Semi-blind Sparse Image Reconstruction with Application to MRFM
We propose a solution to the image deconvolution problem where the
convolution kernel or point spread function (PSF) is assumed to be only
partially known. Small perturbations generated from the model are exploited to
produce a few principal components explaining the PSF uncertainty in a high
dimensional space. Unlike recent developments on blind deconvolution of natural
images, we assume the image is sparse in the pixel basis, a natural sparsity
arising in magnetic resonance force microscopy (MRFM). Our approach adopts a
Bayesian Metropolis-within-Gibbs sampling framework. The performance of our
Bayesian semi-blind algorithm for sparse images is superior to previously
proposed semi-blind algorithms such as the alternating minimization (AM)
algorithm and blind algorithms developed for natural images. We illustrate our
myopic algorithm on real MRFM tobacco virus data.Comment: This work has been submitted to the IEEE Trans. Image Processing for
possible publicatio
Conditional Gradient Algorithms for Rank-One Matrix Approximations with a Sparsity Constraint
The sparsity constrained rank-one matrix approximation problem is a difficult
mathematical optimization problem which arises in a wide array of useful
applications in engineering, machine learning and statistics, and the design of
algorithms for this problem has attracted intensive research activities. We
introduce an algorithmic framework, called ConGradU, that unifies a variety of
seemingly different algorithms that have been derived from disparate
approaches, and allows for deriving new schemes. Building on the old and
well-known conditional gradient algorithm, ConGradU is a simplified version
with unit step size and yields a generic algorithm which either is given by an
analytic formula or requires a very low computational complexity. Mathematical
properties are systematically developed and numerical experiments are given.Comment: Minor changes. Final version. To appear in SIAM Revie
Adaptive Image Denoising by Targeted Databases
We propose a data-dependent denoising procedure to restore noisy images.
Different from existing denoising algorithms which search for patches from
either the noisy image or a generic database, the new algorithm finds patches
from a database that contains only relevant patches. We formulate the denoising
problem as an optimal filter design problem and make two contributions. First,
we determine the basis function of the denoising filter by solving a group
sparsity minimization problem. The optimization formulation generalizes
existing denoising algorithms and offers systematic analysis of the
performance. Improvement methods are proposed to enhance the patch search
process. Second, we determine the spectral coefficients of the denoising filter
by considering a localized Bayesian prior. The localized prior leverages the
similarity of the targeted database, alleviates the intensive Bayesian
computation, and links the new method to the classical linear minimum mean
squared error estimation. We demonstrate applications of the proposed method in
a variety of scenarios, including text images, multiview images and face
images. Experimental results show the superiority of the new algorithm over
existing methods.Comment: 15 pages, 13 figures, 2 tables, journa
- âŠ