7,623 research outputs found
Sparse canonical correlation analysis from a predictive point of view
Canonical correlation analysis (CCA) describes the associations between two
sets of variables by maximizing the correlation between linear combinations of
the variables in each data set. However, in high-dimensional settings where the
number of variables exceeds the sample size or when the variables are highly
correlated, traditional CCA is no longer appropriate. This paper proposes a
method for sparse CCA. Sparse estimation produces linear combinations of only a
subset of variables from each data set, thereby increasing the interpretability
of the canonical variates. We consider the CCA problem from a predictive point
of view and recast it into a regression framework. By combining an alternating
regression approach together with a lasso penalty, we induce sparsity in the
canonical vectors. We compare the performance with other sparse CCA techniques
in different simulation settings and illustrate its usefulness on a genomic
data set
Sparse multinomial kernel discriminant analysis (sMKDA)
Dimensionality reduction via canonical variate analysis (CVA) is important for pattern recognition and has been extended variously to permit more flexibility, e.g. by "kernelizing" the formulation. This can lead to over-fitting, usually ameliorated by regularization. Here, a method for sparse, multinomial kernel discriminant analysis (sMKDA) is proposed, using a sparse basis to control complexity. It is based on the connection between CVA and least-squares, and uses forward selection via orthogonal least-squares to approximate a basis, generalizing a similar approach for binomial problems. Classification can be performed directly via minimum Mahalanobis distance in the canonical variates. sMKDA achieves state-of-the-art performance in terms of accuracy and sparseness on 11 benchmark datasets
Penalized Orthogonal Iteration for Sparse Estimation of Generalized Eigenvalue Problem
We propose a new algorithm for sparse estimation of eigenvectors in
generalized eigenvalue problems (GEP). The GEP arises in a number of modern
data-analytic situations and statistical methods, including principal component
analysis (PCA), multiclass linear discriminant analysis (LDA), canonical
correlation analysis (CCA), sufficient dimension reduction (SDR) and invariant
co-ordinate selection. We propose to modify the standard generalized orthogonal
iteration with a sparsity-inducing penalty for the eigenvectors. To achieve
this goal, we generalize the equation-solving step of orthogonal iteration to a
penalized convex optimization problem. The resulting algorithm, called
penalized orthogonal iteration, provides accurate estimation of the true
eigenspace, when it is sparse. Also proposed is a computationally more
efficient alternative, which works well for PCA and LDA problems. Numerical
studies reveal that the proposed algorithms are competitive, and that our
tuning procedure works well. We demonstrate applications of the proposed
algorithm to obtain sparse estimates for PCA, multiclass LDA, CCA and SDR.
Supplementary materials are available online
Robust Sparse Canonical Correlation Analysis
Canonical correlation analysis (CCA) is a multivariate statistical method
which describes the associations between two sets of variables. The objective
is to find linear combinations of the variables in each data set having maximal
correlation. This paper discusses a method for Robust Sparse CCA. Sparse
estimation produces canonical vectors with some of their elements estimated as
exactly zero. As such, their interpretability is improved. We also robustify
the method such that it can cope with outliers in the data. To estimate the
canonical vectors, we convert the CCA problem into an alternating regression
framework, and use the sparse Least Trimmed Squares estimator. We illustrate
the good performance of the Robust Sparse CCA method in several simulation
studies and two real data examples
Fast Multi-Task SCCA Learning with Feature Selection for Multi-Modal Brain Imaging Genetics
Brain imaging genetics studies the genetic basis of brain structures and functions via integrating both genotypic data such as single nucleotide polymorphism (SNP) and imaging quantitative traits (QTs). In this area, both multi-task learning (MTL) and sparse canonical correlation analysis (SCCA) methods are widely used since they are superior to those independent and pairwise univariate analyses. MTL methods generally incorporate a few of QTs and are not designed for feature selection from a large number of QTs; while existing SCCA methods typically employ only one modality of QTs to study its association with SNPs. Both MTL and SCCA encounter computational challenges as the number of SNPs increases. In this paper, combining the merits of MTL and SCCA, we propose a novel multi-task SCCA (MTSCCA) learning framework to identify bi-multivariate associations between SNPs and multi-modal imaging QTs. MTSCCA could make use of the complementary information carried by different imaging modalities. Using the G2,1-norm regularization, MTSCCA treats all SNPs in the same group together to enforce sparsity at the group level. The l2,1-norm penalty is used to jointly select features across multiple tasks for SNPs, and across multiple modalities for QTs. A fast optimization algorithm is proposed using the grouping information of SNPs. Compared with conventional SCCA methods, MTSCCA obtains improved performance regarding both correlation coefficients and canonical weights patterns. In addition, our method runs very fast and is easy-to-implement, and thus could provide a powerful tool for genome-wide brain-wide imaging genetic studies
- …