70,683 research outputs found
Representing complex data using localized principal components with application to astronomical data
Often the relation between the variables constituting a multivariate data
space might be characterized by one or more of the terms: ``nonlinear'',
``branched'', ``disconnected'', ``bended'', ``curved'', ``heterogeneous'', or,
more general, ``complex''. In these cases, simple principal component analysis
(PCA) as a tool for dimension reduction can fail badly. Of the many alternative
approaches proposed so far, local approximations of PCA are among the most
promising. This paper will give a short review of localized versions of PCA,
focusing on local principal curves and local partitioning algorithms.
Furthermore we discuss projections other than the local principal components.
When performing local dimension reduction for regression or classification
problems it is important to focus not only on the manifold structure of the
covariates, but also on the response variable(s). Local principal components
only achieve the former, whereas localized regression approaches concentrate on
the latter. Local projection directions derived from the partial least squares
(PLS) algorithm offer an interesting trade-off between these two objectives. We
apply these methods to several real data sets. In particular, we consider
simulated astrophysical data from the future Galactic survey mission Gaia.Comment: 25 pages. In "Principal Manifolds for Data Visualization and
Dimension Reduction", A. Gorban, B. Kegl, D. Wunsch, and A. Zinovyev (eds),
Lecture Notes in Computational Science and Engineering, Springer, 2007, pp.
180--204,
http://www.springer.com/dal/home/generic/search/results?SGWID=1-40109-22-173750210-
Accelerating Wireless Federated Learning via Nesterov's Momentum and Distributed Principle Component Analysis
A wireless federated learning system is investigated by allowing a server and
workers to exchange uncoded information via orthogonal wireless channels. Since
the workers frequently upload local gradients to the server via
bandwidth-limited channels, the uplink transmission from the workers to the
server becomes a communication bottleneck. Therefore, a one-shot distributed
principle component analysis (PCA) is leveraged to reduce the dimension of
uploaded gradients such that the communication bottleneck is relieved. A
PCA-based wireless federated learning (PCA-WFL) algorithm and its accelerated
version (i.e., PCA-AWFL) are proposed based on the low-dimensional gradients
and the Nesterov's momentum. For the non-convex loss functions, a finite-time
analysis is performed to quantify the impacts of system hyper-parameters on the
convergence of the PCA-WFL and PCA-AWFL algorithms. The PCA-AWFL algorithm is
theoretically certified to converge faster than the PCA-WFL algorithm. Besides,
the convergence rates of PCA-WFL and PCA-AWFL algorithms quantitatively reveal
the linear speedup with respect to the number of workers over the vanilla
gradient descent algorithm. Numerical results are used to demonstrate the
improved convergence rates of the proposed PCA-WFL and PCA-AWFL algorithms over
the benchmarks
Masking Strategies for Image Manifolds
We consider the problem of selecting an optimal mask for an image manifold,
i.e., choosing a subset of the pixels of the image that preserves the
manifold's geometric structure present in the original data. Such masking
implements a form of compressive sensing through emerging imaging sensor
platforms for which the power expense grows with the number of pixels acquired.
Our goal is for the manifold learned from masked images to resemble its full
image counterpart as closely as possible. More precisely, we show that one can
indeed accurately learn an image manifold without having to consider a large
majority of the image pixels. In doing so, we consider two masking methods that
preserve the local and global geometric structure of the manifold,
respectively. In each case, the process of finding the optimal masking pattern
can be cast as a binary integer program, which is computationally expensive but
can be approximated by a fast greedy algorithm. Numerical experiments show that
the relevant manifold structure is preserved through the data-dependent masking
process, even for modest mask sizes
Manifold Elastic Net: A Unified Framework for Sparse Dimension Reduction
It is difficult to find the optimal sparse solution of a manifold learning
based dimensionality reduction algorithm. The lasso or the elastic net
penalized manifold learning based dimensionality reduction is not directly a
lasso penalized least square problem and thus the least angle regression (LARS)
(Efron et al. \cite{LARS}), one of the most popular algorithms in sparse
learning, cannot be applied. Therefore, most current approaches take indirect
ways or have strict settings, which can be inconvenient for applications. In
this paper, we proposed the manifold elastic net or MEN for short. MEN
incorporates the merits of both the manifold learning based dimensionality
reduction and the sparse learning based dimensionality reduction. By using a
series of equivalent transformations, we show MEN is equivalent to the lasso
penalized least square problem and thus LARS is adopted to obtain the optimal
sparse solution of MEN. In particular, MEN has the following advantages for
subsequent classification: 1) the local geometry of samples is well preserved
for low dimensional data representation, 2) both the margin maximization and
the classification error minimization are considered for sparse projection
calculation, 3) the projection matrix of MEN improves the parsimony in
computation, 4) the elastic net penalty reduces the over-fitting problem, and
5) the projection matrix of MEN can be interpreted psychologically and
physiologically. Experimental evidence on face recognition over various popular
datasets suggests that MEN is superior to top level dimensionality reduction
algorithms.Comment: 33 pages, 12 figure
Generalized power method for sparse principal component analysis
In this paper we develop a new approach to sparse principal component
analysis (sparse PCA). We propose two single-unit and two block optimization
formulations of the sparse PCA problem, aimed at extracting a single sparse
dominant principal component of a data matrix, or more components at once,
respectively. While the initial formulations involve nonconvex functions, and
are therefore computationally intractable, we rewrite them into the form of an
optimization program involving maximization of a convex function on a compact
set. The dimension of the search space is decreased enormously if the data
matrix has many more columns (variables) than rows. We then propose and analyze
a simple gradient method suited for the task. It appears that our algorithm has
best convergence properties in the case when either the objective function or
the feasible set are strongly convex, which is the case with our single-unit
formulations and can be enforced in the block case. Finally, we demonstrate
numerically on a set of random and gene expression test problems that our
approach outperforms existing algorithms both in quality of the obtained
solution and in computational speed.Comment: Submitte
- …