138 research outputs found
A note on sparse least-squares regression
We compute a \emph{sparse} solution to the classical least-squares problem
where is an arbitrary matrix. We describe a novel
algorithm for this sparse least-squares problem. The algorithm operates as
follows: first, it selects columns from , and then solves a least-squares
problem only with the selected columns. The column selection algorithm that we
use is known to perform well for the well studied column subset selection
problem. The contribution of this article is to show that it gives favorable
results for sparse least-squares as well. Specifically, we prove that the
solution vector obtained by our algorithm is close to the solution vector
obtained via what is known as the "SVD-truncated regularization approach".Comment: Information Processing Letters, to appea
Provable Deterministic Leverage Score Sampling
We explain theoretically a curious empirical phenomenon: "Approximating a
matrix by deterministically selecting a subset of its columns with the
corresponding largest leverage scores results in a good low-rank matrix
surrogate". To obtain provable guarantees, previous work requires randomized
sampling of the columns with probabilities proportional to their leverage
scores.
In this work, we provide a novel theoretical analysis of deterministic
leverage score sampling. We show that such deterministic sampling can be
provably as accurate as its randomized counterparts, if the leverage scores
follow a moderately steep power-law decay. We support this power-law assumption
by providing empirical evidence that such decay laws are abundant in real-world
data sets. We then demonstrate empirically the performance of deterministic
leverage score sampling, which many times matches or outperforms the
state-of-the-art techniques.Comment: 20th ACM SIGKDD Conference on Knowledge Discovery and Data Minin
Optimal CUR Matrix Decompositions
The CUR decomposition of an matrix finds an
matrix with a subset of columns of together with an matrix with a subset of rows of as well as a
low-rank matrix such that the matrix approximates the matrix
that is, , where
denotes the Frobenius norm and is the best matrix
of rank constructed via the SVD. We present input-sparsity-time and
deterministic algorithms for constructing such a CUR decomposition where
and and rank. Up to constant
factors, our algorithms are simultaneously optimal in and rank.Comment: small revision in lemma 4.
- …