4,592 research outputs found
Column Subset Selection, Matrix Factorization, and Eigenvalue Optimization
Given a fixed matrix, the problem of column subset selection requests a
column submatrix that has favorable spectral properties. Most research from the
algorithms and numerical linear algebra communities focuses on a variant called
rank-revealing {\sf QR}, which seeks a well-conditioned collection of columns
that spans the (numerical) range of the matrix. The functional analysis
literature contains another strand of work on column selection whose
algorithmic implications have not been explored. In particular, a celebrated
result of Bourgain and Tzafriri demonstrates that each matrix with normalized
columns contains a large column submatrix that is exceptionally well
conditioned. Unfortunately, standard proofs of this result cannot be regarded
as algorithmic.
This paper presents a randomized, polynomial-time algorithm that produces the
submatrix promised by Bourgain and Tzafriri. The method involves random
sampling of columns, followed by a matrix factorization that exposes the
well-conditioned subset of columns. This factorization, which is due to
Grothendieck, is regarded as a central tool in modern functional analysis. The
primary novelty in this work is an algorithm, based on eigenvalue minimization,
for constructing the Grothendieck factorization. These ideas also result in a
novel approximation algorithm for the norm of a matrix, which is
generally {\sf NP}-hard to compute exactly. As an added bonus, this work
reveals a surprising connection between matrix factorization and the famous
{\sc maxcut} semidefinite program.Comment: Conference versio
Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions
Low-rank matrix approximations, such as the truncated singular value decomposition and the rank-revealing QR decomposition, play a central role in data analysis and scientific computing. This work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low-rank matrix approximation. These techniques exploit modern computational architectures more fully than classical methods and open the possibility of dealing with truly massive data sets. This paper presents a modular framework for constructing randomized algorithms that compute partial matrix decompositions. These methods use random sampling to identify a subspace that captures most of the action of a matrix. The input matrix is then compressed—either explicitly or
implicitly—to this subspace, and the reduced matrix is manipulated deterministically to obtain the desired low-rank factorization. In many cases, this approach beats its classical competitors in terms of accuracy, robustness, and/or speed. These claims are supported by extensive numerical experiments and a detailed error analysis. The specific benefits of randomized techniques depend on the computational environment. Consider the model problem of finding the k dominant components of the singular value decomposition of an m × n matrix. (i) For a dense input matrix, randomized algorithms require O(mn log(k))
floating-point operations (flops) in contrast to O(mnk) for classical algorithms. (ii) For a sparse input matrix, the flop count matches classical Krylov subspace methods, but the randomized approach is more robust and can easily be reorganized to exploit multiprocessor architectures. (iii) For a matrix that is too large to fit in fast memory, the randomized techniques require only a constant number of passes over the data, as opposed to O(k) passes for classical algorithms. In fact, it is sometimes possible to perform matrix approximation with a single pass over the data
Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions
Low-rank matrix approximations, such as the truncated singular value
decomposition and the rank-revealing QR decomposition, play a central role in
data analysis and scientific computing. This work surveys and extends recent
research which demonstrates that randomization offers a powerful tool for
performing low-rank matrix approximation. These techniques exploit modern
computational architectures more fully than classical methods and open the
possibility of dealing with truly massive data sets.
This paper presents a modular framework for constructing randomized
algorithms that compute partial matrix decompositions. These methods use random
sampling to identify a subspace that captures most of the action of a matrix.
The input matrix is then compressed---either explicitly or implicitly---to this
subspace, and the reduced matrix is manipulated deterministically to obtain the
desired low-rank factorization. In many cases, this approach beats its
classical competitors in terms of accuracy, speed, and robustness. These claims
are supported by extensive numerical experiments and a detailed error analysis
- …