2 research outputs found
Low rank approximation and decomposition of large matrices using error correcting codes
Low rank approximation is an important tool used in many applications of
signal processing and machine learning. Recently, randomized sketching
algorithms were proposed to effectively construct low rank approximations and
obtain approximate singular value decompositions of large matrices. Similar
ideas were used to solve least squares regression problems. In this paper, we
show how matrices from error correcting codes can be used to find such low rank
approximations and matrix decompositions, and extend the framework to linear
least squares regression problems. The benefits of using these code matrices
are the following: (i) They are easy to generate and they reduce randomness
significantly. (ii) Code matrices with mild properties satisfy the subspace
embedding property, and have a better chance of preserving the geometry of an
entire subspace of vectors. (iii) For parallel and distributed applications,
code matrices have significant advantages over structured random matrices and
Gaussian random matrices. (iv) Unlike Fourier or Hadamard transform matrices,
which require sampling columns for a rank- approximation, the
log factor is not necessary for certain types of code matrices. That is,
optimal Frobenius norm error can be achieved for a rank-
approximation with samples. (v) Fast multiplication is possible
with structured code matrices, so fast approximations can be achieved for
general dense input matrices. (vi) For least squares regression problem
where , the
relative error approximation can be achieved with samples, with
high probability, when certain code matrices are used
Sampling and multilevel coarsening algorithms for fast matrix approximations
This paper addresses matrix approximation problems for matrices that are
large, sparse and/or that are representations of large graphs. To tackle these
problems, we consider algorithms that are based primarily on coarsening
techniques, possibly combined with random sampling. A multilevel coarsening
technique is proposed which utilizes a hypergraph associated with the data
matrix and a graph coarsening strategy based on column matching. Theoretical
results are established that characterize the quality of the dimension
reduction achieved by a coarsening step, when a proper column matching strategy
is employed. We consider a number of standard applications of this technique as
well as a few new ones. Among the standard applications we first consider the
problem of computing the partial SVD for which a combination of sampling and
coarsening yields significantly improved SVD results relative to sampling
alone. We also consider the Column subset selection problem, a popular low rank
approximation method used in data related applications, and show how multilevel
coarsening can be adapted for this problem. Similarly, we consider the problem
of graph sparsification and show how coarsening techniques can be employed to
solve it. Numerical experiments illustrate the performances of the methods in
various applications