799 research outputs found
Almost Optimal Unrestricted Fast Johnson-Lindenstrauss Transform
The problems of random projections and sparse reconstruction have much in
common and individually received much attention. Surprisingly, until now they
progressed in parallel and remained mostly separate. Here, we employ new tools
from probability in Banach spaces that were successfully used in the context of
sparse reconstruction to advance on an open problem in random pojection. In
particular, we generalize and use an intricate result by Rudelson and Vershynin
for sparse reconstruction which uses Dudley's theorem for bounding Gaussian
processes. Our main result states that any set of real
vectors in dimensional space can be linearly mapped to a space of dimension
k=O(\log N\polylog(n)), while (1) preserving the pairwise distances among the
vectors to within any constant distortion and (2) being able to apply the
transformation in time on each vector. This improves on the best
known achieved by Ailon and Liberty and by Ailon and Chazelle.
The dependence in the distortion constant however is believed to be
suboptimal and subject to further investigation. For constant distortion, this
settles the open question posed by these authors up to a \polylog(n) factor
while considerably simplifying their constructions
Johnson-Lindenstrauss Transformations
With the quick progression of technology and the increasing need to process large data, there has been an increased interest in data-dependent and data-independent dimension reduction techniques such as principle component analysis (PCA) and Johnson\-Lindenstrauss (JL) transformations, respectively. In 1984, Johnson and Lindenstrauss proved that any finite set of data in a high-dimensional space can be projected into a low-dimensional space while preserving the pairwise Euclidean distance within any desired accuracy, provided the projected dimension is sufficiently large; however, if the desired projected dimension is too small, Woodruff and Jayram, and Kane, Nelson, and Meka in 2011 separately proved such a projection does not exist. In this thesis, we answer an open problem by providing a precise threshold for the projected dimension, above which, there exists a projection approximately preserving the Euclidean distance, but below which, there does not exist such a projection. We, also, give a brief survey of JL constructions, covering the initial constructions and those based on fast-Fourier transforms and codes, and discuss applications in which JL transformations have been implemented
Isometric sketching of any set via the Restricted Isometry Property
In this paper we show that for the purposes of dimensionality reduction
certain class of structured random matrices behave similarly to random Gaussian
matrices. This class includes several matrices for which matrix-vector multiply
can be computed in log-linear time, providing efficient dimensionality
reduction of general sets. In particular, we show that using such matrices any
set from high dimensions can be embedded into lower dimensions with near
optimal distortion. We obtain our results by connecting dimensionality
reduction of any set to dimensionality reduction of sparse vectors via a
chaining argument.Comment: 17 page
A Sparse Johnson--Lindenstrauss Transform
Dimension reduction is a key algorithmic tool with many applications
including nearest-neighbor search, compressed sensing and linear algebra in the
streaming model. In this work we obtain a {\em sparse} version of the
fundamental tool in dimension reduction --- the Johnson--Lindenstrauss
transform. Using hashing and local densification, we construct a sparse
projection matrix with just non-zero entries
per column. We also show a matching lower bound on the sparsity for a large
class of projection matrices. Our bounds are somewhat surprising, given the
known lower bounds of both on the number of rows
of any projection matrix and on the sparsity of projection matrices generated
by natural constructions.
Using this, we achieve an update time per
non-zero element for a -approximate projection, thereby
substantially outperforming the update time
required by prior approaches. A variant of our method offers the same
guarantees for sparse vectors, yet its worst case running time
matches the best approach of Ailon and Liberty.Comment: 10 pages, conference version
- …