1,172 research outputs found
A Sparse Johnson--Lindenstrauss Transform
Dimension reduction is a key algorithmic tool with many applications
including nearest-neighbor search, compressed sensing and linear algebra in the
streaming model. In this work we obtain a {\em sparse} version of the
fundamental tool in dimension reduction --- the Johnson--Lindenstrauss
transform. Using hashing and local densification, we construct a sparse
projection matrix with just non-zero entries
per column. We also show a matching lower bound on the sparsity for a large
class of projection matrices. Our bounds are somewhat surprising, given the
known lower bounds of both on the number of rows
of any projection matrix and on the sparsity of projection matrices generated
by natural constructions.
Using this, we achieve an update time per
non-zero element for a -approximate projection, thereby
substantially outperforming the update time
required by prior approaches. A variant of our method offers the same
guarantees for sparse vectors, yet its worst case running time
matches the best approach of Ailon and Liberty.Comment: 10 pages, conference version
Toward a unified theory of sparse dimensionality reduction in Euclidean space
Let be a sparse Johnson-Lindenstrauss
transform [KN14] with non-zeroes per column. For a subset of the unit
sphere, given, we study settings for required to
ensure i.e. so that preserves the norm of every
simultaneously and multiplicatively up to . We
introduce a new complexity parameter, which depends on the geometry of , and
show that it suffices to choose and such that this parameter is small.
Our result is a sparse analog of Gordon's theorem, which was concerned with a
dense having i.i.d. Gaussian entries. We qualitatively unify several
results related to the Johnson-Lindenstrauss lemma, subspace embeddings, and
Fourier-based restricted isometries. Our work also implies new results in using
the sparse Johnson-Lindenstrauss transform in numerical linear algebra,
classical and model-based compressed sensing, manifold learning, and
constrained least squares problems such as the Lasso
Recommended from our members
A Derandomized Sparse Johnson-Lindenstrauss Transform
Recent work of [Dasgupta-Kumar-Sarl´os, STOC 2010] gave a sparse Johnson-Lindenstrauss transform and left as a main open question whether their construction could be efficiently derandomized. We answer their question affirmatively by giving an alternative proof of their result requiring only bounded independence hash functions. Furthermore, the sparsity bound obtained in our proof is improved. Our work implies the first implementation of a JohnsonLindenstrauss transform in data streams with sublinear update time.Engineering and Applied Science
Recommended from our members
Toward a unified theory of sparse dimensionality reduction in Euclidean space
Let be a sparse Johnson-Lindenstrauss transform [KN14] with s non-zeroes per column. For a subset T of the unit sphere, given, we study settings for m,s required to ensure
, i.e.\ so that preserves the norm of every simultaneously and multiplicatively up to . We introduce a new complexity parameter, which depends on the geometry of T, and show that it suffices to choose s and m such that this parameter is small. Our result is a sparse analog of Gordon's theorem, which was concerned with a dense having i.i.d. gaussian entries. We qualitatively unify several results related to the Johnson-Lindenstrauss lemma, subspace embeddings, and Fourier-based restricted isometries. Our work also implies new results in using the sparse Johnson-Lindenstrauss transform in numerical linear algebra, classical and model-based compressed sensing, manifold learning, and constrained least squares problems such as the Lasso.Engineering and Applied Science
Almost Optimal Unrestricted Fast Johnson-Lindenstrauss Transform
The problems of random projections and sparse reconstruction have much in
common and individually received much attention. Surprisingly, until now they
progressed in parallel and remained mostly separate. Here, we employ new tools
from probability in Banach spaces that were successfully used in the context of
sparse reconstruction to advance on an open problem in random pojection. In
particular, we generalize and use an intricate result by Rudelson and Vershynin
for sparse reconstruction which uses Dudley's theorem for bounding Gaussian
processes. Our main result states that any set of real
vectors in dimensional space can be linearly mapped to a space of dimension
k=O(\log N\polylog(n)), while (1) preserving the pairwise distances among the
vectors to within any constant distortion and (2) being able to apply the
transformation in time on each vector. This improves on the best
known achieved by Ailon and Liberty and by Ailon and Chazelle.
The dependence in the distortion constant however is believed to be
suboptimal and subject to further investigation. For constant distortion, this
settles the open question posed by these authors up to a \polylog(n) factor
while considerably simplifying their constructions
- …