We give two different and simple constructions for dimensionality reduction
in $\ell_2$ via linear mappings that are sparse: only an
$O(\varepsilon)$-fraction of entries in each column of our embedding matrices
are non-zero to achieve distortion $1+\varepsilon$ with high probability, while
still achieving the asymptotically optimal number of rows. These are the first
constructions to provide subconstant sparsity for all values of parameters,
improving upon previous works of Achlioptas (JCSS 2003) and Dasgupta, Kumar,
and Sarl\'{o}s (STOC 2010). Such distributions can be used to speed up
applications where $\ell_2$ dimensionality reduction is used.Comment: v6: journal version, minor changes, added Remark 23; v5: modified
  abstract, fixed typos, added open problem section; v4: simplified section 4
  by giving 1 analysis that covers both constructions; v3: proof of Theorem 25
  in v2 was written incorrectly, now fixed; v2: Added another construction
  achieving same upper bound, and added proof of near-tight lower bound for DKS
  schem

Kane, Daniel M.

Nelson, Jelani

English

arXiv

We give two different Johnson-Lindenstrauss distributions, each with column sparsity $s = \Theta(\epsilon^{−1} log(1/\delta))$ and embedding into optimal dimension $k = O(\epsilon^{−2} log(1/\delta))$ to achieve distortion $1\pm \epsilon$ with probability $1−\delta$. That is, only an $O(\epsilon)$-fraction of entries are non-zero in each embedding matrix in the supports of our distributions. These are the first distributions to provide o(k) sparsity for all values of $\epsilon$, $\delta$. Previously the best known construction obtained $s = \overset \sim \Theta (\epsilon^{-1} log^2(1/\delta))^1$ [Dasgupta-Kumar-Sarlós, STOC 2010]. In addition, one of our distributions can be sampled from a seed of $O(log(1/\delta) log d)$ uniform random bits. Some applications that use Johnson-Lindenstrauss embeddings as a black box, such as those in approximate numerical linear algebra ([Sarlós, FOCS 2006], [Clarkson-Woodruff, STOC 2009]), require exponentially small $\delta$. Our linear dependence on $log(1/\delta)$ in the sparsity is thus crucial in these applications to obtain speedup.Engineering and Applied Science

Harvard University - DASH 

Sparser Johnson-Lindenstrauss Transforms

http://dash.harvard.edu/bitstream/handle/1/13820495/riphash.pdf?sequence=1

Sparser Johnson-Lindenstrauss Transforms

Abstract

Similar works

Full text

Available Versions

Harvard University - DASH