Impossibility of dimension reduction in the nuclear norm

Let $\mathsf{S}_1$ (the Schatten--von Neumann trace class) denote the Banach space of all compact linear operators $T:\ell_2\to \ell_2$ whose nuclear norm $\|T\|_{\mathsf{S}_1}=\sum_{j=1}^\infty\sigma_j(T)$ is finite, where $\{\sigma_j(T)\}_{j=1}^\infty$ are the singular values of $T$. We prove that for arbitrarily large $n\in \mathbb{N}$ there exists a subset $\mathcal{C}\subseteq \mathsf{S}_1$ with $|\mathcal{C}|=n$ that cannot be embedded with bi-Lipschitz distortion $O(1)$ into any $n^{o(1)}$-dimensional linear subspace of $\mathsf{S}_1$. $\mathcal{C}$ is not even a $O(1)$-Lipschitz quotient of any subset of any $n^{o(1)}$-dimensional linear subspace of $\mathsf{S}_1$. Thus, $\mathsf{S}_1$ does not admit a dimension reduction result \'a la Johnson and Lindenstrauss (1984), which complements the work of Harrow, Montanaro and Short (2011) on the limitations of quantum dimension reduction under the assumption that the embedding into low dimensions is a quantum channel. Such a statement was previously known with $\mathsf{S}_1$ replaced by the Banach space $\ell_1$ of absolutely summable sequences via the work of Brinkman and Charikar (2003). In fact, the above set $\mathcal{C}$ can be taken to be the same set as the one that Brinkman and Charikar considered, viewed as a collection of diagonal matrices in $\mathsf{S}_1$. The challenge is to demonstrate that $\mathcal{C}$ cannot be faithfully realized in an arbitrary low-dimensional subspace of $\mathsf{S}_1$, while Brinkman and Charikar obtained such an assertion only for subspaces of $\mathsf{S}_1$ that consist of diagonal operators (i.e., subspaces of $\ell_1$). We establish this by proving that the Markov 2-convexity constant of any finite dimensional linear subspace $X$ of $\mathsf{S}_1$ is at most a universal constant multiple of $\sqrt{\log \mathrm{dim}(X)}$

Low-distortion Subspace Embeddings in Input-sparsity Time and Applications to Robust Linear Regression

Low-distortion embeddings are critical building blocks for developing random sampling and random projection algorithms for linear algebra problems. We show that, given a matrix $A \in \R^{n \times d}$ with $n \gg d$ and a $p \in [1, 2)$, with a constant probability, we can construct a low-distortion embedding matrix \Pi \in \R^{O(\poly(d)) \times n} that embeds \A_p, the $\ell_p$ subspace spanned by $A$'s columns, into (\R^{O(\poly(d))}, \| \cdot \|_p); the distortion of our embeddings is only O(\poly(d)), and we can compute $\Pi A$ in O(\nnz(A)) time, i.e., input-sparsity time. Our result generalizes the input-sparsity time $\ell_2$ subspace embedding by Clarkson and Woodruff [STOC'13]; and for completeness, we present a simpler and improved analysis of their construction for $\ell_2$. These input-sparsity time $\ell_p$ embeddings are optimal, up to constants, in terms of their running time; and the improved running time propagates to applications such as $(1\pm \epsilon)$-distortion $\ell_p$ subspace embedding and relative-error $\ell_p$ regression. For $\ell_2$, we show that a $(1+\epsilon)$-approximate solution to the $\ell_2$ regression problem specified by the matrix $A$ and a vector $b \in \R^n$ can be computed in O(\nnz(A) + d^3 \log(d/\epsilon) /\epsilon^2) time; and for $\ell_p$, via a subspace-preserving sampling procedure, we show that a $(1\pm \epsilon)$-distortion embedding of \A_p into \R^{O(\poly(d))} can be computed in O(\nnz(A) \cdot \log n) time, and we also show that a $(1+\epsilon)$-approximate solution to the $\ell_p$ regression problem $\min_{x \in \R^d} \|A x - b\|_p$ can be computed in O(\nnz(A) \cdot \log n + \poly(d) \log(1/\epsilon)/\epsilon^2) time. Moreover, we can improve the embedding dimension or equivalently the sample size to $O(d^{3+p/2} \log(1/\epsilon) / \epsilon^2)$ without increasing the complexity.Comment: 22 page