Lower Bounds for Sparse Recovery

We consider the following k-sparse recovery problem: design an m x n matrix A, such that for any signal x, given Ax we can efficiently recover x' satisfying ||x-x'||_1 <= C min_{k-sparse} x"} ||x-x"||_1. It is known that there exist matrices A with this property that have only O(k log (n/k)) rows. In this paper we show that this bound is tight. Our bound holds even for the more general /randomized/ version of the problem, where A is a random variable and the recovery algorithm is required to work for any fixed x with constant probability (over A).Comment: 11 pages. Appeared at SODA 201

Error correction via linear programming

Suppose we wish to transmit a vector f Є R^n reliably. A frequently discussed approach consists in encoding f with an m by n coding matrix A. Assume now that a fraction of the entries of Af are corrupted in a completely arbitrary fashion. We do not know which entries are affected nor do we know how they are affected. Is it possible to recover f exactly from the corrupted m-dimensional vector y? This paper proves that under suitable conditions on the coding matrix A, the input f is the unique solution to the ℓ_1 -minimization problem (‖x‖ℓ_1: = ∑_i |xi|) min ‖y − Ag‖ℓ_1 g^∈Rn provided that the fraction of corrupted entries is not too large, i.e. does not exceed some strictly positive constant ρ ∗ (numerical values for ρ ^∗ are given). In other words, f can be recovered exactly by solving a simple convex optimization problem; in fact, a linear program. We report on numerical experiments suggesting that ℓ_1-minimization is amazingly effective; f is recovered exactly even in situations where a very significant fraction of the output is corrupted. In the case when the measurement matrix A is Gaussian, the problem is equivalent to that of counting lowdimensional facets of a convex polytope, and in particular of a random section of the unit cube. In this case we can strengthen the results somewhat by using a geometric functional analysis approach

Reticulados em problemas de comunicação

Orientadores: Sueli Irene Rodrigues Costa, Vinay Anant VaishampayanTese (doutorado) - Universidade Estadual de Campinas, Instituto de Matemática Estatística e Computação CientíficaResumo: O estudo de códigos no contexto de reticulados e outras constelações discretas para aplicações em comunicações é um tópico de interesse na área de teoria da informação. Certas construções de reticulados, como é o caso das Construções A e D, e de outras constelações que não são reticulados, como a Construção C, são utilizadas na decodificação multi-estágio e para quantização vetorial eficiente. Isso motiva a primeira contribuição deste trabalho, que consiste em investigar características da Construção C e propor uma nova construção baseada em códigos lineares, que chamamos de Construção $C^\star,$ analisando suas propriedades (condições para ser reticulado, uniformidade geométrica e distância mínima) e relação com a Construção C. Problemas na área de comunicações envolvendo reticulados podem ser computacionalmente difíceis à medida que a dimensão aumenta, como é o caso de, dado um vetor no espaço real $n-$dimensional, determinar o ponto do reticulado mais próximo a este. A segunda contribuição deste trabalho é a análise desse problema restrito a um sistema distribuído, ou seja, onde o vetor a ser decodificado possui cada uma de suas coordenadas disponíveis em um nó distinto desse sistema. Nessa investigação, encontramos uma solução aproximada para duas e três dimensões considerando a partição de Babai e também estudamos o custo de comunicação envolvidoAbstract: The study of codes in the context of lattices and other discrete constellations for applications in communications is a topic of interest in the area of information theory. Some lattice constructions, such as the known Constructions A and D, and other special nonlattice constellations, as Construction C, are used in multi-stage decoding and efficient vector quantization. This motivates the first contribution of this work, which is to investigate characteristics of Construction C and to propose a new construction based on linear codes that we called Construction $C^\star,$ analyzing its properties (latticeness, geometric uniformity and minimum distance) and relations with Construction C. Communication problems related to lattices can be computationally hard when the dimension increases, as it is the case of, given a real vector in the $n-$dimensional space, determine the closest lattice point to it. The second contribution of this work is the analysis of this problem restricted to a distributed system, i.e., where the vector to be decoded has each coordinate available in a separated node in this system. In this investigation, we find the approximate solution for two and three dimensions considering the Babai partition and study the communication cost involvedDoutoradoMatematica AplicadaDoutora em Matemática Aplicada140797/2017-3CNPQCAPE

On Deterministic Sketching and Streaming for Sparse Recovery and Norm Estimation

We study classic streaming and sparse recovery problems using deterministic linear sketches, including l1/l1 and linf/l1 sparse recovery problems (the latter also being known as l1-heavy hitters), norm estimation, and approximate inner product. We focus on devising a fixed matrix A in R^{m x n} and a deterministic recovery/estimation procedure which work for all possible input vectors simultaneously. Our results improve upon existing work, the following being our main contributions: * A proof that linf/l1 sparse recovery and inner product estimation are equivalent, and that incoherent matrices can be used to solve both problems. Our upper bound for the number of measurements is m=O(eps^{-2}*min{log n, (log n / log(1/eps))^2}). We can also obtain fast sketching and recovery algorithms by making use of the Fast Johnson-Lindenstrauss transform. Both our running times and number of measurements improve upon previous work. We can also obtain better error guarantees than previous work in terms of a smaller tail of the input vector. * A new lower bound for the number of linear measurements required to solve l1/l1 sparse recovery. We show Omega(k/eps^2 + klog(n/k)/eps) measurements are required to recover an x' with |x - x'|_1 <= (1+eps)|x_{tail(k)}|_1, where x_{tail(k)} is x projected onto all but its largest k coordinates in magnitude. * A tight bound of m = Theta(eps^{-2}log(eps^2 n)) on the number of measurements required to solve deterministic norm estimation, i.e., to recover |x|_2 +/- eps|x|_1. For all the problems we study, tight bounds are already known for the randomized complexity from previous work, except in the case of l1/l1 sparse recovery, where a nearly tight bound is known. Our work thus aims to study the deterministic complexities of these problems

Information- and Coding-Theoretic Analysis of the RLWE Channel

Several cryptosystems based on the \emph{Ring Learning with Errors} (RLWE) problem have been proposed within the NIST post-quantum cryptography standardization process, e.g. NewHope. Furthermore, there are systems like Kyber which are based on the closely related MLWE assumption. Both previously mentioned schemes feature a non-zero decryption failure rate (DFR). The combination of encryption and decryption for these kinds of algorithms can be interpreted as data transmission over noisy channels. To the best of our knowledge this paper is the first work that analyzes the capacity of this channel. We show how to modify the encryption schemes such that the input alphabets of the corresponding channels are increased. In particular, we present lower bounds on their capacities which show that the transmission rate can be significantly increased compared to standard proposals in the literature. Furthermore, under the common assumption of stochastically independent coefficient failures, we give lower bounds on achievable rates based on both the Gilbert-Varshamov bound and concrete code constructions using BCH codes. By means of our constructions, we can either increase the total bitrate (by a factor of $1.84$ for Kyber and by factor of $7$ for NewHope) while guaranteeing the same \emph{decryption failure rate} (DFR). Moreover, for the same bitrate, we can significantly reduce the DFR for all schemes considered in this work (e.g., for NewHope from $2^{-216}$ to $2^{-12769}$).Comment: 13 pages, 4 figures, 3 table

Multiple Packing: Lower and Upper Bounds

We study the problem of high-dimensional multiple packing in Euclidean space. Multiple packing is a natural generalization of sphere packing and is defined as follows. Let $N>0$ and $L\in\mathbb{Z}_{\ge2}$. A multiple packing is a set $\mathcal{C}$ of points in $\mathbb{R}^n$ such that any point in $\mathbb{R}^n$ lies in the intersection of at most $L-1$ balls of radius $\sqrt{nN}$ around points in $\mathcal{C}$. We study the multiple packing problem for both bounded point sets whose points have norm at most $\sqrt{nP}$ for some constant $P>0$ and unbounded point sets whose points are allowed to be anywhere in $\mathbb{R}^n$. Given a well-known connection with coding theory, multiple packings can be viewed as the Euclidean analog of list-decodable codes, which are well-studied for finite fields. In this paper, we derive various bounds on the largest possible density of a multiple packing in both bounded and unbounded settings. A related notion called average-radius multiple packing is also studied. Some of our lower bounds exactly pin down the asymptotics of certain ensembles of average-radius list-decodable codes, e.g., (expurgated) Gaussian codes and (expurgated) spherical codes. In particular, our lower bound obtained from spherical codes is the best known lower bound on the optimal multiple packing density and is the first lower bound that approaches the known large $L$ limit under the average-radius notion of multiple packing. To derive these results, we apply tools from high-dimensional geometry and large deviation theory.Comment: The paper arXiv:2107.05161 has been split into three parts with new results added and significant revision. This paper is one of the three parts. The other two are arXiv:2211.04408 and arXiv:2211.0440

Combinatorial Methods in Coding Theory

This thesis is devoted to a range of questions in applied mathematics and signal processing motivated by applications in error correction, compressed sensing, and writing on non-volatile memories. The underlying thread of our results is the use of diverse combinatorial methods originating in coding theory and computer science. The thesis addresses three groups of problems. The first of them is aimed at the construction and analysis of codes for error correction. Here we examine properties of codes that are constructed using random and structured graphs and hypergraphs, with the main purpose of devising new decoding algorithms as well as estimating the distribution of Hamming weights in the resulting codes. Some of the results obtained give the best known estimates of the number of correctable errors for codes whose decoding relies on local operations on the graph. In the second part we address the question of constructing sampling operators for the compressed sensing problem. This topic has been the subject of a large body of works in the literature. We propose general constructions of sampling matrices based on ideas from coding theory that act as near-isometric maps on almost all sparse signal. This matrices can be used for dimensionality reduction and compressed sensing. In the third part we study the problem of reliable storage of information in non-volatile memories such as flash drives. This problem gives rise to a writing scheme that relies on relative magnitudes of neighboring cells, known as rank modulation. We establish the exact asymptotic behavior of the size of codes for rank modulation and suggest a number of new general constructions of such codes based on properties of finite fields as well as combinatorial considerations