On Resource-bounded versions of the van Lambalgen theorem

The van Lambalgen theorem is a surprising result in algorithmic information theory concerning the symmetry of relative randomness. It establishes that for any pair of infinite sequences $A$ and $B$, $B$ is Martin-L\"of random and $A$ is Martin-L\"of random relative to $B$ if and only if the interleaved sequence $A \uplus B$ is Martin-L\"of random. This implies that $A$ is relative random to $B$ if and only if $B$ is random relative to $A$ \cite{vanLambalgen}, \cite{Nies09}, \cite{HirschfeldtBook}. This paper studies the validity of this phenomenon for different notions of time-bounded relative randomness. We prove the classical van Lambalgen theorem using martingales and Kolmogorov compressibility. We establish the failure of relative randomness in these settings, for both time-bounded martingales and time-bounded Kolmogorov complexity. We adapt our classical proofs when applicable to the time-bounded setting, and construct counterexamples when they fail. The mode of failure of the theorem may depend on the notion of time-bounded randomness

On Maximum Contention-Free Interleavers and Permutation Polynomials over Integer Rings

An interleaver is a critical component for the channel coding performance of turbo codes. Algebraic constructions are of particular interest because they admit analytical designs and simple, practical hardware implementation. Contention-free interleavers have been recently shown to be suitable for parallel decoding of turbo codes. In this correspondence, it is shown that permutation polynomials generate maximum contention-free interleavers, i.e., every factor of the interleaver length becomes a possible degree of parallel processing of the decoder. Further, it is shown by computer simulations that turbo codes using these interleavers perform very well for the 3rd Generation Partnership Project (3GPP) standard.Comment: 13 pages, 2 figures, submitted as a correspondence to the IEEE Transactions on Information Theory, revised versio

Support Recovery of Sparse Signals

We consider the problem of exact support recovery of sparse signals via noisy measurements. The main focus is the sufficient and necessary conditions on the number of measurements for support recovery to be reliable. By drawing an analogy between the problem of support recovery and the problem of channel coding over the Gaussian multiple access channel, and exploiting mathematical tools developed for the latter problem, we obtain an information theoretic framework for analyzing the performance limits of support recovery. Sharp sufficient and necessary conditions on the number of measurements in terms of the signal sparsity level and the measurement noise level are derived. Specifically, when the number of nonzero entries is held fixed, the exact asymptotics on the number of measurements for support recovery is developed. When the number of nonzero entries increases in certain manners, we obtain sufficient conditions tighter than existing results. In addition, we show that the proposed methodology can deal with a variety of models of sparse signal recovery, hence demonstrating its potential as an effective analytical tool.Comment: 33 page

Frame Permutation Quantization

Frame permutation quantization (FPQ) is a new vector quantization technique using finite frames. In FPQ, a vector is encoded using a permutation source code to quantize its frame expansion. This means that the encoding is a partial ordering of the frame expansion coefficients. Compared to ordinary permutation source coding, FPQ produces a greater number of possible quantization rates and a higher maximum rate. Various representations for the partitions induced by FPQ are presented, and reconstruction algorithms based on linear programming, quadratic programming, and recursive orthogonal projection are derived. Implementations of the linear and quadratic programming algorithms for uniform and Gaussian sources show performance improvements over entropy-constrained scalar quantization for certain combinations of vector dimension and coding rate. Monte Carlo evaluation of the recursive algorithm shows that mean-squared error (MSE) decays as 1/M^4 for an M-element frame, which is consistent with previous results on optimal decay of MSE. Reconstruction using the canonical dual frame is also studied, and several results relate properties of the analysis frame to whether linear reconstruction techniques provide consistent reconstructions.Comment: 29 pages, 5 figures; detailed added to proof of Theorem 4.3 and a few minor correction

Second-Order Weight Distributions

A fundamental property of codes, the second-order weight distribution, is proposed to solve the problems such as computing second moments of weight distributions of linear code ensembles. A series of results, parallel to those for weight distributions, is established for second-order weight distributions. In particular, an analogue of MacWilliams identities is proved. The second-order weight distributions of regular LDPC code ensembles are then computed. As easy consequences, the second moments of weight distributions of regular LDPC code ensembles are obtained. Furthermore, the application of second-order weight distributions in random coding approach is discussed. The second-order weight distributions of the ensembles generated by a so-called 2-good random generator or parity-check matrix are computed, where a 2-good random matrix is a kind of generalization of the uniformly distributed random matrix over a finite filed and is very useful for solving problems that involve pairwise or triple-wise properties of sequences. It is shown that the 2-good property is reflected in the second-order weight distribution, which thus plays a fundamental role in some well-known problems in coding theory and combinatorics. An example of linear intersecting codes is finally provided to illustrate this fact.Comment: 10 pages, accepted for publication in IEEE Transactions on Information Theory, May 201