24,462 research outputs found
Efficiently Extracting Randomness from Imperfect Stochastic Processes
We study the problem of extracting a prescribed number of random bits by
reading the smallest possible number of symbols from non-ideal stochastic
processes. The related interval algorithm proposed by Han and Hoshi has
asymptotically optimal performance; however, it assumes that the distribution
of the input stochastic process is known. The motivation for our work is the
fact that, in practice, sources of randomness have inherent correlations and
are affected by measurement's noise. Namely, it is hard to obtain an accurate
estimation of the distribution. This challenge was addressed by the concepts of
seeded and seedless extractors that can handle general random sources with
unknown distributions. However, known seeded and seedless extractors provide
extraction efficiencies that are substantially smaller than Shannon's entropy
limit. Our main contribution is the design of extractors that have a variable
input-length and a fixed output length, are efficient in the consumption of
symbols from the source, are capable of generating random bits from general
stochastic processes and approach the information theoretic upper bound on
efficiency.Comment: 2 columns, 16 page
Universal lossless source coding with the Burrows Wheeler transform
The Burrows Wheeler transform (1994) is a reversible sequence transformation used in a variety of practical lossless source-coding algorithms. In each, the BWT is followed by a lossless source code that attempts to exploit the natural ordering of the BWT coefficients. BWT-based compression schemes are widely touted as low-complexity algorithms giving lossless coding rates better than those of the Ziv-Lempel codes (commonly known as LZ'77 and LZ'78) and almost as good as those achieved by prediction by partial matching (PPM) algorithms. To date, the coding performance claims have been made primarily on the basis of experimental results. This work gives a theoretical evaluation of BWT-based coding. The main results of this theoretical evaluation include: (1) statistical characterizations of the BWT output on both finite strings and sequences of length n → ∞, (2) a variety of very simple new techniques for BWT-based lossless source coding, and (3) proofs of the universality and bounds on the rates of convergence of both new and existing BWT-based codes for finite-memory and stationary ergodic sources. The end result is a theoretical justification and validation of the experimentally derived conclusions: BWT-based lossless source codes achieve universal lossless coding performance that converges to the optimal coding performance more quickly than the rate of convergence observed in Ziv-Lempel style codes and, for some BWT-based codes, within a constant factor of the optimal rate of convergence for finite-memory source
Compression-Based Compressed Sensing
Modern compression algorithms exploit complex structures that are present in
signals to describe them very efficiently. On the other hand, the field of
compressed sensing is built upon the observation that "structured" signals can
be recovered from their under-determined set of linear projections. Currently,
there is a large gap between the complexity of the structures studied in the
area of compressed sensing and those employed by the state-of-the-art
compression codes. Recent results in the literature on deterministic signals
aim at bridging this gap through devising compressed sensing decoders that
employ compression codes. This paper focuses on structured stochastic processes
and studies the application of rate-distortion codes to compressed sensing of
such signals. The performance of the formerly-proposed compressible signal
pursuit (CSP) algorithm is studied in this stochastic setting. It is proved
that in the very low distortion regime, as the blocklength grows to infinity,
the CSP algorithm reliably and robustly recovers instances of a stationary
process from random linear projections as long as their count is slightly more
than times the rate-distortion dimension (RDD) of the source. It is also
shown that under some regularity conditions, the RDD of a stationary process is
equal to its information dimension (ID). This connection establishes the
optimality of the CSP algorithm at least for memoryless stationary sources, for
which the fundamental limits are known. Finally, it is shown that the CSP
algorithm combined by a family of universal variable-length fixed-distortion
compression codes yields a family of universal compressed sensing recovery
algorithms
Network vector quantization
We present an algorithm for designing locally optimal vector quantizers for general networks. We discuss the algorithm's implementation and compare the performance of the resulting "network vector quantizers" to traditional vector quantizers (VQs) and to rate-distortion (R-D) bounds where available. While some special cases of network codes (e.g., multiresolution (MR) and multiple description (MD) codes) have been studied in the literature, we here present a unifying approach that both includes these existing solutions as special cases and provides solutions to previously unsolved examples
- …