Quantization and erasures in frame representations

Thesis (Sc. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (p. 123-126).Frame representations, which correspond to overcomplete generalizations to basis expansions, are often used in signal processing to provide robustness to errors. In this thesis robustness is provided through the use of projections to compensate for errors in the representation coefficients, with specific focus on quantization and erasure errors. The projections are implemented by modifying the unaffected coefficients using an additive term, which is linear in the error. This low-complexity implementation only assumes linear reconstruction using a pre-determined synthesis frame, and makes no assumption on how the representation coefficients are generated. In the context of quantization, the limits of scalar quantization of frame representations are first examined, assuming the analysis is using inner products with the frame vectors. Bounds on the error and the bit-efficiency are derived, demonstrating that scalar quantization of the coefficients is suboptimal. As an alternative to scalar quantization, a generalization of Sigma-Delta noise shaping to arbitrary frame representations is developed by reformulating noise shaping as a sequence of compensations for the quantization error using projections.(cont.) The total error is quantified using both the additive noise model of quantization, and a deterministic upper bound based on the triangle inequality. It is thus shown that the average and the worst-case error is reduced compared to scalar quantization of the coefficients. The projection principle is also used to provide robustness to erasures. Specifically, the case of a transmitter that is aware of the erasure occurrence is considered, which compensates for the erasure error by projecting it to the subsequent frame vectors. It is further demonstrated that the transmitter can be split to a transmitter/receiver combination that performs the same compensation, but in which only the receiver is aware of the erasure occurrence. Furthermore, an algorithm to puncture dense representations in order to produce sparse approximate ones is introduced. In this algorithm the error due to the puncturing is also projected to the span of the remaining coefficients. The algorithm can be combined with quantization to produce quantized sparse representations approximating the original dense representation.by Petros T. Boufounos.Sc.D

Dual frames compensating for erasures

We discuss the problem of recovering signal from frame coefficients with erasures. Such problems arise naturally from applications where some of the coefficients could be corrupted or erased during the data transmission. Provided that the erasure set satisfies the minimal redundancy condition, we construct a suitable synthesizing dual frame which enables us to perfectly reconstruct the original signal without recovering the lost coefficients. Such dual frames which compensate for erasures are described from various viewpoints

Multiple-Description Coding by Dithered Delta-Sigma Quantization

We address the connection between the multiple-description (MD) problem and Delta-Sigma quantization. The inherent redundancy due to oversampling in Delta-Sigma quantization, and the simple linear-additive noise model resulting from dithered lattice quantization, allow us to construct a symmetric and time-invariant MD coding scheme. We show that the use of a noise shaping filter makes it possible to trade off central distortion for side distortion. Asymptotically as the dimension of the lattice vector quantizer and order of the noise shaping filter approach infinity, the entropy rate of the dithered Delta-Sigma quantization scheme approaches the symmetric two-channel MD rate-distortion function for a memoryless Gaussian source and MSE fidelity criterion, at any side-to-central distortion ratio and any resolution. In the optimal scheme, the infinite-order noise shaping filter must be minimum phase and have a piece-wise flat power spectrum with a single jump discontinuity. An important advantage of the proposed design is that it is symmetric in rate and distortion by construction, so the coding rates of the descriptions are identical and there is therefore no need for source splitting.Comment: Revised, restructured, significantly shortened and minor typos has been fixed. Accepted for publication in the IEEE Transactions on Information Theor

Quantization and Compressive Sensing

Quantization is an essential step in digitizing signals, and, therefore, an indispensable component of any modern acquisition system. This book chapter explores the interaction of quantization and compressive sensing and examines practical quantization strategies for compressive acquisition systems. Specifically, we first provide a brief overview of quantization and examine fundamental performance bounds applicable to any quantization approach. Next, we consider several forms of scalar quantizers, namely uniform, non-uniform, and 1-bit. We provide performance bounds and fundamental analysis, as well as practical quantizer designs and reconstruction algorithms that account for quantization. Furthermore, we provide an overview of Sigma-Delta ($\Sigma\Delta$) quantization in the compressed sensing context, and also discuss implementation issues, recovery algorithms and performance bounds. As we demonstrate, proper accounting for quantization and careful quantizer design has significant impact in the performance of a compressive acquisition system.Comment: 35 pages, 20 figures, to appear in Springer book "Compressed Sensing and Its Applications", 201

Signal Reconstruction from Frame and Sampling Erasures

This dissertation is concerned with the efficient reconstruction of signals from frame coefficient erasures at known locations. Three methods of perfect reconstruction from frame coefficient erasures will be discussed. These reconstructions are more efficient than older methods in the literature because they only require an L × L matrix inversion, where L denotes the cardinality of the erased set of indices. This is a significant improvement over older methods which require an n × n matrix inversion, where n denotes the dimension of the underlying Hilbert space. The first of these methods, called Nilpotent Bridging, uses a small subset of the non-erased coefficients to reconstruct the erased coefficients. This subset is called the bridge set. To perform the reconstruction an equation, known as the bridge equation, must be solved. A proof is given that under a very mild assumption there exists a bridge set of size L for which the bridge equation has a solution. A stronger result is also proven that shows that for a very large class of frames, no bridge set search is required. We call this set of frames the set of full skew-spark frames. Using the Baire Category Theorem and tools from Matrix Theory, the set of full skew-spark frames is shown to be an open, dense subset of the set of all frames in finite dimensions. The second method of reconstruction is called Reduced Direct Inversion because it provides a basis-free, closed-form formula for inverting a particular n × n matrix, which only requires the inversion of an L × L matrix. By inverting this matrix we obtain another efficient reconstruction formula. The final method considered is a continuation of work by Han and Sun. The method utilizes an Erasure Recovery Matrix, which is a matrix that annihilates the range of the analysis operator for a frame. Because of this, the erased coefficients can be reconstructed using a simple pseudo-inverse technique. For each method, a discussion of the stability of our algorithms is presented. In particular, we present numerical experiments to investigate the effects of normally distributed additive channel noise on our reconstruction. For Reduced Direct Inversion and Erasure Recovery Matrices, using the Restricted Isometry Property, we construct classes of frames which are numerically robust to sparse channel noise. For these frames, we provide error bounds for sparse channel noise

Frame Theory for Signal Processing in Psychoacoustics

This review chapter aims to strengthen the link between frame theory and signal processing tasks in psychoacoustics. On the one side, the basic concepts of frame theory are presented and some proofs are provided to explain those concepts in some detail. The goal is to reveal to hearing scientists how this mathematical theory could be relevant for their research. In particular, we focus on frame theory in a filter bank approach, which is probably the most relevant view-point for audio signal processing. On the other side, basic psychoacoustic concepts are presented to stimulate mathematicians to apply their knowledge in this field