Distributed Functional Scalar Quantization Simplified

Distributed functional scalar quantization (DFSQ) theory provides optimality conditions and predicts performance of data acquisition systems in which a computation on acquired data is desired. We address two limitations of previous works: prohibitively expensive decoder design and a restriction to sources with bounded distributions. We rigorously show that a much simpler decoder has equivalent asymptotic performance as the conditional expectation estimator previously explored, thus reducing decoder design complexity. The simpler decoder has the feature of decoupled communication and computation blocks. Moreover, we extend the DFSQ framework with the simpler decoder to acquire sources with infinite-support distributions such as Gaussian or exponential distributions. Finally, through simulation results we demonstrate that performance at moderate coding rates is well predicted by the asymptotic analysis, and we give new insight on the rate of convergence

Chatting in distributed quantization networks

Abstract—Several key results in source coding offer the intuition that distributed encoding via vector-quantize-and-bin is only slightly suboptimal to joint encoding and oftentimes is just as good. However, when source acquisition requires the blocklength to be small, collaboration between sensors can greatly reduce distortion. For a distributed acquisition network where sensors are allowed to “chat ” using a side channel, we provide exact characterization of distortion performance and quantizer design in the high-resolution (low-distortion) regime using a framework called distributed functional scalar quantization (DFSQ). The key result is that chatting can dramatically improve performance even when the intersensor communication is at very low rate. We also solve the rate allo-cation problem when communication links have heterogeneous costs and provide examples to demonstrate that this theory predicts performance at practical communication rates. I

Distributed Scalar Quantization for Computing: High-Resolution Analysis and Extensions

Communication of quantized information is frequently followed by a computation. We consider situations of \emph{distributed functional scalar quantization}: distributed scalar quantization of (possibly correlated) sources followed by centralized computation of a function. Under smoothness conditions on the sources and function, companding scalar quantizer designs are developed to minimize mean-squared error (MSE) of the computed function as the quantizer resolution is allowed to grow. Striking improvements over quantizers designed without consideration of the function are possible and are larger in the entropy-constrained setting than in the fixed-rate setting. As extensions to the basic analysis, we characterize a large class of functions for which regular quantization suffices, consider certain functions for which asymptotic optimality is achieved without arbitrarily fine quantization, and allow limited collaboration between source encoders. In the entropy-constrained setting, a single bit per sample communicated between encoders can have an arbitrarily-large effect on functional distortion. In contrast, such communication has very little effect in the fixed-rate setting.Comment: 36 pages, 10 figure

Quantization in acquisition and computation networks

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 151-165).In modern systems, it is often desirable to extract relevant information from large amounts of data collected at different spatial locations. Applications include sensor networks, wearable health-monitoring devices and a variety of other systems for inference. Several existing source coding techniques, such as Slepian-Wolf and Wyner-Ziv coding, achieve asymptotic compression optimality in distributed systems. However, these techniques are rarely used in sensor networks because of decoding complexity and prohibitively long code length. Moreover, the fundamental limits that arise from existing techniques are intractable to describe for a complicated network topology or when the objective of the system is to perform some computation on the data rather than to reproduce the data. This thesis bridges the technological gap between the needs of real-world systems and the optimistic bounds derived from asymptotic analysis. Specifically, we characterize fundamental trade-offs when the desired computation is incorporated into the compression design and the code length is one. To obtain both performance guarantees and achievable schemes, we use high-resolution quantization theory, which is complementary to the Shannon-theoretic analyses previously used to study distributed systems. We account for varied network topologies, such as those where sensors are allowed to collaborate or the communication links are heterogeneous. In these settings, a small amount of intersensor communication can provide a significant improvement in compression performance. As a result, this work suggests new compression principles and network design for modern distributed systems. Although the ideas in the thesis are motivated by current and future sensor network implementations, the framework applies to a wide range of signal processing questions. We draw connections between the fidelity criteria studied in the thesis and distortion measures used in perceptual coding. As a consequence, we determine the optimal quantizer for expected relative error (ERE), a measure that is widely useful but is often neglected in the source coding community. We further demonstrate that applying the ERE criterion to psychophysical models can explain the Weber-Fechner law, a longstanding hypothesis of how humans perceive the external world. Our results are consistent with the hypothesis that human perception is Bayesian optimal for information acquisition conditioned on limited cognitive resources, thereby supporting the notion that the brain is efficient at acquisition and adaptation.by John Z. Sun.Ph.D

Robust Techniques for Signal Processing: A Survey

Coordinated Science Laboratory was formerly known as Control Systems LaboratoryU.S. Army Research Office / DAAG29-81-K-0062U.S. Air Force Office of Scientific Research / AFOSR 82-0022Joint Services Electronics Program / N00014-84-C-0149National Science Foundation / ECS-82-12080U.S. Office of Naval Research / N00014-80-K-0945 and N00014-81-K-001

Studies on the Asymptotic Behavior of Parameters in Optimal Scalar Quantization.

The goal in digital device design is to achieve high performance at low cost, and to pursue this goal, all components of the device must be designed accordingly. A principal component common in digital devices is the quantizer, and frequently used is the minimum mean-squared error (MSE) or emph{optimal}, fixed-rate scalar quantizer. In this thesis, we focus on aids to the design of such quantizers. For an exponential source with variance $sigma^2$, we estimate the largest finite quantization threshold by providing upper and lower bounds which are functions of the number of quantization levels $N$. The upper bound is $3sigmalog N$, $Ngeq1$, and the lower bound is $3sigmalog N + o_Nleft(1right)sigma-1.46004,sigma$, $N>9$. Using these bounds, we derive an upper bound to the convergence rate of $N^2Dleft(Nright)$ to the Panter-Dite constant, where $Dleft(Nright)$ is the least MSE of any $N$-level scalar quantizer. Furthermore, we present two, very simple, non-iterative and non-recursive suboptimal quantizer design methods for exponential sources that produce quantizers with good MSE performance. For an improved understanding of the half steps and quantization thresholds in optimal quantizers as functions of $N$, we use as inspiration the result by Nitadori~cite{Nitadori1965} where, exploiting a key side effect of the source's memoryless property, he derived an infinite sequence such that for any $N$, the $k$th term of the sequence is equal to the $k$th half step (counting from the right) of the optimal $N$-level quantizer designed for a unit variance exponential source. In our work, using an asymptotic version of this key side effect which holds for general exponential (GE) sources parameterized by an exponential power $p$ and a utilizing a method of our own devising, we show that for such a source, the $k$th half step of an optimal $N$-level quantizer multiplied by the ($p-1$)st power of the $k$th threshold approaches the $k$th term of the Nitadori sequence as $N$ grows to infinity. Thus, the Nitadori sequence asymptotically characterizes the cells of MMSE quantizers for GE-sources, as well as exponential.Ph.D.Electrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/76011/1/vbyee_1.pd