Quantization Design for Distributed Optimization

We consider the problem of solving a distributed optimization problem using a distributed computing platform, where the communication in the network is limited: each node can only communicate with its neighbours and the channel has a limited data-rate. A common technique to address the latter limitation is to apply quantization to the exchanged information. We propose two distributed optimization algorithms with an iteratively refining quantization design based on the inexact proximal gradient method and its accelerated variant. We show that if the parameters of the quantizers, i.e. the number of bits and the initial quantization intervals, satisfy certain conditions, then the quantization error is bounded by a linearly decreasing function and the convergence of the distributed algorithms is guaranteed. Furthermore, we prove that after imposing the quantization scheme, the distributed algorithms still exhibit a linear convergence rate, and show complexity upper-bounds on the number of iterations to achieve a given accuracy. Finally, we demonstrate the performance of the proposed algorithms and the theoretical findings for solving a distributed optimal control problem

Dynamic Quantized Consensus of General Linear Multi-agent Systems under Denial-of-Service Attacks

In this paper, we study multi-agent consensus problems under Denial-of-Service (DoS) attacks with data rate constraints. We first consider the leaderless consensus problem and after that we briefly present the analysis of leader-follower consensus. The dynamics of the agents take general forms modeled as homogeneous linear time-invariant systems. In our analysis, we derive lower bounds on the data rate for the multi-agent systems to achieve leaderless and leader-follower consensus in the presence of DoS attacks, under which the issue of overflow of quantizer is prevented. The main contribution of the paper is the characterization of the trade-off between the tolerable DoS attack levels for leaderless and leader-follower consensus and the required data rates for the quantizers during the communication attempts among the agents. To mitigate the influence of DoS attacks, we employ dynamic quantization with zooming-in and zooming-out capabilities for avoiding quantizer saturation

Optimal Identical Binary Quantizer Design for Distributed Estimation

We consider the design of identical one-bit probabilistic quantizers for distributed estimation in sensor networks. We assume the parameter-range to be finite and known and use the maximum Cram\'er-Rao Lower Bound (CRB) over the parameter-range as our performance metric. We restrict our theoretical analysis to the class of antisymmetric quantizers and determine a set of conditions for which the probabilistic quantizer function is greatly simplified. We identify a broad class of noise distributions, which includes Gaussian noise in the low-SNR regime, for which the often used threshold-quantizer is found to be minimax-optimal. Aided with theoretical results, we formulate an optimization problem to obtain the optimum minimax-CRB quantizer. For a wide range of noise distributions, we demonstrate the superior performance of the new quantizer - particularly in the moderate to high-SNR regime.Comment: 6 pages, 3 figures, This paper has been accepted for publication in IEEE Transactions in Signal Processin

On Distributed Linear Estimation With Observation Model Uncertainties

We consider distributed estimation of a Gaussian source in a heterogenous bandwidth constrained sensor network, where the source is corrupted by independent multiplicative and additive observation noises, with incomplete statistical knowledge of the multiplicative noise. For multi-bit quantizers, we derive the closed-form mean-square-error (MSE) expression for the linear minimum MSE (LMMSE) estimator at the FC. For both error-free and erroneous communication channels, we propose several rate allocation methods named as longest root to leaf path, greedy and integer relaxation to (i) minimize the MSE given a network bandwidth constraint, and (ii) minimize the required network bandwidth given a target MSE. We also derive the Bayesian Cramer-Rao lower bound (CRLB) and compare the MSE performance of our proposed methods against the CRLB. Our results corroborate that, for low power multiplicative observation noises and adequate network bandwidth, the gaps between the MSE of our proposed methods and the CRLB are negligible, while the performance of other methods like individual rate allocation and uniform is not satisfactory

Algorithms for Optimal Control with Fixed-Rate Feedback

We consider a discrete-time linear quadratic Gaussian networked control setting where the (full information) observer and controller are separated by a fixed-rate noiseless channel. The minimal rate required to stabilize such a system has been well studied. However, for a given fixed rate, how to quantize the states so as to optimize performance is an open question of great theoretical and practical significance. We concentrate on minimizing the control cost for first-order scalar systems. To that end, we use the Lloyd-Max algorithm and leverage properties of logarithmically-concave functions and sequential Bayesian filtering to construct the optimal quantizer that greedily minimizes the cost at every time instant. By connecting the globally optimal scheme to the problem of scalar successive refinement, we argue that its gain over the proposed greedy algorithm is negligible. This is significant since the globally optimal scheme is often computationally intractable. All the results are proven for the more general case of disturbances with logarithmically-concave distributions and rate-limited time-varying noiseless channels. We further extend the framework to event-triggered control by allowing to convey information via an additional "silent symbol", i.e., by avoiding transmitting bits; by constraining the minimal probability of silence we attain a tradeoff between the transmission rate and the control cost for rates below one bit per sample

Dynamic Quantizer Design Under Communication Rate Constraints

Feedback type dynamic quantizers such as delta-sigma modulators are typically effective for encoding high-resolution data into lower resolution data. The dynamic quantizers include a filter and a static quantizer. When it is required to control under a communication rate constraint, the data rate of the quantizer output should be minimized appropriately by quantization. This technical note provides numerical methods for the complete design of a type of dynamic quantizers, including the selection of all the quantizer parameters in order to minimize a specific performance index and satisfy a communication constraint. The design method of the dynamic quantizer is proposed using a particle swarm optimization (PSO) method. A part of the initial quantizers in PSO are designed based on an invariant set analysis and an iteration algorithm. Effectiveness of the system with the proposed quantizer is assessed through numerical examples

Quantization and Compressive Sensing

Quantization is an essential step in digitizing signals, and, therefore, an indispensable component of any modern acquisition system. This book chapter explores the interaction of quantization and compressive sensing and examines practical quantization strategies for compressive acquisition systems. Specifically, we first provide a brief overview of quantization and examine fundamental performance bounds applicable to any quantization approach. Next, we consider several forms of scalar quantizers, namely uniform, non-uniform, and 1-bit. We provide performance bounds and fundamental analysis, as well as practical quantizer designs and reconstruction algorithms that account for quantization. Furthermore, we provide an overview of Sigma-Delta ($\Sigma\Delta$) quantization in the compressed sensing context, and also discuss implementation issues, recovery algorithms and performance bounds. As we demonstrate, proper accounting for quantization and careful quantizer design has significant impact in the performance of a compressive acquisition system.Comment: 35 pages, 20 figures, to appear in Springer book "Compressed Sensing and Its Applications", 201

Fusing Censored Dependent Data for Distributed Detection

In this paper, we consider a distributed detection problem for a censoring sensor network where each sensor's communication rate is significantly reduced by transmitting only "informative" observations to the Fusion Center (FC), and censoring those deemed "uninformative". While the independence of data from censoring sensors is often assumed in previous research, we explore spatial dependence among observations. Our focus is on designing the fusion rule under the Neyman-Pearson (NP) framework that takes into account the spatial dependence among observations. Two transmission scenarios are considered, one where uncensored observations are transmitted directly to the FC and second where they are first quantized and then transmitted to further improve transmission efficiency. Copula-based Generalized Likelihood Ratio Test (GLRT) for censored data is proposed with both continuous and discrete messages received at the FC corresponding to different transmission strategies. We address the computational issues of the copula-based GLRTs involving multidimensional integrals by presenting more efficient fusion rules, based on the key idea of injecting controlled noise at the FC before fusion. Although, the signal-to-noise ratio (SNR) is reduced by introducing controlled noise at the receiver, simulation results demonstrate that the resulting noise-aided fusion approach based on adding artificial noise performs very closely to the exact copula-based GLRTs. Copula-based GLRTs and their noise-aided counterparts by exploiting the spatial dependence greatly improve detection performance compared with the fusion rule under independence assumption