Distributed Signal Processing over Large-Scale Complex Systems

Large-scale and complex dynamical networks with high-dimension states have been emerging in the era of big data, which potentially generate massive data sets. To deal with the massive data sets, one promising method is the distributed collaboration strategy over the network. This dissertation proposes the schemes of distributed estimation and distributed quickest detection and also studies the performance of the distributed schemes with the large deviation analysis, which answers a fundamental question on how to quantify the rate at which the distributed scheme approaches the centralized performance. First, the distributed Kalman filtering scheme with the Gossip interaction among sensors is proposed to estimate the high-dimension states at each node, where sensors exchange their filtered states (estimates and error covariance) and propagate their observations via inter-sensor communications. The conditional estimation error covariance sequence at each sensor under this scheme is proven to evolve as a random Riccati equation (RRE) with Markov modulated switching. By formulating the RRE as a random dynamical system, it is shown that the network consensus over the estimation at each node is achieved. The large deviation analysis further shows that the distributed scheme converges to the optimal centralized one at an exponentially fast rate. By considering the energy and bandwidth constrains, a Quantized Gossip-based Interactive Kalman Filtering algorithm for scalar dynamic systems is also proposed, where the sensors exchange their quantized states with neighbors via inter-sensor communications. It is shown that, in the countable infinite quantization alphabet case, the network can still achieve weak consensus with the additional information loss caused by quantization. It is also proved that, under certain conditions, the network can also achieve weak consensus with the finite quantization alphabet, which is more restricted and practical. Then, the distributed quickest detection scheme is proposed with multiple rounds of inter-sensor communications to propagate observations during the sampling interval. By modeling the information propagation dynamics in the network as a Markov process, the two-layer large deviation analysis is used to analyze the performance of the distributed scheme. The first layer analysis proves that the probability of false alarm decays to zero exponentially fast with the increasing of the averaged detection delay, where the Kullback-Leibler (KL) information number is established as a crucial factor. The second-layer analysis shows that the probability of the rare event that not all observations are available at a sensor decays to zero at an exponentially fast rate when the number of communications increases, where the large deviation upper and lower bounds for this rate are also derived, based on which it is shown that the performance of the distributed algorithm converges exponentially fast to that of the centralized one, by proving that the defined distributed KL information number converges to the centralized KL information number

Data-Efficient Quickest Change Detection with On-Off Observation Control

In this paper we extend the Shiryaev's quickest change detection formulation by also accounting for the cost of observations used before the change point. The observation cost is captured through the average number of observations used in the detection process before the change occurs. The objective is to select an on-off observation control policy, that decides whether or not to take a given observation, along with the stopping time at which the change is declared, so as to minimize the average detection delay, subject to constraints on both the probability of false alarm and the observation cost. By considering a Lagrangian relaxation of the constraint problem, and using dynamic programming arguments, we obtain an \textit{a posteriori} probability based two-threshold algorithm that is a generalized version of the classical Shiryaev algorithm. We provide an asymptotic analysis of the two-threshold algorithm and show that the algorithm is asymptotically optimal, i.e., the performance of the two-threshold algorithm approaches that of the Shiryaev algorithm, for a fixed observation cost, as the probability of false alarm goes to zero. We also show, using simulations, that the two-threshold algorithm has good observation cost-delay trade-off curves, and provides significant reduction in observation cost as compared to the naive approach of fractional sampling, where samples are skipped randomly. Our analysis reveals that, for practical choices of constraints, the two thresholds can be set independent of each other: one based on the constraint of false alarm and another based on the observation cost constraint alone.Comment: Preliminary version of this paper has been presented at ITA Workshop UCSD 201

The 1st International Electronic Conference on Algorithms

This book presents 22 of the accepted presentations at the 1st International Electronic Conference on Algorithms which was held completely online from September 27 to October 10, 2021. It contains 16 proceeding papers as well as 6 extended abstracts. The works presented in the book cover a wide range of fields dealing with the development of algorithms. Many of contributions are related to machine learning, in particular deep learning. Another main focus among the contributions is on problems dealing with graphs and networks, e.g., in connection with evacuation planning problems