Problems in Signal Processing and Inference on Graphs

Modern datasets are often massive due to the sharp decrease in the cost of collecting and storing data. Many are endowed with relational structure modeled by a graph, an object comprising a set of points and a set of pairwise connections between them. A ``signal on a graph'' has elements related to each other through a graph---it could model, for example, measurements from a sensor network. In this dissertation we study several problems in signal processing and inference on graphs. We begin by introducing an analogue to Heisenberg's time-frequency uncertainty principle for signals on graphs. We use spectral graph theory and the standard extension of Fourier analysis to graphs. Our spectral graph uncertainty principle makes precise the notion that a highly localized signal on a graph must have a broad spectrum, and vice versa. Next, we consider the problem of detecting a random walk on a graph from noisy observations. We characterize the performance of the optimal detector through the (type-II) error exponent, borrowing techniques from statistical physics to develop a lower bound exhibiting a phase transition. Strong performance is only guaranteed when the signal to noise ratio exceeds twice the random walk's entropy rate. Monte Carlo simulations show that the lower bound is quite close to the true exponent. Next, we introduce a technique for inferring the source of an epidemic from observations at a few nodes. We develop a Monte Carlo technique to simulate the infection process, and use statistics computed from these simulations to approximate the likelihood, which we then maximize to locate the source. We further introduce a logistic autoregressive model (ALARM), a simple model for binary processes on graphs that can still capture a variety of behavior. We demonstrate its simplicity by showing how to easily infer the underlying graph structure from measurements; a technique versatile enough that it can work under model mismatch. Finally, we introduce the exact formula for the error of the randomized Kaczmarz algorithm, a linear system solver for sparse systems, which often arise in graph theory. This is important because, as we show, existing performance bounds are quite loose.Engineering and Applied Sciences - Engineering Science

A Spectral Graph Uncertainty Principle

The spectral theory of graphs provides a bridge between classical signal processing and the nascent field of graph signal processing. In this paper, a spectral graph analogy to Heisenberg's celebrated uncertainty principle is developed. Just as the classical result provides a tradeoff between signal localization in time and frequency, this result provides a fundamental tradeoff between a signal's localization on a graph and in its spectral domain. Using the eigenvectors of the graph Laplacian as a surrogate Fourier basis, quantitative definitions of graph and spectral "spreads" are given, and a complete characterization of the feasibility region of these two quantities is developed. In particular, the lower boundary of the region, referred to as the uncertainty curve, is shown to be achieved by eigenvectors associated with the smallest eigenvalues of an affine family of matrices. The convexity of the uncertainty curve allows it to be found to within $\varepsilon$ by a fast approximation algorithm requiring $O(\varepsilon^{-1/2})$ typically sparse eigenvalue evaluations. Closed-form expressions for the uncertainty curves for some special classes of graphs are derived, and an accurate analytical approximation for the expected uncertainty curve of Erd\H{o}s-R\'enyi random graphs is developed. These theoretical results are validated by numerical experiments, which also reveal an intriguing connection between diffusion processes on graphs and the uncertainty bounds.Comment: 40 pages, 8 figure

Randomized Kaczmarz algorithms: Exact MSE analysis and optimal sampling probabilities

The Kaczmarz method, or the algebraic reconstruction technique (ART), is a popular method for solving large-scale overdetermined systems of equations. Recently, Strohmer et al. proposed the randomized Kaczmarz algorithm, an improvement that guarantees exponential convergence to the solution. This has spurred much interest in the algorithm and its extensions. We provide in this paper an exact formula for the mean squared error (MSE) in the value reconstructed by the algorithm. We also compute the exponential decay rate of the MSE, which we call the “annealed” error exponent. We show that the typical performance of the algorithm is far better than the average performance. We define the “quenched” error exponent to characterize the typical performance. This is far harder to computethan the annealed error exponent, but we provide an approximation that matches empirical results. We also explore optimizing the algorithm’s row-selection probabilities to speed up the algorithm’s convergence.Engineering and Applied Science

A Distributed Scheme for Detection of Information Flows

Abstract—Distributed detection of information flows spanning many nodes in a wireless sensor network is considered. In such a system, eavesdroppers are deployed near several nodes in the network. As data may be encrypted or padded, the eavesdroppers can only measure packet timestamps. Each eavesdropper, given a sequence of timestamps, must compress the information for transmission to a fusion center. Given the compressed data, the fusion center must decide whether the monitored nodes are part of an information flow. Information flows may be embedded with chaff noise, and packets may be perturbed by a random but bounded delay. A specific quantizer and algorithmic detection scheme are proposed. Performance is characterized by the maximum fraction of chaff that may be inserted in an information flow while still achieving vanishing error probabilities. A lower bound on the performance of the optimal system is derived. An upper bound on the performance of a system using the proposed quantizer is also found. Index Terms—Intrusion detection, Traffic analysis, Network surveillanc

Distributed Detection of Multi-Hop Information Flows With Fusion Capacity Constraints

The problem of detecting multihop information flows subject to communication constraints is considered. In a distributed detection scheme, eavesdroppers are deployed near nodes in a network, each able to measure the transmission timestamps of a single node. The eavesdroppers must then compress the information and transmit it to a fusion center, which then decides whether a sequence of monitored nodes are transmitting an information flow. A performance measure is defined based on the maximum fraction of chaff packets under which flows are still detectable. The performance of a detector becomes a function of the communication constraints and the number of nodes in the sequence. Achievability results are obtained by designing a practical distributed detection scheme, including a new flow finding algorithm that has vanishing error probabilities for a limited fraction of chaff packets. Converse results are obtained by characterizing the fraction of chaff packets sufficient for an information flow to mimic the distributions of independent traffic under the proposed compression scheme.National Science Foundation (U.S.) (Grant No. CCF-0635070)United States. Office of Naval Research (MURI W911NF-08-1-0238

Distributed recovery of a Gaussian source in interference with successive lattice processing

A scheme for recovery of a signal by distributed listeners in the presence of Gaussian interference is constructed by exhausting an "iterative power reduction" property. An upper bound for the scheme's achieved mean-squared-error distortion is derived. The strategy exposes a parameter search problem, which, when solved, causes the scheme to outperform others of its kind. Performance of a blocklength-one scheme is simulated and is seen to improve over plain source coding without compression in the presence of many interferers, and experiences less outages over ensembles of channels. Keywords: network information theory; distributed source coding; lattice code