Common Information and Decentralized Inference with Dependent Observations

Wyner\u27s common information was originally defined for a pair of dependent discrete random variables. This thesis generalizes its definition in two directions: the number of dependent variables can be arbitrary, so are the alphabets of those random variables. New properties are determined for the generalized Wyner\u27s common information of multiple dependent variables. More importantly, a lossy source coding interpretation of Wyner\u27s common information is developed using the Gray-Wyner network. It is established that the common information equals to the smallest common message rate when the total rate is arbitrarily close to the rate distortion function with joint decoding if the distortions are within some distortion region. The application of Wyner\u27s common information to inference problems is also explored in the thesis. A central question is under what conditions does Wyner\u27s common information capture the entire information about the inference object. Under a simple Bayesian model, it is established that for infinitely exchangeable random variables that the common information is asymptotically equal to the information of the inference object. For finite exchangeable random variables, connection between common information and inference performance metrics are also established. The problem of decentralized inference is generally intractable with conditional dependent observations. A promising approach for this problem is to utilize a hierarchical conditional independence model. Utilizing the hierarchical conditional independence model, we identify a more general condition under which the distributed detection problem becomes tractable, thereby broadening the classes of distributed detection problems with dependent observations that can be readily solved. We then develop the sufficiency principle for data reduction for decentralized inference. For parallel networks, the hierarchical conditional independence model is used to obtain conditions such that local sufficiency implies global sufficiency. For tandem networks, the notion of conditional sufficiency is introduced and the related theory and tools are developed. Connections between the sufficiency principle and distributed source coding problems are also explored. Furthermore, we examine the impact of quantization on decentralized data reduction. The conditions under which sufficiency based data reduction with quantization constraints is optimal are identified. They include the case when the data at decentralized nodes are conditionally independent as well as a class of problems with conditionally dependent observations that admit conditional independence structure through the hierarchical conditional independence model

Detection of fast radio transients with multiple stations: a case study using the Very Long Baseline Array

Recent investigations reveal an important new class of transient radio phenomena that occur on sub-millisecond timescales. Often transient surveys' data volumes are too large to archive exhaustively. Instead, an on-line automatic system must excise impulsive interference and detect candidate events in real-time. This work presents a case study using data from multiple geographically distributed stations to perform simultaneous interference excision and transient detection. We present several algorithms that incorporate dedispersed data from multiple sites, and report experiments with a commensal real-time transient detection system on the Very Long Baseline Array (VLBA). We test the system using observations of pulsar B0329+54. The multiple-station algorithms enhanced sensitivity for detection of individual pulses. These strategies could improve detection performance for a future generation of geographically distributed arrays such as the Australian Square Kilometre Array Pathfinder and the Square Kilometre Array.Comment: 12 pages, 14 figures. Accepted for Ap

Distributed Binary Detection over Fading Channels: Cooperative and Parallel Architectures

This paper considers the problem of binary distributed detection of a known signal in correlated Gaussian sensing noise in a wireless sensor network, where the sensors are restricted to use likelihood ratio test (LRT), and communicate with the fusion center (FC) over bandwidth-constrained channels that are subject to fading and noise. To mitigate the deteriorating effect of fading encountered in the conventional parallel fusion architecture, in which the sensors directly communicate with the FC, we propose new fusion architectures that enhance the detection performance, via harvesting cooperative gain (so-called decision diversity gain). In particular, we propose: (i) cooperative fusion architecture with Alamouti's space-time coding (STC) scheme at sensors, (ii) cooperative fusion architecture with signal fusion at sensors, and (iii) parallel fusion architecture with local threshold changing at sensors. For these schemes, we derive the LRT and majority fusion rules at the FC, and provide upper bounds on the average error probabilities for homogeneous sensors, subject to uncorrelated Gaussian sensing noise, in terms of signal-to-noise ratio (SNR) of communication and sensing channels. Our simulation results indicate that, when the FC employs the LRT rule, unless for low communication SNR and moderate/high sensing SNR, performance improvement is feasible with the new fusion architectures. When the FC utilizes the majority rule, such improvement is possible, unless for high sensing SNR

Distributed Detection and Estimation in Wireless Sensor Networks

In this article we consider the problems of distributed detection and estimation in wireless sensor networks. In the first part, we provide a general framework aimed to show how an efficient design of a sensor network requires a joint organization of in-network processing and communication. Then, we recall the basic features of consensus algorithm, which is a basic tool to reach globally optimal decisions through a distributed approach. The main part of the paper starts addressing the distributed estimation problem. We show first an entirely decentralized approach, where observations and estimations are performed without the intervention of a fusion center. Then, we consider the case where the estimation is performed at a fusion center, showing how to allocate quantization bits and transmit powers in the links between the nodes and the fusion center, in order to accommodate the requirement on the maximum estimation variance, under a constraint on the global transmit power. We extend the approach to the detection problem. Also in this case, we consider the distributed approach, where every node can achieve a globally optimal decision, and the case where the decision is taken at a central node. In the latter case, we show how to allocate coding bits and transmit power in order to maximize the detection probability, under constraints on the false alarm rate and the global transmit power. Then, we generalize consensus algorithms illustrating a distributed procedure that converges to the projection of the observation vector onto a signal subspace. We then address the issue of energy consumption in sensor networks, thus showing how to optimize the network topology in order to minimize the energy necessary to achieve a global consensus. Finally, we address the problem of matching the topology of the network to the graph describing the statistical dependencies among the observed variables.Comment: 92 pages, 24 figures. To appear in E-Reference Signal Processing, R. Chellapa and S. Theodoridis, Eds., Elsevier, 201

Bayesian Design of Tandem Networks for Distributed Detection With Multi-bit Sensor Decisions

We consider the problem of decentralized hypothesis testing under communication constraints in a topology where several peripheral nodes are arranged in tandem. Each node receives an observation and transmits a message to its successor, and the last node then decides which hypothesis is true. We assume that the observations at different nodes are, conditioned on the true hypothesis, independent and the channel between any two successive nodes is considered error-free but rate-constrained. We propose a cyclic numerical design algorithm for the design of nodes using a person-by-person methodology with the minimum expected error probability as a design criterion, where the number of communicated messages is not necessarily equal to the number of hypotheses. The number of peripheral nodes in the proposed method is in principle arbitrary and the information rate constraints are satisfied by quantizing the input of each node. The performance of the proposed method for different information rate constraints, in a binary hypothesis test, is compared to the optimum rate-one solution due to Swaszek and a method proposed by Cover, and it is shown numerically that increasing the channel rate can significantly enhance the performance of the tandem network. Simulation results for $M$-ary hypothesis tests also show that by increasing the channel rates the performance of the tandem network significantly improves