Universal Behavior in Large-scale Aggregation of Independent Noisy Observations

Aggregation of noisy observations involves a difficult tradeoff between observation quality, which can be increased by increasing the number of observations, and aggregation quality which decreases if the number of observations is too large. We clarify this behavior for a protypical system in which arbitrarily large numbers of observations exceeding the system capacity can be aggregated using lossy data compression. We show the existence of a scaling relation between the collective error and the system capacity, and show that large scale lossy aggregation can outperform lossless aggregation above a critical level of observation noise. Further, we show that universal results for scaling and critical value of noise which are independent of system capacity can be obtained by considering asymptotic behavior when the system capacity increases toward infinity.Comment: 10 pages, 3 figure

On Distributed Linear Estimation With Observation Model Uncertainties

We consider distributed estimation of a Gaussian source in a heterogenous bandwidth constrained sensor network, where the source is corrupted by independent multiplicative and additive observation noises, with incomplete statistical knowledge of the multiplicative noise. For multi-bit quantizers, we derive the closed-form mean-square-error (MSE) expression for the linear minimum MSE (LMMSE) estimator at the FC. For both error-free and erroneous communication channels, we propose several rate allocation methods named as longest root to leaf path, greedy and integer relaxation to (i) minimize the MSE given a network bandwidth constraint, and (ii) minimize the required network bandwidth given a target MSE. We also derive the Bayesian Cramer-Rao lower bound (CRLB) and compare the MSE performance of our proposed methods against the CRLB. Our results corroborate that, for low power multiplicative observation noises and adequate network bandwidth, the gaps between the MSE of our proposed methods and the CRLB are negligible, while the performance of other methods like individual rate allocation and uniform is not satisfactory

Source-channel coding for coordination over a noisy two-node network

Recently, the concept of coordinating actions between distributed agents has emerged in the information theory literature. It was first introduced by Cuff in 2008 for the point-to-point case of coordination. However, Cuff’s work and the vast majority of the follow-up research are based on establishing coordination over noise-free communication links. In contrast, this thesis investigates the open problem of coordination over noisy point-to-point links. The aim of this study is to examine Shannon’s source-channel separation theorem in the context of coordination. To that end, a general joint scheme to achieve the strong notion of coordination over a discrete memoryless channel is introduced. The strong coordination notion requires that the L1 distance between the induced joint distribution of action sequences selected by the nodes and a prescribed joint distribution vanishes exponentially fast with the sequence block length. From the general joint scheme, three special cases are constructed, one of which resembles Shannon’s separation scheme. As a surprising result, the proposed joint scheme has been found to be able to perform better than a strictly separate scheme. Finally, the last part of the thesis provides simulation results to confirm the presented argument based on comparing the achievable rate regions for the scheme resembling Shannon’s separation and a special case of the general joint scheme

Correlation-based Cross-layer Communication in Wireless Sensor Networks

Wireless sensor networks (WSN) are event based systems that rely on the collective effort of densely deployed sensor nodes continuously observing a physical phenomenon. The spatio-temporal correlation between the sensor observations and the cross-layer design advantages are significant and unique to the design of WSN. Due to the high density in the network topology, sensor observations are highly correlated in the space domain. Furthermore, the nature of the energy-radiating physical phenomenon constitutes the temporal correlation between each consecutive observation of a sensor node. This unique characteristic of WSN can be exploited through a cross-layer design of communication functionalities to improve energy efficiency of the network. In this thesis, several key elements are investigated to capture and exploit the correlation in the WSN for the realization of advanced efficient communication protocols. A theoretical framework is developed to capture the spatial and temporal correlations in WSN and to enable the development of efficient communication protocols. Based on this framework, spatial Correlation-based Collaborative Medium Access Control (CC-MAC) protocol is described, which exploits the spatial correlation in the WSN in order to achieve efficient medium access. Furthermore, the cross-layer module (XLM), which melts common protocol layer functionalities into a cross-layer module for resource-constrained sensor nodes, is developed. The cross-layer analysis of error control in WSN is then presented to enable a comprehensive comparison of error control schemes for WSN. Finally, the cross-layer packet size optimization framework is described.Ph.D.Committee Chair: Ian F. Akyildiz; Committee Member: Douglas M. Blough; Committee Member: Mostafa Ammar; Committee Member: Raghupathy Sivakumar; Committee Member: Ye (Geoffrey) L

Sparse Signal Processing Concepts for Efficient 5G System Design

As it becomes increasingly apparent that 4G will not be able to meet the emerging demands of future mobile communication systems, the question what could make up a 5G system, what are the crucial challenges and what are the key drivers is part of intensive, ongoing discussions. Partly due to the advent of compressive sensing, methods that can optimally exploit sparsity in signals have received tremendous attention in recent years. In this paper we will describe a variety of scenarios in which signal sparsity arises naturally in 5G wireless systems. Signal sparsity and the associated rich collection of tools and algorithms will thus be a viable source for innovation in 5G wireless system design. We will discribe applications of this sparse signal processing paradigm in MIMO random access, cloud radio access networks, compressive channel-source network coding, and embedded security. We will also emphasize important open problem that may arise in 5G system design, for which sparsity will potentially play a key role in their solution.Comment: 18 pages, 5 figures, accepted for publication in IEEE Acces

Fundamental limits in Gaussian channels with feedback: confluence of communication, estimation, and control

The emerging study of integrating information theory and control systems theory has attracted tremendous attention, mainly motivated by the problems of control under communication constraints, feedback information theory, and networked systems. An often overlooked element is the estimation aspect; however, estimation cannot be studied isolatedly in those problems. Therefore, it is natural to investigate systems from the perspective of unifying communication, estimation, and control;This thesis is the first work to advocate such a perspective. To make Matters concrete, we focus on communication systems over Gaussian channels with feedback. For some of these channels, their fundamental limits for communication have been studied using information theoretic methods and control-oriented methods but remain open. In this thesis, we address the problems of characterizing and achieving the fundamental limits for these Gaussian channels with feedback by applying the unifying perspective;We establish a general equivalence among feedback communication, estimation, and feedback stabilization over the same Gaussian channels. As a consequence, we see that the information transmission (communication), information processing (estimation), and information utilization (control), seemingly different and usually separately treated, are in fact three sides of the same entity. We then reveal that the fundamental limitations in feedback communication, estimation, and control coincide: The achievable communication rates in the feedback communication problems can be alternatively given by the decay rates of the Cramer-Rao bounds (CRB) in the associated estimation problems or by the Bode sensitivity integrals in the associated control problems. Utilizing the general equivalence, we design optimal feedback communication schemes based on the celebrated Kalman filtering algorithm; these are the first deterministic, optimal communication schemes for these channels with feedback (except for the degenerated AWGN case). These schemes also extend the Schalkwijk-Kailath (SK) coding scheme and inherit its useful features, such as reduced coding complexity and improved performance. Hence, this thesis demonstrates that the new perspective plays a significant role in gaining new insights and new results in studying Gaussian feedback communication systems. We anticipate that the perspective could be extended to more general problems and helpful in building a theoretically and practically sound paradigm that unifies information, estimation, and control

Reliable Inference from Unreliable Agents

Distributed inference using multiple sensors has been an active area of research since the emergence of wireless sensor networks (WSNs). Several researchers have addressed the design issues to ensure optimal inference performance in such networks. The central goal of this thesis is to analyze distributed inference systems with potentially unreliable components and design strategies to ensure reliable inference in such systems. The inference process can be that of detection or estimation or classification, and the components/agents in the system can be sensors and/or humans. The system components can be unreliable due to a variety of reasons: faulty sensors, security attacks causing sensors to send falsified information, or unskilled human workers sending imperfect information. This thesis first quantifies the effect of such unreliable agents on the inference performance of the network and then designs schemes that ensure a reliable overall inference. In the first part of this thesis, we study the case when only sensors are present in the system, referred to as sensor networks. For sensor networks, the presence of malicious sensors, referred to as Byzantines, are considered. Byzantines are sensors that inject false information into the system. In such systems, the effect of Byzantines on the overall inference performance is characterized in terms of the optimal attack strategies. Game-theoretic formulations are explored to analyze two-player interactions. Next, Byzantine mitigation schemes are designed that address the problem from the system\u27s perspective. These mitigation schemes are of two kinds: Byzantine identification schemes and Byzantine tolerant schemes. Using learning based techniques, Byzantine identification schemes are designed that learn the identity of Byzantines in the network and use this information to improve system performance. When such schemes are not possible, Byzantine tolerant schemes using error-correcting codes are developed that tolerate the effect of Byzantines and maintain good performance in the network. Error-correcting codes help in correcting the erroneous information from these Byzantines and thereby counter their attack. The second line of research in this thesis considers humans-only networks, referred to as human networks. A similar research strategy is adopted for human networks where, the effect of unskilled humans sharing beliefs with a central observer called \emph{CEO} is analyzed, and the loss in performance due to the presence of such unskilled humans is characterized. This problem falls under the family of problems in information theory literature referred to as the \emph{CEO Problem}, but for belief sharing. The asymptotic behavior of the minimum achievable mean squared error distortion at the CEO is studied in the limit when the number of agents $L$ and the sum rate $R$ tend to infinity. An intermediate regime of performance between the exponential behavior in discrete CEO problems and the $1/R$ behavior in Gaussian CEO problems is established. This result can be summarized as the fact that sharing beliefs (uniform) is fundamentally easier in terms of convergence rate than sharing measurements (Gaussian), but sharing decisions is even easier (discrete). Besides theoretical analysis, experimental results are reported for experiments designed in collaboration with cognitive psychologists to understand the behavior of humans in the network. The act of fusing decisions from multiple agents is observed for humans and the behavior is statistically modeled using hierarchical Bayesian models. The implications of such modeling on the design of large human-machine systems is discussed. Furthermore, an error-correcting codes based scheme is proposed to improve system performance in the presence of unreliable humans in the inference process. For a crowdsourcing system consisting of unskilled human workers providing unreliable responses, the scheme helps in designing easy-to-perform tasks and also mitigates the effect of erroneous data. The benefits of using the proposed approach in comparison to the majority voting based approach are highlighted using simulated and real datasets. In the final part of the thesis, a human-machine inference framework is developed where humans and machines interact to perform complex tasks in a faster and more efficient manner. A mathematical framework is built to understand the benefits of human-machine collaboration. Such a study is extremely important for current scenarios where humans and machines are constantly interacting with each other to perform even the simplest of tasks. While machines perform best in some tasks, humans still give better results in tasks such as identifying new patterns. By using humans and machines together, one can extract complete information about a phenomenon of interest. Such an architecture, referred to as Human-Machine Inference Networks (HuMaINs), provides promising results for the two cases of human-machine collaboration: \emph{machine as a coach} and \emph{machine as a colleague}. For simple systems, we demonstrate tangible performance gains by such a collaboration which provides design modules for larger, and more complex human-machine systems. However, the details of such larger systems needs to be further explored