Consistency in Models for Distributed Learning under Communication Constraints

Motivated by sensor networks and other distributed settings, several models for distributed learning are presented. The models differ from classical works in statistical pattern recognition by allocating observations of an independent and identically distributed (i.i.d.) sampling process amongst members of a network of simple learning agents. The agents are limited in their ability to communicate to a central fusion center and thus, the amount of information available for use in classification or regression is constrained. For several basic communication models in both the binary classification and regression frameworks, we question the existence of agent decision rules and fusion rules that result in a universally consistent ensemble. The answers to this question present new issues to consider with regard to universal consistency. Insofar as these models present a useful picture of distributed scenarios, this paper addresses the issue of whether or not the guarantees provided by Stone's Theorem in centralized environments hold in distributed settings.Comment: To appear in the IEEE Transactions on Information Theor

Distributed Regression in Sensor Networks: Training Distributively with Alternating Projections

Wireless sensor networks (WSNs) have attracted considerable attention in recent years and motivate a host of new challenges for distributed signal processing. The problem of distributed or decentralized estimation has often been considered in the context of parametric models. However, the success of parametric methods is limited by the appropriateness of the strong statistical assumptions made by the models. In this paper, a more flexible nonparametric model for distributed regression is considered that is applicable in a variety of WSN applications including field estimation. Here, starting with the standard regularized kernel least-squares estimator, a message-passing algorithm for distributed estimation in WSNs is derived. The algorithm can be viewed as an instantiation of the successive orthogonal projection (SOP) algorithm. Various practical aspects of the algorithm are discussed and several numerical simulations validate the potential of the approach.Comment: To appear in the Proceedings of the SPIE Conference on Advanced Signal Processing Algorithms, Architectures and Implementations XV, San Diego, CA, July 31 - August 4, 200

Fusing Censored Dependent Data for Distributed Detection

In this paper, we consider a distributed detection problem for a censoring sensor network where each sensor's communication rate is significantly reduced by transmitting only "informative" observations to the Fusion Center (FC), and censoring those deemed "uninformative". While the independence of data from censoring sensors is often assumed in previous research, we explore spatial dependence among observations. Our focus is on designing the fusion rule under the Neyman-Pearson (NP) framework that takes into account the spatial dependence among observations. Two transmission scenarios are considered, one where uncensored observations are transmitted directly to the FC and second where they are first quantized and then transmitted to further improve transmission efficiency. Copula-based Generalized Likelihood Ratio Test (GLRT) for censored data is proposed with both continuous and discrete messages received at the FC corresponding to different transmission strategies. We address the computational issues of the copula-based GLRTs involving multidimensional integrals by presenting more efficient fusion rules, based on the key idea of injecting controlled noise at the FC before fusion. Although, the signal-to-noise ratio (SNR) is reduced by introducing controlled noise at the receiver, simulation results demonstrate that the resulting noise-aided fusion approach based on adding artificial noise performs very closely to the exact copula-based GLRTs. Copula-based GLRTs and their noise-aided counterparts by exploiting the spatial dependence greatly improve detection performance compared with the fusion rule under independence assumption

Distributed Learning in Wireless Sensor Networks

The problem of distributed or decentralized detection and estimation in applications such as wireless sensor networks has often been considered in the framework of parametric models, in which strong assumptions are made about a statistical description of nature. In certain applications, such assumptions are warranted and systems designed from these models show promise. However, in other scenarios, prior knowledge is at best vague and translating such knowledge into a statistical model is undesirable. Applications such as these pave the way for a nonparametric study of distributed detection and estimation. In this paper, we review recent work of the authors in which some elementary models for distributed learning are considered. These models are in the spirit of classical work in nonparametric statistics and are applicable to wireless sensor networks.Comment: Published in the Proceedings of the 42nd Annual Allerton Conference on Communication, Control and Computing, University of Illinois, 200

Efficient calculation of sensor utility and sensor removal in wireless sensor networks for adaptive signal estimation and beamforming

Wireless sensor networks are often deployed over a large area of interest and therefore the quality of the sensor signals may vary significantly across the different sensors. In this case, it is useful to have a measure for the importance or the so-called "utility" of each sensor, e.g., for sensor subset selection, resource allocation or topology selection. In this paper, we consider the efficient calculation of sensor utility measures for four different signal estimation or beamforming algorithms in an adaptive context. We use the definition of sensor utility as the increase in cost (e.g., mean-squared error) when the sensor is removed from the estimation procedure. Since each possible sensor removal corresponds to a new estimation problem (involving less sensors), calculating the sensor utilities would require a continuous updating of different signal estimators (where is the number of sensors), increasing computational complexity and memory usage by a factor. However, we derive formulas to efficiently calculate all sensor utilities with hardly any increase in memory usage and computational complexity compared to the signal estimation algorithm already in place. When applied in adaptive signal estimation algorithms, this allows for on-line tracking of all the sensor utilities at almost no additional cost. Furthermore, we derive efficient formulas for sensor removal, i.e., for updating the signal estimator coefficients when a sensor is removed, e.g., due to a failure in the wireless link or when its utility is too low. We provide a complexity evaluation of the derived formulas, and demonstrate the significant reduction in computational complexity compared to straightforward implementations

Energy-Efficient Data Acquisition in Wireless Sensor Networks through Spatial Correlation

The application of Wireless Sensor Networks (WSNs) is restrained by their often-limited lifetime. A sensor node's lifetime is fundamentally linked to the volume of data that it senses, processes and reports. Spatial correlation between sensor nodes is an inherent phenomenon to WSNs, induced by redundant nodes which report duplicated information. In this paper, we report on the design of a distributed sampling scheme referred to as the 'Virtual Sampling Scheme' (VSS). This scheme is formed from two components: an algorithm for forming virtual clusters, and a distributed sampling method. VSS primarily utilizes redundancy of sensor nodes to get only a subset to sense the environment at any one time. Sensor nodes that are not sensing the environment are in a low-power sleep state, thus conserving energy. Furthermore, VSS balances the energy consumption amongst nodes by using a round robin method