On the Development of Distributed Estimation Techniques for Wireless Sensor Networks

Wireless sensor networks (WSNs) have lately witnessed tremendous demand, as evidenced by the increasing number of day-to-day applications. The sensor nodes aim at estimating the parameters of their corresponding adaptive filters to achieve the desired response for the event of interest. Some of the burning issues related to linear parameter estimation in WSNs have been addressed in this thesis mainly focusing on reduction of communication overhead and latency, and robustness to noise. The first issue deals with the high communication overhead and latency in distributed parameter estimation techniques such as diffusion least mean squares (DLMS) and incremental least mean squares (ILMS) algorithms. Subsequently the poor performance demonstrated by these distributed techniques in presence of impulsive noise has been dealt separately. The issue of source localization i.e. estimation of source bearing in WSNs, where the existing decentralized algorithms fail to perform satisfactorily, has been resolved in this thesis. Further the same issue has been dealt separately independent of nodal connectivity in WSNs. This thesis proposes two algorithms namely the block diffusion least mean squares (BDLMS) and block incremental least mean squares (BILMS) algorithms for reducing the communication overhead in WSNs. The theoretical and simulation studies demonstrate that BDLMS and BILMS algorithms provide the same performances as that of DLMS and ILMS, but with significant reduction in communication overheads per node. The latency also reduces by a factor as high as the block-size used in the proposed algorithms. With an aim to develop robustness towards impulsive noise, this thesis proposes three robust distributed algorithms i.e. saturation nonlinearity incremental LMS (SNILMS), saturation nonlinearity diffusion LMS (SNDLMS) and Wilcoxon norm diffusion LMS (WNDLMS) algorithms. The steady-state analysis of SNILMS algorithm is carried out based on spatial-temporal energy conservation principle. The theoretical and simulation results show that these algorithms are robust to impulsive noise. The SNDLMS algorithm is found to provide better performance than SNILMS and WNDLMS algorithms. In order to develop a distributed source localization technique, a novel diffusion maximum likelihood (ML) bearing estimation algorithm is proposed in this thesis which needs less communication overhead than the centralized algorithms. After forming a random array with its neighbours, each sensor node estimates the source bearing by optimizing the ML function locally using a diffusion particle swarm optimization algorithm. The simulation results show that the proposed algorithm performs better than the centralized multiple signal classification (MUSIC) algorithm in terms of probability of resolution and root mean square error. Further, in order to make the proposed algorithm independent of nodal connectivity, a distributed in-cluster bearing estimation technique is proposed. Each cluster of sensors estimates the source bearing by optimizing the ML function locally in cooperation with other clusters. The simulation results demonstrate improved performance of the proposed method in comparison to the centralized and decentralized MUSIC algorithms, and the distributed in-network algorith

Privacy-Preserving Decentralized Optimization and Event Localization

This dissertation considers decentralized optimization and its applications. On the one hand, we address privacy preservation for decentralized optimization, where N agents cooperatively minimize the sum of N convex functions private to these individual agents. In most existing decentralized optimization approaches, participating agents exchange and disclose states explicitly, which may not be desirable when the states contain sensitive information of individual agents. The problem is more acute when adversaries exist which try to steal information from other participating agents. To address this issue, we first propose two privacy-preserving decentralized optimization approaches based on ADMM (alternating direction method of multipliers) and subgradient method, respectively, by leveraging partially homomorphic cryptography. To our knowledge, this is the first time that cryptographic techniques are incorporated in a fully decentralized setting to enable privacy preservation in decentralized optimization in the absence of any third party or aggregator. To facilitate the incorporation of encryption in a fully decentralized manner, we also introduce a new ADMM which allows time-varying penalty matrices and rigorously prove that it has a convergence rate of O(1/t). However, given that encryption-based algorithms unavoidably bring about extra computational and communication overhead in real-time optimization [61], we then propose another novel privacy solution for decentralized optimization based on function decomposition and ADMM which enables privacy without incurring large communication/computational overhead. On the other hand, we address the application of decentralized optimization to the event localization problem, which plays a fundamental role in many wireless sensor network applications such as environmental monitoring, homeland security, medical treatment, and health care. The event localization problem is essentially a non-convex and non-smooth problem. We address such a problem in two ways. First, a completely decentralized solution based on augmented Lagrangian methods and ADMM is proposed to solve the non-smooth and non-convex problem directly, rather than using conventional convex relaxation techniques. However, this algorithm requires the target event to be within the convex hull of the deployed sensors. To address this issue, we propose another two scalable distributed algorithms based on ADMM and convex relaxation, which do not require the target event to be within the convex hull of the deployed sensors. Simulation results confirm effectiveness of the proposed algorithms

Iterative Algorithms for Distributed Optimization with Applications to Multi-Agent Estimation and Control

Optimization is a prevalent tool in control and estimation. This work explores the theoretical and practical challenges in the design and analysis of distributed algorithms to solve optimization problems related to multi-agent systems.We begin by considering a problem related to parameter estimation in sensor networks. We show that the maximum likelihood estimation formulation of several localization problems based on inter-sensor measurements reduces to the form of a common constrained optimization. We then design a distributed algorithm that utilizes only the most recent measurements and the estimates from the neighboring sensors, to iteratively compute the optimal solution. Our analysis shows that the solutions obtained from this algorithm converge locally to the maximum likelihood estimates, nevertheless simulations show that this convergence may occur globally. Furthermore, in experimental results using custom ultra-wideband radio frequency devices, this algorithm outperformed other distributed methods tested for a given localization problem.Next, we consider a multi-agent coordination problem formulated as a finite horizon optimization of the type used in model predictive control. We present two distributed and iterative algorithms, in which each agent is assigned a cost function, which it optimizes to compute its own control action. These cost functions depend on the states and the estimates of the control variables of the agents' neighbors, which are obtained through inter-agent communication. For the first algorithm, the agents are able to receive estimates from 2-hop neighbors, whereas the second algorithm utilizes only 1-hop neighbor information. For the first algorithm, our results show that the local solutions converge to the solution of the original model predictive control problem, regardless of how the algorithm is initialized. Because this convergence is asymptotic, we derive practical conditions for terminating the algorithm in a finite number of iterations, such that the closed-loop system achieves the desired coordination. For the second algorithm, due to more restrictive constraints, the convergence occurs to suboptimal solutions of the model predictive control problem. Nevertheless, simulations demonstrate that the optimality gap is small, and in some cases zero.A key takeaway from these results is that in many problems of multi-agent systems, the communication between the agents can be leveraged to design distributed algorithms that match the quality of solutions that one would obtain from centralized approaches

Linear Estimation in Interconnected Sensor Systems with Information Constraints

A ubiquitous challenge in many technical applications is to estimate an unknown state by means of data that stems from several, often heterogeneous sensor sources. In this book, information is interpreted stochastically, and techniques for the distributed processing of data are derived that minimize the error of estimates about the unknown state. Methods for the reconstruction of dependencies are proposed and novel approaches for the distributed processing of noisy data are developed

Linear Estimation in Interconnected Sensor Systems with Information Constraints

A ubiquitous challenge in many technical applications is to estimate an unknown state by means of data that stems from several, often heterogeneous sensor sources. In this book, information is interpreted stochastically, and techniques for the distributed processing of data are derived that minimize the error of estimates about the unknown state. Methods for the reconstruction of dependencies are proposed and novel approaches for the distributed processing of noisy data are developed

Target localization using RSS measurements in wireless sensor networks

The subject of this thesis is the development of localization algorithms for target localization in wireless sensor networks using received signal strength (RSS) measurements or Quantized RSS (QRSS) measurements. In chapter 3 of the thesis, target localization using RSS measurements is investigated. Many existing works on RSS localization assumes that the shadowing components are uncorrelated. However, here, shadowing is assumed to be spatially correlated. It can be shown that localization accuracy can be improved with the consideration of correlation between pairs of RSS measurements. By linearizing the corresponding Maximum Likelihood (ML) objective function, a weighted least squares (WLS) algorithm is formulated to obtain the target location. An iterative technique based on Newtons method is utilized to give a solution. Numerical simulations show that the proposed algorithms achieves better performance than existing algorithms with reasonable complexity. In chapter 4, target localization with an unknown path loss model parameter is investigated. Most published work estimates location and these parameters jointly using iterative methods with a good initialization of path loss exponent (PLE). To avoid finding an initialization, a global optimization algorithm, particle swarm optimization (PSO) is employed to optimize the ML objective function. By combining PSO with a consensus algorithm, the centralized estimation problem is extended to a distributed version so that can be implemented in distributed WSN. Although suboptimal, the distributed approach is very suitable for implementation in real sensor networks, as it is scalable, robust against changing of network topology and requires only local communication. Numerical simulations show that the accuracy of centralized PSO can attain the Cramer Rao Lower Bound (CRLB). Also, as expected, there is some degradation in performance of the distributed PSO with respect to the centralized PSO. In chapter 5, a distributed gradient algorithm for RSS based target localization using only quantized data is proposed. The ML of the Quantized RSS is derived and PSO is used to provide an initial estimate for the gradient algorithm. A practical quantization threshold designer is presented for RSS data. To derive a distributed algorithm using only the quantized signal, the local estimate at each node is also quantized. The RSS measurements and the local estimate at each sensor node are quantized in different ways. By using a quantization elimination scheme, a quantized distributed gradient method is proposed. In the distributed algorithm, the quantization noise in the local estimate is gradually eliminated with each iteration. Simulations show that the performance of the centralized algorithm can reach the CRLB. The proposed distributed algorithm using a small number of bits can achieve the performance of the distributed gradient algorithm using unquantized data

Efficient Low Cost Range-Based Localization Algorithm for Ad-hoc Wireless Sensors Networks

Revised version submitted to Ad Hoc NetworksBuilding an efficient node localization system in wireless sensor networks is facing several challenges. For example, calculating the square root consumes computational resources and utilizing flooding techniques to broadcast nodes location wastes bandwidth and energy. Reducing computational complexity and communication overhead is essential in order to reduce power consumption, extend the life time of the battery operated nodes, and improve the performance of the limited computational resources of these sensor nodes. In this paper, we revise the mathematical model,the analysis and the simulation experiments of the Trigonometric based Ad-hoc Localiza-tion System (TALS), a range-based localization system presented previously. Furthermore, the study is extended, and a new technique to optimize the system is proposed. An analysis and an extensive simulation for the optimized TALS (OTALS) is presented showing its cost, accuracy, and efficiency, thus deducing the impact of its parameters on performance. Hence, the contribution of this work can be summarized as follows: 1) Proposing and employing a novel modified Manhattan distance norm in the TALS localization process. 2) Analyzing and simulating of OTALS showing its computational cost and accuracy and comparing them with other related work. 3) Studying the impacts of different parameters like anchor density, node density, noisy measurements, transmission range, and non-convex network areas. 4) Extending our previous joint work, TALS, to consider base anchors to be located in positions other than the origin and analyzing this work to illustrate the possibility of selecting a wrong quadrant at the first iteration and how this problem is overcome. Through mathematical analysis and intensive simulation, OTALS proved to be iterative , distributed, and computationally simple. It presented superior performance compared to other localization techniques

Some New Results in Distributed Tracking and Optimization

The current age of Big Data is built on the foundation of distributed systems, and efficient distributed algorithms to run on these systems.With the rapid increase in the volume of the data being fed into these systems, storing and processing all this data at a central location becomes infeasible. Such a central \textit{server} requires a gigantic amount of computational and storage resources. Even when it is possible to have central servers, it is not always desirable, due to privacy concerns. Also, sending huge amounts of data to such servers incur often infeasible bandwidth requirements. In this dissertation, we consider two kinds of distributed architectures: 1) star-shaped topology, where multiple worker nodes are connected to, and communicate with a server, but the workers do not communicate with each other; and 2) mesh topology or network of interconnected workers, where each worker can communicate with a small number of neighboring workers. In the first half of this dissertation (Chapters 2 and 3), we consider distributed systems with mesh topology.We study two different problems in this context. First, we study the problem of simultaneous localization and multi-target tracking. Multiple mobile agents localize themselves cooperatively, while also tracking multiple, unknown number of mobile targets, in the presence of measurement-origin uncertainty. In situations with limited GPS signal availability, agents (like self-driving cars in urban canyons, or autonomous vehicles in hazardous environments) need to rely on inter-agent measurements for localization. The agents perform the additional task of tracking multiple targets (pedestrians and road-signs for self-driving cars). We propose a decentralized algorithm for this problem. To be effective in real-time applications, we propose efficient Gaussian and Gaussian-mixture based filters, rather than the computationally expensive particle-based methods in the existing literature. Our novel factor-graph based approach gives better performance, in terms of both agent localization errors, and target-location and cardinality errors. Next, we study an online convex optimization problem, where a network of agents cooperate to minimize a global time-varying objective function. Only the local functions are revealed to individual agents. The agents also need to satisfy their individual constraints. We propose a primal-dual update based decentralized algorithm for this problem. Under standard assumptions, we prove that the proposed algorithm achieves sublinear regret and constraint violation across the network. In other words, over a long enough time horizon, the decisions taken by the agents are, on average, as good as if all the information was revealed ahead of time. In addition, the individual constraint violations of the agents, averaged over time, are zero. In the next part of the dissertation (Chapters 4), we study distributed systems with a star-shaped topology. The problem we study is distributed nonconvex optimization. With the recent success of deep learning, coupled with the use of distributed systems to solve large-scale problems, this problem has gained prominence over the past decade. The recently proposed paradigm of Federated Learning (which has already been deployed by Google/Apple in Android/iOS phones) has further catalyzed research in this direction. The problem we consider is minimizing the average of local smooth, nonconvex functions. Each node has access only to its own loss function, but can communicate with the server, which aggregates updates from all the nodes, before distributing them to all the nodes. With the advent of more and more complex neural network architectures, these updates can be high dimensional. To save resources, the problem needs to be solved via communication-efficient approaches. We propose a novel algorithm, which combines the idea of variance-reduction, with the paradigm of carrying out multiple local updates at each node before averaging. We prove the convergence of the approach to a first-order stationary point. Our algorithm is optimal in terms of computation, and state-of-the-art in terms of the communication requirements. Lastly in Chapter 5, we consider the situation when the nodes do not have access to function gradients, and need to minimize the loss function using only function values. This problem lies in the domain of zeroth-order optimization. For simplicity of analysis, we study this problem only in the single-node case. This problem finds application in simulation-based optimization, and adversarial example generation for attacking deep neural networks. We propose a novel function value based gradient estimator, which has better variance, and better query-efficiency compared to existing estimators. The proposed estimator covers the most commonly used existing estimators as special cases. We conduct a comprehensive convergence analysis under different conditions. We also demonstrate its effectiveness through a real-world application to generating adversarial examples from a black-box deep neural network