Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Collaborative Information Processing in Wireless Sensor Networks for Diffusive Source Estimation

In this dissertation, we address the issue of collaborative information processing for diffusive source parameter estimation using wireless sensor networks (WSNs) capable of sensing in dispersive medium/environment, from signal processing perspective. We begin the dissertation by focusing on the mathematical formulation of a special diffusion phenomenon, i.e., an underwater oil spill, along with statistical algorithms for meaningful analysis of sensor data leading to efficient estimation of desired parameters of interest. The objective is to obtain an analytical solution to the problem, rather than using non-model based sophisticated numerical techniques. We tried to make the physical diffusion model as much appropriate as possible, while maintaining some pragmatic and reasonable assumptions for the simplicity of exposition and analytical derivation. The dissertation studies both source localization and tracking for static and moving diffusive sources respectively. For static diffusive source localization, we investigate two parametric estimation techniques based on the maximum-likelihood (ML) and the best linear unbiased estimator (BLUE) for a special case of our obtained physical dispersion model. We prove the consistency and asymptotic normality of the obtained ML solution when the number of sensor nodes and samples approach infinity, and derive the Cramer-Rao lower bound (CRLB) on its performance. In case of a moving diffusive source, we propose a particle filter (PF) based target tracking scheme for moving diffusive source, and analytically derive the posterior Cramer-Rao lower bound (PCRLB) for the moving source state estimates as a theoretical performance bound. Further, we explore nonparametric, machine learning based estimation technique for diffusive source parameter estimation using Dirichlet process mixture model (DPMM). Since real data are often complicated, no parametric model is suitable. As an alternative, we exploit the rich tools of nonparametric Bayesian methods, in particular the DPMM, which provides us with a flexible and data-driven estimation process. We propose DPMM based static diffusive source localization algorithm and provide analytical proof of convergence. The proposed algorithm is also extended to the scenario when multiple diffusive sources of same kind are present in the diffusive field of interest. Efficient power allocation can play an important role in extending the lifetime of a resource constrained WSN. Resource-constrained WSNs rely on collaborative signal and information processing for efficient handling of large volumes of data collected by the sensor nodes. In this dissertation, the problem of collaborative information processing for sequential parameter estimation in a WSN is formulated in a cooperative game-theoretic framework, which addresses the issue of fair resource allocation for estimation task at the Fusion center (FC). The framework allows addressing either resource allocation or commitment for information processing as solutions of cooperative games with underlying theoretical justifications. Different solution concepts found in cooperative games, namely, the Shapley function and Nash bargaining are used to enforce certain kinds of fairness among the nodes in a WSN

Applications of Repeated Games in Wireless Networks: A Survey

A repeated game is an effective tool to model interactions and conflicts for players aiming to achieve their objectives in a long-term basis. Contrary to static noncooperative games that model an interaction among players in only one period, in repeated games, interactions of players repeat for multiple periods; and thus the players become aware of other players' past behaviors and their future benefits, and will adapt their behavior accordingly. In wireless networks, conflicts among wireless nodes can lead to selfish behaviors, resulting in poor network performances and detrimental individual payoffs. In this paper, we survey the applications of repeated games in different wireless networks. The main goal is to demonstrate the use of repeated games to encourage wireless nodes to cooperate, thereby improving network performances and avoiding network disruption due to selfish behaviors. Furthermore, various problems in wireless networks and variations of repeated game models together with the corresponding solutions are discussed in this survey. Finally, we outline some open issues and future research directions.Comment: 32 pages, 15 figures, 5 tables, 168 reference

Game Theory for Multi-Access Edge Computing:Survey, Use Cases, and Future Trends

Game theory (GT) has been used with significant success to formulate, and either design or optimize, the operation of many representative communications and networking scenarios. The games in these scenarios involve, as usual, diverse players with conflicting goals. This paper primarily surveys the literature that has applied theoretical games to wireless networks, emphasizing use cases of upcoming multiaccess edge computing (MEC). MEC is relatively new and offers cloud services at the network periphery, aiming to reduce service latency backhaul load, and enhance relevant operational aspects such as quality of experience or security. Our presentation of GT is focused on the major challenges imposed by MEC services over the wireless resources. The survey is divided into classical and evolutionary games. Then, our discussion proceeds to more specific aspects which have a considerable impact on the game's usefulness, namely, rational versus evolving strategies, cooperation among players, available game information, the way the game is played (single turn, repeated), the game's model evaluation, and how the model results can be applied for both optimizing resource-constrained resources and balancing diverse tradeoffs in real edge networking scenarios. Finally, we reflect on lessons learned, highlighting future trends and research directions for applying theoretical model games in upcoming MEC services, considering both network design issues and usage scenarios