Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Coalition Formation Games for Distributed Cooperation Among Roadside Units in Vehicular Networks

Vehicle-to-roadside (V2R) communications enable vehicular networks to support a wide range of applications for enhancing the efficiency of road transportation. While existing work focused on non-cooperative techniques for V2R communications between vehicles and roadside units (RSUs), this paper investigates novel cooperative strategies among the RSUs in a vehicular network. We propose a scheme whereby, through cooperation, the RSUs in a vehicular network can coordinate the classes of data being transmitted through V2R communications links to the vehicles. This scheme improves the diversity of the information circulating in the network while exploiting the underlying content-sharing vehicle-to-vehicle communication network. We model the problem as a coalition formation game with transferable utility and we propose an algorithm for forming coalitions among the RSUs. For coalition formation, each RSU can take an individual decision to join or leave a coalition, depending on its utility which accounts for the generated revenues and the costs for coalition coordination. We show that the RSUs can self-organize into a Nash-stable partition and adapt this partition to environmental changes. Simulation results show that, depending on different scenarios, coalition formation presents a performance improvement, in terms of the average payoff per RSU, ranging between 20.5% and 33.2%, relative to the non-cooperative case.Comment: accepted and to appear in IEEE Journal on Selected Areas in Communications (JSAC), Special issue on Vehicular Communications and Network

Game Theoretic Approaches to Massive Data Processing in Wireless Networks

Wireless communication networks are becoming highly virtualized with two-layer hierarchies, in which controllers at the upper layer with tasks to achieve can ask a large number of agents at the lower layer to help realize computation, storage, and transmission functions. Through offloading data processing to the agents, the controllers can accomplish otherwise prohibitive big data processing. Incentive mechanisms are needed for the agents to perform the controllers' tasks in order to satisfy the corresponding objectives of controllers and agents. In this article, a hierarchical game framework with fast convergence and scalability is proposed to meet the demand for real-time processing for such situations. Possible future research directions in this emerging area are also discussed