Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Coverage Protocols for Wireless Sensor Networks: Review and Future Directions

The coverage problem in wireless sensor networks (WSNs) can be generally defined as a measure of how effectively a network field is monitored by its sensor nodes. This problem has attracted a lot of interest over the years and as a result, many coverage protocols were proposed. In this survey, we first propose a taxonomy for classifying coverage protocols in WSNs. Then, we classify the coverage protocols into three categories (i.e. coverage aware deployment protocols, sleep scheduling protocols for flat networks, and cluster-based sleep scheduling protocols) based on the network stage where the coverage is optimized. For each category, relevant protocols are thoroughly reviewed and classified based on the adopted coverage techniques. Finally, we discuss open issues (and recommend future directions to resolve them) associated with the design of realistic coverage protocols. Issues such as realistic sensing models, realistic energy consumption models, realistic connectivity models and sensor localization are covered

Lifetime Maximization of Wireless Sensor Networks with a Mobile Source Node

We study the problem of routing in sensor networks where the goal is to maximize the network's lifetime. Previous work has considered this problem for fixed-topology networks. Here, we add mobility to the source node, which requires a new definition of the network lifetime. In particular, we redefine lifetime to be the time until the source node depletes its energy. When the mobile node's trajectory is unknown in advance, we formulate three versions of an optimal control problem aiming at this lifetime maximization. We show that in all cases, the solution can be reduced to a sequence of Non- Linear Programming (NLP) problems solved on line as the source node trajectory evolves.Comment: A shorter version of this work will be published in Proceedings of 2016 IEEE Conference on Decision and Contro

Energy Harvesting Wireless Communications: A Review of Recent Advances

This article summarizes recent contributions in the broad area of energy harvesting wireless communications. In particular, we provide the current state of the art for wireless networks composed of energy harvesting nodes, starting from the information-theoretic performance limits to transmission scheduling policies and resource allocation, medium access and networking issues. The emerging related area of energy transfer for self-sustaining energy harvesting wireless networks is considered in detail covering both energy cooperation aspects and simultaneous energy and information transfer. Various potential models with energy harvesting nodes at different network scales are reviewed as well as models for energy consumption at the nodes.Comment: To appear in the IEEE Journal of Selected Areas in Communications (Special Issue: Wireless Communications Powered by Energy Harvesting and Wireless Energy Transfer

Wireless Backhaul Node Placement for Small Cell Networks

Small cells have been proposed as a vehicle for wireless networks to keep up with surging demand. Small cells come with a significant challenge of providing backhaul to transport data to(from) a gateway node in the core network. Fiber based backhaul offers the high rates needed to meet this requirement, but is costly and time-consuming to deploy, when not readily available. Wireless backhaul is an attractive option for small cells as it provides a less expensive and easy-to-deploy alternative to fiber. However, there are multitude of bands and features (e.g. LOS/NLOS, spatial multiplexing etc.) associated with wireless backhaul that need to be used intelligently for small cells. Candidate bands include: sub-6 GHz band that is useful in non-line-of-sight (NLOS) scenarios, microwave band (6-42 GHz) that is useful in point-to-point line-of-sight (LOS) scenarios, and millimeter wave bands (e.g. 60, 70 and 80 GHz) that are recently being commercially used in LOS scenarios. In many deployment topologies, it is advantageous to use aggregator nodes, located at the roof tops of tall buildings near small cells. These nodes can provide high data rate to multiple small cells in NLOS paths, sustain the same data rate to gateway nodes using LOS paths and take advantage of all available bands. This work performs the joint cost optimal aggregator node placement, power allocation, channel scheduling and routing to optimize the wireless backhaul network. We formulate mixed integer nonlinear programs (MINLP) to capture the different interference and multiplexing patterns at sub-6 GHz and microwave band. We solve the MINLP through linear relaxation and branch-and-bound algorithm and apply our algorithm in an example wireless backhaul network of downtown Manhattan.Comment: Invited paper at Conference on Information Science & Systems (CISS) 201