Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Combined Coverage Area Reporting and Geographical Routing in Wireless Sensor-Actuator Networks for Cooperating with Unmanned Aerial Vehicles

In wireless sensor network (WSN) applications with multiple gateways, it is key to route location dependent subscriptions efficiently at two levels in the system. At the gateway level, data sinks must not waste the energy of the WSN by injecting subscriptions that are not relevant for the nodes in their coverage area and at WSN level, energy-efficient delivery of subscriptions to target areas is required. In this paper, we propose a mechanism in which (1) the WSN provides an accurate and up-to-date coverage area description to gateways and (2) the wireless sensor network re-uses the collected coverage area information to enable efficient geographical routing of location dependent subscriptions and other messages. The latter has a focus on routing of messages injected from sink nodes to nodes in the region of interest. Our proposed mechanisms are evaluated in simulation

Energy Efficient Node Deployment in Wireless Ad-hoc Sensor Networks

We study a wireless ad-hoc sensor network (WASN) where $N$ sensors gather data from the surrounding environment and transmit their sensed information to $M$ fusion centers (FCs) via multi-hop wireless communications. This node deployment problem is formulated as an optimization problem to make a trade-off between the sensing uncertainty and energy consumption of the network. Our primary goal is to find an optimal deployment of sensors and FCs to minimize a Lagrange combination of the sensing uncertainty and energy consumption. To support arbitrary routing protocols in WASNs, the routing-dependent necessary conditions for the optimal deployment are explored. Based on these necessary conditions, we propose a routing-aware Lloyd algorithm to optimize node deployment. Simulation results show that, on average, the proposed algorithm outperforms the existing deployment algorithms.Comment: 7 pages, 6 figure