Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Dynamic Packet Scheduling in Wireless Networks

We consider protocols that serve communication requests arising over time in a wireless network that is subject to interference. Unlike previous approaches, we take the geometry of the network and power control into account, both allowing to increase the network's performance significantly. We introduce a stochastic and an adversarial model to bound the packet injection. Although taken as the primary motivation, this approach is not only suitable for models based on the signal-to-interference-plus-noise ratio (SINR). It also covers virtually all other common interference models, for example the multiple-access channel, the radio-network model, the protocol model, and distance-2 matching. Packet-routing networks allowing each edge or each node to transmit or receive one packet at a time can be modeled as well. Starting from algorithms for the respective scheduling problem with static transmission requests, we build distributed stable protocols. This is more involved than in previous, similar approaches because the algorithms we consider do not necessarily scale linearly when scaling the input instance. We can guarantee a throughput that is as large as the one of the original static algorithm. In particular, for SINR models the competitive ratios of the protocol in comparison to optimal ones in the respective model are between constant and O(log^2 m) for a network of size m.Comment: 23 page

Energy and bursty packet loss tradeoff over fading channels: a system-level model

Energy efficiency and quality of service (QoS) guarantees are the key design goals for the 5G wireless communication systems. In this context, we discuss a multiuser scheduling scheme over fading channels for loss tolerant applications. The loss tolerance of the application is characterized in terms of different parameters that contribute to quality of experience (QoE) for the application. The mobile users are scheduled opportunistically such that a minimum QoS is guaranteed. We propose an opportunistic scheduling scheme and address the cross-layer design framework when channel state information (CSI) is not perfectly available at the transmitter and the receiver. We characterize the system energy as a function of different QoS and channel state estimation error parameters. The optimization problem is formulated using Markov chain framework and solved using stochastic optimization techniques. The results demonstrate that the parameters characterizing the packet loss are tightly coupled and relaxation of one parameter does not benefit the system much if the other constraints are tight. We evaluate the energy-performance tradeoff numerically and show the effect of channel uncertainty on the packet scheduler design

Energy Efficient Multiuser Scheduling: Statistical Guarantees on Bursty Packet Loss

In this paper, we consider energy efficient multiuser scheduling. Packet loss tolerance of the applications is exploited to minimize average system energy. There is a constraint on average packet drop rate and maximum number of packets dropped successively (bursty loss). A finite buffer size is assumed. We propose a scheme which schedules the users opportunistically according to the channel conditions, packet loss constraints and buffer size parameters. We assume imperfect channel state information at the transmitter side and analyze the scheme in large user limit using stochastic optimization techniques. First, we optimize system energy for a fixed buffer size which results in a corresponding statistical guarantee on successive packet drop. Then, we determine the minimum buffer size to achieve a target (improved) energy efficiency for the same (or better) statistical guarantee. We show that buffer size can be traded effectively to achieve system energy efficiency for target statistical guarantees on packet loss parameters.Comment: Proc. Physcomnet in conjunction with WIOPT 201