Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

To Stay Or To Switch: Multiuser Dynamic Channel Access

In this paper we study opportunistic spectrum access (OSA) policies in a multiuser multichannel random access cognitive radio network, where users perform channel probing and switching in order to obtain better channel condition or higher instantaneous transmission quality. However, unlikely many prior works in this area, including those on channel probing and switching policies for a single user to exploit spectral diversity, and on probing and access policies for multiple users over a single channel to exploit temporal and multiuser diversity, in this study we consider the collective switching of multiple users over multiple channels. In addition, we consider finite arrivals, i.e., users are not assumed to always have data to send and demand for channel follow a certain arrival process. Under such a scenario, the users' ability to opportunistically exploit temporal diversity (the temporal variation in channel quality over a single channel) and spectral diversity (quality variation across multiple channels at a given time) is greatly affected by the level of congestion in the system. We investigate the optimal decision process in this case, and evaluate the extent to which congestion affects potential gains from opportunistic dynamic channel switching

On QoS-assured degraded provisioning in service-differentiated multi-layer elastic optical networks

The emergence of new network applications is driving network operators to not only fulfill dynamic bandwidth requirements, but offer various grades of service. Degraded provisioning provides an effective solution to flexibly allocate resources in various dimensions to reduce blocking for differentiated demands when network congestion occurs. In this work, we investigate the novel problem of online degraded provisioning in service-differentiated multi-layer networks with optical elasticity. Quality of Service (QoS) is assured by service-holding-time prolongation and immediate access as soon as the service arrives without set-up delay. We decompose the problem into degraded routing and degraded resource allocation stages, and design polynomial-time algorithms with the enhanced multi-layer architecture to increase the network flexibility in temporal and spectral dimensions. Illustrative results verify that we can achieve significant reduction of network service failures, especially for requests with higher priorities. The results also indicate that degradation in optical layer can increase the network capacity, while the degradation in electric layer provides flexible time-bandwidth exchange.Comment: accepted by IEEE GLOBECOM 201

Power-Optimal Feedback-Based Random Spectrum Access for an Energy Harvesting Cognitive User

In this paper, we study and analyze cognitive radio networks in which secondary users (SUs) are equipped with Energy Harvesting (EH) capability. We design a random spectrum sensing and access protocol for the SU that exploits the primary link's feedback and requires less average sensing time. Unlike previous works proposed earlier in literature, we do not assume perfect feedback. Instead, we take into account the more practical possibilities of overhearing unreliable feedback signals and accommodate spectrum sensing errors. Moreover, we assume an interference-based channel model where the receivers are equipped with multi-packet reception (MPR) capability. Furthermore, we perform power allocation at the SU with the objective of maximizing the secondary throughput under constraints that maintain certain quality-of-service (QoS) measures for the primary user (PU)