Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Joint spectrum sensing and access for stable dynamic spectrum aggregation

International audienc

Wideband Autonomous Cognitive Radios: Spectrum Awareness and PHY/MAC Decision Making

The cognitive radios (CRs) have opened up new ways of better utilizing the scarce wireless spectrum resources. The CRs have been made feasible by recent advances in software-defined radios (SDRs), smart antennas, reconfigurable radio frequency (RF) front-ends, and full-duplex RF front-end architectures, to name a few. Generally, a CR is considered as a dynamically reconfigurable radio capable of adapting its operating parameters to the surrounding environment. Recent developments in spectrum policy and regulatory domains also allow more flexible and efficient utilization of wider RF spectrum range in the future. In line with the future directions of CRs, a new vision of a future autonomous CR device, called Radiobots, was previously proposed. The goals of the proposed Radiobot surpass the dynamic spectrum access (DSA) to achieve wideband operability and the main features of cognition. In order to ensure the practicality and robust operation of the Radiobot structure, the research focus of this dissertation includes the following aspects: 1) robust spectrum sensing and operability in a centralized CR network setup; 2) robust multivariate non-parametric quickest detection for dynamic spectrum usage tracking in an alien RF environment; 3) joint physical layer and medium access control layer (PHY/MAC) decision-making for wideband bandwidth aggregation (simultaneous operation over multiple modes/networks); and 4) autonomous spectrum sensing scheduling solutions in an alien ultra wideband RF environment. The major contribution of this dissertation is to investigate the feasibility of the autonomous CR operation in heterogeneous RF environments, and to provide novel solutions to the fundamental and crucial problems/challenges, including spectrum sensing, spectrum awareness, wideband operability, and autonomous PHY/MAC protocols, thus bringing the autonomous Radiobot one step closer to reality

Cognitive MAC protocols for mobile Ad-Hoc networks

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The term of Cognitive Radio (CR) used to indicate that spectrum radio could be accessed dynamically and opportunistically by unlicensed users. In CR Networks, Interference between nodes, hidden terminal problem, and spectrum sensing errors are big issues to be widely discussed in the research field nowadays. To improve the performance of such kind of networks, this thesis proposes Cognitive Medium Access Control (MAC) protocols for Mobile Ad-Hoc Networks (MANETs). From the concept of CR, this thesis has been able to develop a cognitive MAC framework in which a cognitive process consisting of cognitive elements is considered, which can make efficient decisions to optimise the CR network. In this context, three different scenarios to maximize the secondary user's throughput have been proposed. We found that the throughput improvement depends on the transition probabilities. However, considering the past information state of the spectrum can dramatically increases the secondary user's throughput by up to 40%. Moreover, by increasing the number of channels, the throughput of the network can be improved about 25%. Furthermore, to study the impact of Physical (PHY) Layer errors on cognitive MAC layer in MANETs, in this thesis, a Sensing Error-Aware MAC protocols for MANETs has been proposed. The developed model has been able to improve the MAC layer performance under the challenge of sensing errors. In this context, the proposed model examined two sensing error probabilities: the false alarm probability and the missed detection probability. The simulation results have shown that both probabilities could be adapted to maintain the false alarm probability at certain values to achieve good results. Finally, in this thesis, a cooperative sensing scheme with interference mitigation for Cognitive Wireless Mesh Networks (CogMesh) has been proposed. Moreover, a prioritybased traffic scenario to analyze the problem of packet delay and a novel technique for dynamic channel allocation in CogMesh is presented. Considering each channel in the system as a sub-server, the average delay of the users' packets is reduced and the cooperative sensing scenario dramatically increases the network throughput 50% more as the number of arrival rate is increased

Joint relay scheduling, channel access, and power allocation for green cognitive radio communications

PublishedJournal Article© 1983-2012 IEEE. The capacity of cognitive radio (CR) systems can be enhanced significantly by deploying relay nodes to exploit the spatial diversity. However, the inevitable imperfect sensing in CR has vital effects on the policy of relay selection, channel access, and power allocation that play pivotal roles in the system capacity. The increase in transmission power can improve the system capacity, but results in high energy consumption, which incurs the increase of carbon emission and network operational cost. Most of the existing schemes for CR systems have not jointly considered the imperfect sensing scenario and the tradeoff between the system capacity and energy consumption. To fill in this gap, this paper proposes an energy-aware centralized relay selection scheme that takes into account the relay selection, channel access, and power allocation jointly in CR with imperfect sensing. Specifically, the CR system is formulated as a partially observable Markov decision process (POMDP) to achieve the goal of balancing the system capacity and energy consumption as well as maximizing the system reward. The optimal policy for relay selection, channel access, and power allocation is then derived by virtue of a dynamic programming approach. A dimension reduction strategy is further applied to reduce its high computation complexity. Extensive simulation experiments and results are presented and analysed to demonstrate the significant performance improvement compared to the existing schemes. The performance results show that the received reward increases more than 50% and the network lifetime increases more than 35%, but the system capacity is reduced less than 6% only.This work was supported by the National Natural Science Foundation of China under Grants 61201219, 61171111, 61472150, and 61173045 and in part by the Fundamental Research Funds for the Central Universities under Grant 2013QN122