Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Efficient radio resource management in next generation wireless networks

The current decade has witnessed a phenomenal growth in mobile wireless communication networks and subscribers. In 2015, mobile wireless devices and connections were reported to have grown to about 7.9 billion, exceeding human population. The explosive growth in mobile wireless communication network subscribers has created a huge demand for wireless network capacity, ubiquitous wireless network coverage, and enhanced Quality of Service (QoS). These demands have led to several challenging problems for wireless communication networks operators and designers. The Next Generation Wireless Networks (NGWNs) will support high mobility communications, such as communication in high-speed rails. Mobile users in such high mobility environment demand reliable QoS, however, such users are plagued with a poor signal-tonoise ratio, due to the high vehicular penetration loss, increased transmission outage and handover information overhead, leading to poor QoS provisioning for the networks' mobile users. Providing a reliable QoS for high mobility users remains one of the unique challenges for NGWNs. The increased wireless network capacity and coverage of NGWNs means that mobile communication users at the cell-edge should have enhanced network performance. However, due to path loss (path attenuation), interference, and radio background noise, mobile communication users at the cell-edge can experience relatively poor transmission channel qualities and subsequently forced to transmit at a low bit transmission rate, even when the wireless communication networks can support high bit transmission rate. Furthermore, the NGWNs are envisioned to be Heterogeneous Wireless Networks (HWNs). The NGWNs are going to be the integration platform of diverse homogeneous wireless communication networks for a convergent wireless communication network. The HWNs support single and multiple calls (group calls), simultaneously. Decision making is an integral core of radio resource management. One crucial decision making in HWNs is network selection. Network selection addresses the problem of how to select the best available access network for a given network user connection. For the integrated platform of HWNs to be truly seamless and efficient, a robust and stable wireless access network selection algorithm is needed. To meet these challenges for the different mobile wireless communication network users, the NGWNs will have to provide a great leap in wireless network capacity, coverage, QoS, and radio resource utilization. Moving wireless communication networks (mobile hotspots) have been proposed as a solution to providing reliable QoS to high mobility users. In this thesis, an Adaptive Thinning Mobility Aware (ATMA) Call Admission Control (CAC) algorithm for improving the QoS and radio resource utilization of the mobile hotspot networks, which are of critical importance for communicating nodes in moving wireless networks is proposed. The performance of proposed ATMA CAC scheme is investigated and compare it with the traditional CAC scheme. The ATMA scheme exploits the mobility events in the highspeed mobility communication environment and the calls (new and handoff calls) generation pattern to enhance the QoS (new call blocking and handoff call dropping probabilities) of the mobile users. The numbers of new and handoff calls in wireless communication networks are dynamic random processes that can be effectively modeled by the Continuous Furthermore, the NGWNs are envisioned to be Heterogeneous Wireless Networks (HWNs). The NGWNs are going to be the integration platform of diverse homogeneous wireless communication networks for a convergent wireless communication network. The HWNs support single and multiple calls (group calls), simultaneously. Decision making is an integral core of radio resource management. One crucial decision making in HWNs is network selection. Network selection addresses the problem of how to select the best available access network for a given network user connection. For the integrated platform of HWNs to be truly seamless and efficient, a robust and stable wireless access network selection algorithm is needed. To meet these challenges for the different mobile wireless communication network users, the NGWNs will have to provide a great leap in wireless network capacity, coverage, QoS, and radio resource utilization. Moving wireless communication networks (mobile hotspots) have been proposed as a solution to providing reliable QoS to high mobility users. In this thesis, an Adaptive Thinning Mobility Aware (ATMA) Call Admission Control (CAC) algorithm for improving the QoS and radio resource utilization of the mobile hotspot networks, which are of critical importance for communicating nodes in moving wireless networks is proposed