Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Energy Management in RFID-Sensor Networks: Taxonomy and Challenges

Ubiquitous Computing is foreseen to play an important role for data production and network connectivity in the coming decades. The Internet of Things (IoT) research which has the capability to encapsulate identification potential and sensing capabilities, strives towards the objective of developing seamless, interoperable and securely integrated systems which can be achieved by connecting the Internet with computing devices. This gives way for the evolution of wireless energy harvesting and power transmission using computing devices. Radio Frequency (RF) based Energy Management (EM) has become the backbone for providing energy to wireless integrated systems. The two main techniques for EM in RFID Sensor Networks (RSN) are Energy Harvesting (EH) and Energy Transfer (ET). These techniques enable the dynamic energy level maintenance and optimisation as well as ensuring reliable communication which adheres to the goal of increased network performance and lifetime. In this paper, we present an overview of RSN, its types of integration and relative applications. We then provide the state-of-the-art EM techniques and strategies for RSN from August 2009 till date, thereby reviewing the existing EH and ET mechanisms designed for RSN. The taxonomy on various challenges for EM in RSN has also been articulated for open research directives

Performance Analysis for 5G cellular networks: Millimeter Wave and UAV Assisted Communications

Recent years have witnessed exponential growth in mobile data and traffic. Limited available spectrum in microwave ($\mu$Wave) bands does not seem to be capable of meeting this demand in the near future, motivating the move to new frequency bands. Therefore, operating with large available bandwidth at millimeter wave (mmWave) frequency bands, between 30 and 300 GHz, has become an appealing choice for the fifth generation (5G) cellular networks. In addition to mmWave cellular networks, the deployment of unmanned aerial vehicle (UAV) base stations (BSs), also known as drone BSs, has attracted considerable attention recently as a possible solution to meet the increasing data demand. UAV BSs are expected to be deployed in a variety of scenarios including public safety communications, data collection in Internet of Things (IoT) applications, disasters, accidents, and other emergencies and also temporary events requiring substantial network resources in the short-term. In these scenarios, UAVs can provide wireless connectivity rapidly. In this thesis, analytical frameworks are developed to analyze and evaluate the performance of mmWave cellular networks and UAV assisted cellular networks. First, the analysis of average symbol error probability (ASEP) in mmWave cellular networks with Poisson Point Process (PPP) distributed BSs is conducted using tools from stochastic geometry. Secondly, we analyze the energy efficiency of relay-assisted downlink mmWave cellular networks. Then, we provide an stochastic geometry framework to study heterogeneous downlink mmWave cellular networks consisting of $K$ tiers of randomly located BSs, assuming that each tier operates in a mmWave frequency band. We further study the uplink performance of the mmWave cellular networks by considering the coexistence of cellular and potential D2D user equipments (UEs) in the same band. In addition to mmWave cellular networks, the performance of UAV assisted cellular networks is also studied. Signal-to-interference-plus-noise ratio (SINR) coverage performance analysis for UAV assisted networks with clustered users is provided. Finally, we study the energy coverage performance of UAV energy harvesting networks with clustered users