Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Machine Learning-Enabled Resource Allocation for Underlay Cognitive Radio Networks

Due to the rapid growth of new wireless communication services and applications, much attention has been directed to frequency spectrum resources and the way they are regulated. Considering that the radio spectrum is a natural limited resource, supporting the ever increasing demands for higher capacity and higher data rates for diverse sets of users, services and applications is a challenging task which requires innovative technologies capable of providing new ways of efficiently exploiting the available radio spectrum. Consequently, dynamic spectrum access (DSA) has been proposed as a replacement for static spectrum allocation policies. The DSA is implemented in three modes including interweave, overlay and underlay mode [1]. The key enabling technology for DSA is cognitive radio (CR), which is among the core prominent technologies for the next generation of wireless communication systems. Unlike conventional radio which is restricted to only operate in designated spectrum bands, a CR has the capability to operate in different spectrum bands owing to its ability in sensing, understanding its wireless environment, learning from past experiences and proactively changing the transmission parameters as needed. These features for CR are provided by an intelligent software package called the cognitive engine (CE). In general, the CE manages radio resources to accomplish cognitive functionalities and allocates and adapts the radio resources to optimize the performance of the network. Cognitive functionality of the CE can be achieved by leveraging machine learning techniques. Therefore, this thesis explores the application of two machine learning techniques in enabling the cognition capability of CE. The two considered machine learning techniques are neural network-based supervised learning and reinforcement learning. Specifically, this thesis develops resource allocation algorithms that leverage the use of machine learning techniques to find the solution to the resource allocation problem for heterogeneous underlay cognitive radio networks (CRNs). The proposed algorithms are evaluated under extensive simulation runs. The first resource allocation algorithm uses a neural network-based learning paradigm to present a fully autonomous and distributed underlay DSA scheme where each CR operates based on predicting its transmission effect on a primary network (PN). The scheme is based on a CE with an artificial neural network that predicts the adaptive modulation and coding configuration for the primary link nearest to a transmitting CR, without exchanging information between primary and secondary networks. By managing the effect of the secondary network (SN) on the primary network, the presented technique maintains the relative average throughput change in the primary network within a prescribed maximum value, while also finding transmit settings for the CRs that result in throughput as large as allowed by the primary network interference limit. The second resource allocation algorithm uses reinforcement learning and aims at distributively maximizing the average quality of experience (QoE) across transmission of CRs with different types of traffic while satisfying a primary network interference constraint. To best satisfy the QoE requirements of the delay-sensitive type of traffics, a cross-layer resource allocation algorithm is derived and its performance is compared against a physical-layer algorithm in terms of meeting end-to-end traffic delay constraints. Moreover, to accelerate the learning performance of the presented algorithms, the idea of transfer learning is integrated. The philosophy behind transfer learning is to allow well-established and expert cognitive agents (i.e. base stations or mobile stations in the context of wireless communications) to teach newly activated and naive agents. Exchange of learned information is used to improve the learning performance of a distributed CR network. This thesis further identifies the best practices to transfer knowledge between CRs so as to reduce the communication overhead. The investigations in this thesis propose a novel technique which is able to accurately predict the modulation scheme and channel coding rate used in a primary link without the need to exchange information between the two networks (e.g. access to feedback channels), while succeeding in the main goal of determining the transmit power of the CRs such that the interference they create remains below the maximum threshold that the primary network can sustain with minimal effect on the average throughput. The investigations in this thesis also provide a physical-layer as well as a cross-layer machine learning-based algorithms to address the challenge of resource allocation in underlay cognitive radio networks, resulting in better learning performance and reduced communication overhead

Markov decision processes with applications in wireless sensor networks: A survey

Ministry of Education, Singapore under its Academic Research Funding Tier

Energy efficient resource allocation for future wireless communication systemsy

Next generation of wireless communication systems envisions a massive number of connected battery powered wireless devices. Replacing the battery of such devices is expensive, costly, or infeasible. To this end, energy harvesting (EH) is a promising technique to prolong the lifetime of such devices. Because of randomness in amount and availability of the harvested energy, existing communication techniques require revisions to address the issues specific to EH systems. In this thesis, we aim at revisiting fundamental wireless communication problems and addressing the future perspective on service based applications with the specific characteristics of the EH in mind. In the first part of the thesis, we address three fundamental problems that exist in the wireless communication systems, namely; multiple access strategy, overcoming the wireless channel, and providing reliability. Since the wireless channel is a shared medium, concurrent transmissions of multiple devices cause interference which results in collision and eventual loss of the transmitted data. Multiple access protocols aim at providing a coordination mechanism between multiple transmissions so as to enable a collision free medium. We revisit the random access protocol for its distributed and low energy characteristics while incorporating the statistical correlation of the EH processes across two transmitters. We design a simple threshold based policy which only allows transmission if the battery state is above a certain threshold. By optimizing the threshold values, we show that by carefully addressing the correlation information, the randomness can be turned into an opportunity in some cases providing optimal coordination between transmitters without any collisions. Upon accessing the channel, a wireless transmitter is faced with a transmission medium that exhibits random and time varying properties. A transmitter can adapt its transmission strategy to the specific state of the channel for an efficient transmission of information. This requires a process known as channel sensing to acquire the channel state which is costly in terms of time and energy. The contribution of the channel sensing operation to the energy consumption in EH wireless transmitters is not negligible and requires proper optimization. We developed an intelligent channel sensing strategy for an EH transmitter communicating over a time-correlated wireless channel. Our results demonstrate that, despite the associated time and energy cost, sensing the channel intelligently to track the channel state improves the achievable long-term throughput significantly as compared to the performance of those protocols lacking this ability as well as the one that always senses the channel. Next, we study an EH receiver employing Hybrid Automatic Repeat reQuest (HARQ) to ensure reliable end-to-end communications. In inherently error-prone wireless communications systems, re-transmissions triggered by decoding errors have a major impact on the energy consumption of wireless devices. We take into account the energy consumption induced by HARQ to develop simple-toimplement optimal algorithms that minimizes the number of retransmissions required to successfully decode the packet. The large number of connected edge devices envisioned in future wireless technologies enable a wide range of resources with significant sensing capabilities. The ability to collect various data from the sensors has enabled many exciting smart applications. Providing data at a certain quality greatly improves the performance of many of such applications. However, providing high quality is demanding for energy limited sensors. Thus, in the second part of the thesis, we optimize the sensing resolution of an EH wireless sensor in order to efficiently utilize the harvested energy to maximize an application dependent utilit

Explainable AI over the Internet of Things (IoT): Overview, State-of-the-Art and Future Directions

Explainable Artificial Intelligence (XAI) is transforming the field of Artificial Intelligence (AI) by enhancing the trust of end-users in machines. As the number of connected devices keeps on growing, the Internet of Things (IoT) market needs to be trustworthy for the end-users. However, existing literature still lacks a systematic and comprehensive survey work on the use of XAI for IoT. To bridge this lacking, in this paper, we address the XAI frameworks with a focus on their characteristics and support for IoT. We illustrate the widely-used XAI services for IoT applications, such as security enhancement, Internet of Medical Things (IoMT), Industrial IoT (IIoT), and Internet of City Things (IoCT). We also suggest the implementation choice of XAI models over IoT systems in these applications with appropriate examples and summarize the key inferences for future works. Moreover, we present the cutting-edge development in edge XAI structures and the support of sixth-generation (6G) communication services for IoT applications, along with key inferences. In a nutshell, this paper constitutes the first holistic compilation on the development of XAI-based frameworks tailored for the demands of future IoT use cases.Comment: 29 pages, 7 figures, 2 tables. IEEE Open Journal of the Communications Society (2022

Unmanned Aerial Vehicle (UAV)-Enabled Wireless Communications and Networking

The emerging massive density of human-held and machine-type nodes implies larger traffic deviatiolns in the future than we are facing today. In the future, the network will be characterized by a high degree of flexibility, allowing it to adapt smoothly, autonomously, and efficiently to the quickly changing traffic demands both in time and space. This flexibility cannot be achieved when the network’s infrastructure remains static. To this end, the topic of UAVs (unmanned aerial vehicles) have enabled wireless communications, and networking has received increased attention. As mentioned above, the network must serve a massive density of nodes that can be either human-held (user devices) or machine-type nodes (sensors). If we wish to properly serve these nodes and optimize their data, a proper wireless connection is fundamental. This can be achieved by using UAV-enabled communication and networks. This Special Issue addresses the many existing issues that still exist to allow UAV-enabled wireless communications and networking to be properly rolled out

Task-Oriented Communication Design in Cyber-Physical Systems: A Survey on Theory and Applications

Communication system design has been traditionally guided by task-agnostic principles, which aim at efficiently transmitting as many correct bits as possible through a given channel. However, in the era of cyber-physical systems, the effectiveness of communications is not dictated simply by the bit rate, but most importantly by the efficient completion of the task in hand, e.g., controlling remotely a robot, automating a production line or collaboratively sensing through a drone swarm. In parallel, it is projected that by 2023, half of the worldwide network connections will be among machines rather than humans. In this context, it is crucial to establish a new paradigm for designing communication strategies for multi-agent cyber-physical systems. This is a daunting task, since it requires a combination of principles from information, communication, control theories and computer science in order to formalize a general framework for task-oriented communication designs. In this direction, this paper reviews and structures the relevant theoretical work across a wide range of scientific communities. Subsequently, it proposes a general conceptual framework for task-oriented communication design, along with its specializations according to targeted use cases. Furthermore, it provides a survey of relevant contributions in dominant applications, such as industrial internet of things, multi-unmanned aerial vehicle (UAV) systems, autonomous vehicles, distributed learning systems, smart manufacturing plants, 5G and beyond self-organizing networks, and tactile internet. Finally, this paper also highlights the most important open research topics from both the theoretical framework and application points of view