Introducing reinforcement learning in the Wi-Fi MAC layer to support sustainable communications in e-Health scenarios

The crisis of energy supplies has led to the need for sustainability in technology, especially in the Internet of Things (IoT) paradigm. One solution is the integration of Energy Harvesting (EH) technologies into IoT systems, which reduces the amount of battery replacement. However, integrating EH technologies within IoT systems is challenging, and it requires adaptations at different layers of the IoT protocol stack, especially at Medium Access Control (MAC) layer due to its energy-hungry features. Since Wi-Fi is a widely used wireless technology in IoT systems, in this paper, we perform an extensive set of simulations in a dense solar-based energy-harvesting Wi-Fi network in an e-Health environment. We introduce optimization algorithms, which benefit from the Reinforcement Learning (RL) methods to efficiently adjust to the complexity and dynamic behaviour of the network. We assume the concept of Access Point (AP) coordination to demonstrate the feasibility of the upcoming Wi-Fi amendment IEEE 802.11bn (Wi-Fi 8). This paper shows that the proposed algorithms reduce the network&amp;#x2019;s energy consumption by up to 25% compared to legacy Wi-Fi while maintaining the required Quality of Service (QoS) for e-Health applications. Moreover, by considering the specific adjustment of MAC layer parameters, up to 37% of the energy of the network can be conserved, which illustrates the viability of reducing the dimensions of solar cells, while concurrently augmenting the flexibility of this EH technique for deployment within the IoT devices. We anticipate this research will shed light on new possibilities for IoT energy harvesting integration, particularly in contexts with restricted QoS environments such as e-Healthcare.</p

The Next Generation Intelligent Transportation System: Connected, Safe and Green

Modern Intelligent Transportation Systems (ITSs) employ communication technologies in order to ameliorate the passenger's commuting experience. Vehicular Networking lies at the core of inaugurating an efficient transportation system and aims at transforming vehicles into smart mobile entities that are able to sense their surroundings, collect information about the environment and communicate with each other as well as with Roadside Units (RSUs) deployed alongside roadways. As such, the novel communication paradigm of vehicular networking gave birth to an ITS that embraces a wide variety of applications including but not limited to: traffic management, passenger and road safety, environment monitoring and road surveillance, hot-spot guidance, Drive Thru Internet access, remote region connectivity, and so forth. Furthermore, with the rapid development of computation and communication technologies, the Internet of Vehicles (IoV) promises huge commercial interest and research value, thereby attracting a significant industrial and academic attention. This thesis studies and analyses fundamentally challenging problems in the context of vehicular environments and proposes new techniques targeting the improvement of the performance of ITSs envisioned to play a remarkable role in the IoV era. Unlike existing wireless mobile networks, vehicular networks possess unique characteristics, including high node mobility and a rapidly-changing topology, which should be carefully accounted for. Four major problems from the pool of existing vehicular networking persisting challenges will be addressed in this thesis, namely: a) establishing a connectivity path in a highly dynamic Vehicular Ad Hoc Network, b) examining the performance of Vehicle-to-Infrastructure communication Medium Access Control schemes, c) addressing the scheduling problem of a vehicular networking scenario encompassing an energy-limited RSU by exploiting machine learning techniques, particularly reinforcement learning, to train an agent to make appropriate decisions and develop a scheduling policy that prolongs the network's operational status and allows for acceptable Quality-of-Service levels and d) overcoming the limitations of reinforcement learning techniques in high-dimensional input scenarios by exploiting recent advances in deep learning in an effort to satisfy the driver's well-being as well as his demand for continuous connectivity in a green, balanced, connected and efficient vehicular network. These problems will be extensively studied throughout this thesis, followed by discussions that highlight open research directions worth further investigations