Dimmer: Self-Adaptive Network-Wide Flooding with Reinforcement Learning

The last decade saw an emergence of Synchronous Transmissions (ST) as an effective communication paradigm in low-power wireless networks. Numerous ST protocols provide high reliability and energy efficiency in normal wireless conditions, for a large variety of traffic requirements. Recently, with the EWSN dependability competitions, the community pushed ST to harsher and highly-interfered environments, improving upon classical ST protocols through the use of custom rules, hand-tailored parameters, and additional retransmissions. The results are sophisticated protocols, that require prior expert knowledge and extensive testing, often tuned for a specific deployment and envisioned scenario. In this paper, we explore how ST protocols can benefit from self-adaptivity; a self-adaptive ST protocol selects itself its best parameters to (1) tackle external environment dynamics and (2) adapt to its topology over time. We introduce Dimmer as a self-adaptive ST protocol. Dimmer builds on LWB and uses Reinforcement Learning to tune its parameters and match the current properties of the wireless medium. By learning how to behave from an unlabeled dataset, Dimmer adapts to different interference types and patterns, and is able to tackle previously unseen interference. With Dimmer, we explore how to efficiently design AI-based systems for constrained devices, and outline the benefits and downfalls of AI-based low-power networking. We evaluate our protocol on two deployments of resource-constrained nodes achieving 95.8% reliability against strong, unknown WiFi interference. Our results outperform baselines such as non-adaptive ST protocols (27%) and PID controllers, and show a performance close to hand-crafted and more sophisticated solutions, such as Crystal (99%)

Reinforcement-based data transmission in temporally-correlated fading channels: Partial CSIT scenario

Reinforcement algorithms refer to the schemes where the results of the previous trials and a reward-punishment rule are used for parameter setting in the next steps. In this paper, we use the concept of reinforcement algorithms to develop different data transmission models in wireless networks. Considering temporally-correlated fading channels, the results are presented for the cases with partial channel state information at the transmitter (CSIT). As demonstrated, the implementation of reinforcement algorithms improves the performance of communication setups remarkably, with the same feedback load/complexity as in the state-of-the-art schemes.Comment: Accepted for publication in ISWCS 201

Minimizing Age of Collection for Multiple Access in Wireless Industrial Internet of Things

This paper investigates the information freshness of Industrial Internet of Things (IIoT) systems, where each IoT device makes a partial observation of a common target and transmits the information update to a central receiver to recover the complete observation. We consider the age of collection (AoC) performance as a measure of information freshness. Unlike the conventional age of information (AoI) metric, the instantaneous AoC decreases only when all cooperative packets for a common observation are successfully received. Hence, effectively allocating wireless time-frequency resources among IoT devices to achieve a low average AoC at the central receiver is paramount. Three multiple access schemes are considered in this paper: time-division multiple access (TDMA) without retransmission, TDMA with retransmission, and frequency-division multiple access (FDMA). First, our theoretical analysis indicates that TDMA with retransmission outperforms the other two schemes in terms of average AoC. Subsequently, we implement information update systems based on the three schemes on software-defined radios. Experimental results demonstrate that considering the medium access control (MAC) overhead in practice, FDMA achieves a lower average AoC than TDMA with or without retransmission in the high signal-to-noise ratio (SNR) regime. In contrast, TDMA with retransmission provides a stable and relatively low average AoC over a wide SNR range, which is favorable for IIoT applications. Overall, we present a theoretical-plus-experimental investigation of AoC in IIoT information update systems

Self-Learning Power Control in Wireless Sensor Networks

Current trends in interconnecting myriad smart objects to monetize on Internet of Things applications have led to high-density communications in wireless sensor networks. This aggravates the already over-congested unlicensed radio bands, calling for new mechanisms to improve spectrum management and energy efficiency, such as transmission power control. Existing protocols are based on simplistic heuristics that often approach interference problems (i.e., packet loss, delay and energy waste) by increasing power, leading to detrimental results. The scope of this work is to investigate how machine learning may be used to bring wireless nodes to the lowest possible transmission power level and, in turn, to respect the quality requirements of the overall network. Lowering transmission power has benefits in terms of both energy consumption and interference. We propose a protocol of transmission power control through a reinforcement learning process that we have set in a multi-agent system. The agents are independent learners using the same exploration strategy and reward structure, leading to an overall cooperative network. The simulation results show that the system converges to an equilibrium where each node transmits at the minimum power while respecting high packet reception ratio constraints. Consequently, the system benefits from low energy consumption and packet delay

Intelligent and bandwidth-efficient medium access control protocols for IEEE 802.11p-based Vehicular Ad hoc Networks

Vehicle-to-Vehicle (V2V) technology aims to enable safer and more sophisticated transportation via the spontaneous formation of Vehicular Ad hoc Networks (VANETs). This type of wireless networks allows the exchange of kinematic and other data among vehicles, for the primary purpose of safer and more efficient driving, as well as efficient traffic management and other third-party services. Their infrastructure-less, unbounded nature allows the formation of dense networks that present a channel sharing issue, which is harder to tackle than in conventional WLANs. This thesis focuses on optimising channel access strategies, which is important for the efficient usage of the available wireless bandwidth and the successful deployment of VANETs. To start with, the default channel access control method for V2V is evaluated hardware via modifying the appropriate wireless interface Linux driver to enable finer on-the-fly control of IEEE 802.11p access control layer parameters. More complex channel sharing scenarios are evaluated via simulations and findings on the behaviour of the access control mechanism are presented. A complete channel sharing efficiency assessment is conducted, including throughput, fairness and latency measurements. A new IEEE 802.11p-compatible Q-Learning-based access control approach that improves upon the studied protocol is presented. The stations feature algorithms that “learn” how to act optimally in VANETs in order to maximise their achieved packet delivery and minimise bandwidth wastage. The feasibility of Q-Learning to be used as the base of selflearning protocols for IEEE 802.11p-based V2V communication access control in dense environments is investigated in terms of parameter tuning, necessary time of exploration, achieving latency requirements, scaling, multi-hop and accommodation of simultaneous applications. Additionally, the novel Collection Contention Estimation (CCE) mechanism for Q-Learning-based access control is presented. By embedding it on the Q-Learning agents, faster convergence, higher throughput, better service separation and short-term fairness are achieved in simulated network deployments. The acquired new insights on the network performance of the proposed algorithms can provide precise guidelines for efficient designs of practical, reliable, fair and ultra-low latency V2V communication systems for dense topologies. These results can potentially have an impact across a range of related areas, including various types of wireless networks and resource allocation for these, network protocol and transceiver design as well as QLearning applicability and considerations for correct use