MDP-based MAC design with deterministic backoffs in virtualized 802.11 WLANs

This paper presents MAC protocols for a virtualized 802.11 network aiming to improve network performance and isolation among service providers (SPs). Taking into account the statistical properties of arrival traffic, a Markov Decision Process (MDP) is formulated to maximize the network throughput subject to SP reservations. By introducing the policy tree of the MDP, we present an optimal access policy. Each user can track this policy tree by carrier sensing and learn its transmission opportunity. As computational complexity of the policy tree grows exponentially with the total number of users, an efficient heuristic algorithm is proposed based on the MDP formulation where each user is assigned a deterministic backoff value. Numerical results show that performance of the proposed heuristic algorithm closely matches to the optimal policy. Moreover, both optimal and heuristic algorithms significantly improve TDMA and CSMA in terms of packet delivery ratio and isolation in unsaturated networks

Contention-based learning MAC protocol for broadcast Vehicle-to-Vehicle Communication

Vehicle-to-Vehicle Communication (V2V) is an upcoming technology that can enable safer, more efficient transportation via wireless connectivity among moving cars. The key enabling technology, specifying the physical and medium access control (MAC) layers of the V2V stack is IEEE 802.11p, which belongs in the IEEE 802.11 family of protocols originally designed for use in WLANs. V2V networks are formed on an ad hoc basis from vehicular stations that rely on the delivery of broadcast transmissions for their envisioned services and applications. Broadcast is inherently more sensitive to channel contention than unicast due to the MAC protocol’s inability to adapt to increased network traffic and colliding packets never being detected or recovered. This paper addresses this inherent scalability problem of the IEEE 802.11p MAC protocol. The density of the network can range from being very sparse to hundreds of stations contenting for access to the channel. A suitable MAC needs to offer the capacity for V2V exchanges even in such dense topologies which will be common in urban networks. We present a modified version of the IEEE 802.11p MAC based on Reinforcement Learning (RL), aiming to reduce the packet collision probability and bandwidth wastage. Implementation details regarding both the learning algorithm tuning and the networking side are provided. We also present simulation results regarding achieved message packet delivery and possible delay overhead of this solution. Our solution shows up to 70% increase in throughput compared to the standard IEEE 802.11p as the network traffic increases, while maintaining the transmission latency within the acceptable levels

Reconfigurable and traffic-aware MAC design for virtualized wireless networks via reinforcement learning

In this paper, we present a reconfigurable MAC scheme where the partition between contention-free and contention-based regimes in each frame is adaptive to the network status leveraging reinforcement learning. In particular, to support a virtualized wireless network consisting of multiple slices, each having heterogeneous and unsaturated devices, the proposed scheme aims to configure the partition for maximizing network throughput while maintaining the slice reservations. Applying complementary geometric programming (CGP) and monomial approximations, an iterative algorithm is developed to find the optimal solution. For a large number of devices, a scalable algorithm with lower computational complexity is also proposed. The partitioning algorithm requires the knowledge of the device traffic statistics. In the absence of such knowledge, we develop a learning algorithm employing Thompson sampling to acquire packet arrival probabilities of devices. Furthermore, we model the problem as a thresholding multi-armed bandit (TMAB) and propose a threshold-based reconfigurable MAC algorithm, which is proved to achieve the optimal regret bound

Intelligent and bandwidth-efficient medium access control protocols for IEEE 802.11p-based Vehicular Ad hoc Networks

Vehicle-to-Vehicle (V2V) technology aims to enable safer and more sophisticated transportation via the spontaneous formation of Vehicular Ad hoc Networks (VANETs). This type of wireless networks allows the exchange of kinematic and other data among vehicles, for the primary purpose of safer and more efficient driving, as well as efficient traffic management and other third-party services. Their infrastructure-less, unbounded nature allows the formation of dense networks that present a channel sharing issue, which is harder to tackle than in conventional WLANs. This thesis focuses on optimising channel access strategies, which is important for the efficient usage of the available wireless bandwidth and the successful deployment of VANETs. To start with, the default channel access control method for V2V is evaluated hardware via modifying the appropriate wireless interface Linux driver to enable finer on-the-fly control of IEEE 802.11p access control layer parameters. More complex channel sharing scenarios are evaluated via simulations and findings on the behaviour of the access control mechanism are presented. A complete channel sharing efficiency assessment is conducted, including throughput, fairness and latency measurements. A new IEEE 802.11p-compatible Q-Learning-based access control approach that improves upon the studied protocol is presented. The stations feature algorithms that “learn” how to act optimally in VANETs in order to maximise their achieved packet delivery and minimise bandwidth wastage. The feasibility of Q-Learning to be used as the base of selflearning protocols for IEEE 802.11p-based V2V communication access control in dense environments is investigated in terms of parameter tuning, necessary time of exploration, achieving latency requirements, scaling, multi-hop and accommodation of simultaneous applications. Additionally, the novel Collection Contention Estimation (CCE) mechanism for Q-Learning-based access control is presented. By embedding it on the Q-Learning agents, faster convergence, higher throughput, better service separation and short-term fairness are achieved in simulated network deployments. The acquired new insights on the network performance of the proposed algorithms can provide precise guidelines for efficient designs of practical, reliable, fair and ultra-low latency V2V communication systems for dense topologies. These results can potentially have an impact across a range of related areas, including various types of wireless networks and resource allocation for these, network protocol and transceiver design as well as QLearning applicability and considerations for correct use

MDP-Based MAC Design With Deterministic Backoffs in Virtualized 802.11 WLANs

(c) 2015 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other worksThis paper presents MAC protocols for a virtualized 802.11 network aiming to improve network performance and isolation among service providers (SPs). Taking into account the statistical properties of arrival traffic, a Markov Decision Process (MDP) is formulated to maximize the network throughput subject to SP reservations. By introducing the policy tree of the MDP, we present an optimal access policy. Each user can track this policy tree by carrier sensing and learn its transmission opportunity. As computational complexity of the policy tree grows exponentially with the total number of users, an efficient heuristic algorithm is proposed based on the MDP formulation where each user is assigned a deterministic backoff value. Numerical results show that performance of the proposed heuristic algorithm closely matches to the optimal policy. Moreover, both optimal and heuristic algorithms significantly improve TDMA and CSMA in terms of packet delivery ratio and isolation in unsaturated networks