Reinforcement Learning Approaches to Improve Spatial Reuse in Wireless Local Area Networks

The ubiquitous deployment of IEEE 802.11 based Wireless Local Area Networks (WLANs) or WiFi networks has resulted in dense deployments of Access Points (APs) in an effort to provide wireless links with high data rates to users. This, however, causes APs and users/stations to experience a higher interference level. This is because of the limited spectrum in which WiFi networks operate, resulting in multiple APs operating on the same channel. This in turn affects the signal-tonoise-plus interference ratio (SINR) at APs and users, leading to low data rates that limit their quality of service (QoS). To improve QoS, interference management is critical. To this end, a key metric of interest is spatial reuse. A high spatial reuse means multiple transmissions are able to transmit concurrently, which leads to a high network capacity. One approach to optimize spatial reuse is by tuning the clear channel access (CCA) threshold employed by the carrier sense multiple access with collision avoidance (CSMA/CA) medium access control (MAC) protocol. Specifically, the CCA threshold of a node determines whether it is allowed to transmit after sensing the channel. A node may increase its CCA threshold, causing it to transmit even when there are other ongoing transmissions. Another parameter to be tuned is transmit power. This helps a transmitting node lower its interference to neighboring cells, and thus allows nodes in these neighboring cells to transmit as well. Apart from that, channel bonding can be applied to improve transmission rate. In particular, by combining/aggregating multiple channels together, the resulting channel has a proportionally higher data rate than the case without channel bonding. However, the issue of spatial reuse remains the same whereby the focus is to maximize the number of concurrent transmissions across multiple channels

Multi-Armed Bandits for Spectrum Allocation in Multi-Agent Channel Bonding WLANs

While dynamic channel bonding (DCB) is proven to boost the capacity of wireless local area networks (WLANs) by adapting the bandwidth on a per-frame basis, its performance is tied to the primary and secondary channel selection. Unfortunately, in uncoordinated high-density deployments where multiple basic service sets (BSSs) may potentially overlap, hand-crafted spectrum management techniques perform poorly given the complex hidden/exposed nodes interactions. To cope with such challenging Wi-Fi environments, in this paper, we first identify machine learning (ML) approaches applicable to the problem at hand and justify why model-free RL suits it the most. We then design a complete RL framework and call into question whether the use of complex RL algorithms helps the quest for rapid learning in realistic scenarios. Through extensive simulations, we derive that stateless RL in the form of lightweight multi-armed-bandits (MABs) is an efficient solution for rapid adaptation avoiding the definition of broad and/or meaningless states. In contrast to most current trends, we envision lightweight MABs as an appropriate alternative to the cumbersome and slowly convergent methods such as Q-learning, and especially, deep reinforcement learning

Quality of service differentiation for multimedia delivery in wireless LANs

Delivering multimedia content to heterogeneous devices over a variable networking environment while maintaining high quality levels involves many technical challenges. The research reported in this thesis presents a solution for Quality of Service (QoS)-based service differentiation when delivering multimedia content over the wireless LANs. This thesis has three major contributions outlined below: 1. A Model-based Bandwidth Estimation algorithm (MBE), which estimates the available bandwidth based on novel TCP and UDP throughput models over IEEE 802.11 WLANs. MBE has been modelled, implemented, and tested through simulations and real life testing. In comparison with other bandwidth estimation techniques, MBE shows better performance in terms of error rate, overhead, and loss. 2. An intelligent Prioritized Adaptive Scheme (iPAS), which provides QoS service differentiation for multimedia delivery in wireless networks. iPAS assigns dynamic priorities to various streams and determines their bandwidth share by employing a probabilistic approach-which makes use of stereotypes. The total bandwidth to be allocated is estimated using MBE. The priority level of individual stream is variable and dependent on stream-related characteristics and delivery QoS parameters. iPAS can be deployed seamlessly over the original IEEE 802.11 protocols and can be included in the IEEE 802.21 framework in order to optimize the control signal communication. iPAS has been modelled, implemented, and evaluated via simulations. The results demonstrate that iPAS achieves better performance than the equal channel access mechanism over IEEE 802.11 DCF and a service differentiation scheme on top of IEEE 802.11e EDCA, in terms of fairness, throughput, delay, loss, and estimated PSNR. Additionally, both objective and subjective video quality assessment have been performed using a prototype system. 3. A QoS-based Downlink/Uplink Fairness Scheme, which uses the stereotypes-based structure to balance the QoS parameters (i.e. throughput, delay, and loss) between downlink and uplink VoIP traffic. The proposed scheme has been modelled and tested through simulations. The results show that, in comparison with other downlink/uplink fairness-oriented solutions, the proposed scheme performs better in terms of VoIP capacity and fairness level between downlink and uplink traffic

Performance modelling of fairness in IEEE 802.11 wireless LAN protocols

PhD ThesisWireless communication has become a key technology in the modern world, allowing network services to be delivered in almost any environment, without the need for potentially expensive and invasive fixed cable solutions. However, the level of performance experienced by wireless devices varies tremendously on location and time. Understanding the factors which can cause variability of service is therefore of clear practical and theoretical interest. In this thesis we explore the performance of the IEEE 802.11 family of wireless protocols, which have become the de facto standard for Wireless Local Area Networks (WLANs). The specific performance issue which is investigated is the unfairness which can arise due to the spatial position of nodes in the network. In this work we characterise unfairness in terms of the difference in performance (e.g. throughput) experienced by different pairs of communicating nodes within a network. Models are presented using the Markovian process algebra PEPA which depict different scenarios with three of the main protocols, IEEE 802.11b, IEEE 802.11g and IEEE 802.11n. The analysis shows that performance is affected by the presence of other nodes (including in the well-known hidden node case), by the speed of data and the size of the frames being transmitted. The collection of models and analysis in this thesis collectively provides not only an insight into fairness in IEEE 802.11 networks, but it also represents a significant use case in modelling network protocols using PEPA. PEPA and other stochastic process algebra are extremely powerful tools for efficiently specifying models which might be very complex to study using conventional simulation approaches. Furthermore the tool support for PEPA facilitates the rapid solution of models to derive key metrics which enable the modeller to gain an understanding of the network behaviour across a wide range of operating conditions. From the results we can see that short frames promote a greater fairness due to the more frequent spaces between frames allowing other senders to transmit. An interesting consequence of these findings is the observation that varying frame length can play a role in addressing topological unfairness, which leads to the analysis of a novel model of IEEE 802.11g with variable frame lengths. While varying frame lengths might not always be practically possible, as frames need to be long enough for collisions to be detected, IEEE 802.11n supports a number of mechanisms for frame aggregation, where successive frames may be sent in series with little or no delay between them. We therefore present a novel model of IEEE 802.11n with frame aggregation to explore how this approach affects fairness and, potentially, can be used to address unfairness by allowing affected nodes to transmit longer frame bursts.Kurdistan Region Government of Iraq (KRG) sponso

Resource-Efficient Wireless Systems for Emerging Wireless Networks

As the wireless medium has become the primary source of communication and Internet connectivity, and as devices and wireless technologies become more sophisticated and capable, there has been a surge in the capacity demands and complexity of applications that run over these wireless devices. To sustain the volume and QoE guarantees of the data generated, the opportunity and need to rethink wireless network design across all the layers of the protocol stack has firmly emerged as a solution to enable the timely and reliable delivery of data, while handling the inherent challenges of a crowded wireless medium, such as congestion, interference, and hidden terminals. The research work presented in this dissertation builds efficient solutions and protocols with a theoretical foundation to address the challenges that arise in rethinking wireless network design. Example challenges include managing the overhead associated with complex systems. My work particularly focuses on the opportunities and challenges of sophisticated technology and systems in emerging wireless networks. I target the main thrusts in the evolution of wireless networks that create significant opportunity to achieve higher theoretical capacity, and have direct implications on our day-to-day wireless interactions: from enabling multifold increase in capacity in wireless physical links, to developing medium access techniques to exploit the high speed links, and making the applications more bandwidth efficient. I build deployable, and resource-aware wireless systems that exploit higher bandwidths by leveraging and advancing diverse research areas such as theory, analysis, protocol design, and wireless networking. Specifically, I identify the erroneous assumptions and fundamental limitations of existing solutions in capturing the true and complex interactions between wireless devices and protocols. I use these insights to guide practical and efficient protocol design, followed by thorough analysis and evaluation in testbed implementations via prototypes and measurements. I show that my proposed solutions achieve significant performance gains, at minimum cost to overhead

Wireless sensor networks

Wireless sensor networks promise an unprecedented fine-grained interface between the virtual and the physical world. They are one of the most rapidly developing new information technologies, with applications in a wide range of fields including industrial process control, security and surveillance, environmental sensing, and structural health monitoring. The subject of this project is motivated by the urgent need to provide a comprehensive and organized survey of the field. It shows how the core challenges of energy efficiency, robustness, and autonomy are addressed in these systems by networking techniques across multiple layers. The topics covered include network deployment, wireless characteristics, time synchronization, congestion and error control, medium access, standards, topology control, routing, security, data transfer, transport protocols and new technologies and materials in fabricating sensors

Enhancing spectrum utilization through cooperation and cognition in wireless systems

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, February 2013.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections."February 2013." Cataloged from student-submitted PDF version of thesis.Includes bibliographical references (p. 201-217).We have seen a proliferation of wireless technologies and devices in recent years. The resulting explosion of wireless demand has put immense pressure on available spectrum. Improving spectrum utilization is therefore necessary to enable wireless networks to keep up with burgeoning demand. This dissertation presents a cognitive and cooperative wireless architecture that significantly enhances spectrum utilization. Specifically, it introduces four new systems that embody a cross-layer design for cognition and cooperation. The first system, SWIFT, is a cognitive cross technology solution that enables wideband devices to exploit higher layer network semantics to adaptively sense which portions of the spectrum are occupied by unknown narrowband devices, and weave the remaining unoccupied spectrum bands into a single high-throughput wideband link. Second, FARA is a cooperative system that enables multi-channel wireless solutions like 802.11 to dynamically use all available channels for all devices in a performance-aware manner by using information from the physical layer and allocating to each link the frequency bands that show the highest performance for that link. SourceSync, the third system, enables wireless nodes in last-hop and wireless mesh networks to cooperatively transmit synchronously in order to exploit channel diversity and increase reliability. Finally, MegaMIMO enables wireless throughput to scale linearly with the number of transmitters by enabling multiple wireless transmitters to transmit simultaneously in the same frequency bands to multiple wireless receivers without interfering with each other. The systems in this dissertation demonstrate the practicality of cognitive and cooperative wireless systems to enable spectrum sharing. Further, as part of these systems, we design several novel primitives - adaptive spectrum sensing, time alignment, frequency synchronization, and distributed phase-coherent transmission, that can serve as fundamental building blocks for wireless cognition and cooperation. Finally, we have implemented all four systems described in this dissertation, and evaluated them in wireless testbeds, demonstrating large gains in practice.by Hariharan Shankar Rahul.Ph.D