20 research outputs found
Self-organization for 5G and beyond mobile networks using reinforcement learning
The next generations of mobile networks 5G and beyond, must overcome current networks limitations as well as improve network performance.
Some of the requirements envisioned for future mobile networks are: addressing the massive growth required in coverage, capacity and traffic; providing better quality of service and experience to end users; supporting ultra high data rates and reliability; ensuring latency as low as one millisecond, among others.
Thus, in order for future networks to enable all of these stringent requirements, a promising concept has emerged, self organising networks (SONs).
SONs consist of making mobile networks more adaptive and autonomous and are divided in three main branches, depending on their use-cases, namely: self-configuration, self-optimisation, and self-healing.
SON is a very promising and broad concept, and in order to enable it, more intelligence needs to be embedded in the mobile network.
As such, one possible solution is the utilisation of machine learning (ML) algorithms.
ML has many branches, such as supervised, unsupervised and Reinforcement Learning (RL), and all can be used in different SON use-cases.
The objectives of this thesis are to explore different RL techniques in the context of SONs, more specifically in self-optimization use-cases.
First, the use-case of user-cell association in future heterogeneous networks is analysed and optimised.
This scenario considers not only Radio Access Network (RAN) constraints, but also in terms of the backhaul.
Based on this, a distributed solution utilizing RL is proposed and compared with other state-of-the-art methods.
Results show that the proposed RL algorithm outperforms current ones and is able to achieve better user satisfaction, while minimizing the number of users in outage.
Another objective of this thesis is the evaluation of Unmanned Aerial vehicles (UAVs) to optimize cellular networks.
It is envisioned that UAVs can be utilized in different SON use-cases and integrated with RL algorithms to determine their optimal 3D positions in space according to network constraints.
As such, two different mobile network scenarios are analysed, one emergency and a pop-up network.
The emergency scenario considers that a major natural disaster destroyed most of the ground network infrastructure and the goal is to provide coverage to the highest number of users possible using UAVs as access points.
The second scenario simulates an event happening in a city and, because of the ground network congestion, network capacity needs to be enhanced by the deployment of aerial base stations.
For both scenarios different types of RL algorithms are considered and their complexity and convergence are analysed.
In both cases it is shown that UAVs coupled with RL are capable of solving network issues in an efficient and quick manner.
Thus, due to its ability to learn from interaction with an environment and from previous experience, without knowing the dynamics of the environment, or relying on previously collected data, RL is considered as a promising solution to enable SON
Scheduler Designs in Wireless Networks
Wireless networks have undergone significant development in recent years, driven by the increasing demand for wireless connectivity and data services. Radio resource schedulers are developed to assign network users available resources, such as frequency and time, based on network conditions to handle the growing user demands, providing transmission opportunities for each user. Well-designed schedulers optimise wireless resource allocation to ensure that all users receive a fair and high quality of service (QoS) and that the network operates at its maximum performance. However, as new types of wireless network services emerge, the existing schedulers can no longer satisfy their QoS requirements and maximise the network performance objective. Thus, new schedulers are urgently needed in wireless networks. In this thesis, we study scheduler designs in cellular and Wi-Fi networks. We discuss the limitations of the existing scheduler design methods regarding flexibility, convergence rate and network-wise coordination and propose new methods to address these limitations. Specifically, we first develop a deep reinforcement learning algorithm to flexibly design wireless schedulers. We then study the acceleration of the scheduler design's convergence using statistical channel state information. We finally propose a graph representing learning method to enable the network-wise coordinated design of schedulers across multiple base stations. Practical implementations of proposed schedulers are also investigated in this thesis
Recommended from our members
Resource Allocation for the Internet of Everything: From Energy Harvesting Tags to Cellular Networks
In the near future, objects equipped with heterogeneous devices such as sensors, actuators, and tags, will be able to interact with each other and cooperate to achieve common goals. These networks are termed the Internet of Things (IoT) and have applications in healthcare, smart buildings, assisted living, manufacturing, supply chain management, and intelligent transportation. The IoT vision is enabled by ubiquitous wireless communications and there are numerous resource allocation challenges to efficiently connect each device to the network. In this thesis, we study wireless resource allocation problems that arise in the IoT, namely in the areas of the energy harvesting tags, termed the Internet of Tags (IoTags), and in cellular networks (mobile and cognitive).
First, we present our experience designing and developing Energy Harvesting Active Networked Tags (EnHANTs). The prototypes harvest indoor light energy using custom organic solar cells, communicate and form multihop networks using ultra-low-power Ultra- Wideband Impulse Radio (UWB-IR) transceivers, and dynamically adapt their communications and networking patterns to the energy harvesting and battery states. Using our custom designed small scale testbed, we evaluate energy-adaptive networking algorithms spanning the protocol stack (link, network, and flow control). Throughout the evaluation of experiments, we highlight numerous phenomena which are typically difficult to capture in simulations and nearly impossible to model in analytical work. We believe that these lessons would be useful for the designers of many different types of energy harvesters and energy harvesting adaptive networks.
Based on the lessons learned from EnHANTs, we present Power Aware Neighbor Discovery Asynchronously (Panda), a Neighbor Discovery (ND) protocol optimized for networks of energy harvesting nodes. To enable object tracking and monitoring applications for IoTags, Panda is designed to efficiently identify nodes which are within wireless communication range of one another. By accounting for numerous hardware constraints which are typically ignored (i.e., energy costs for transmission/reception, and transceiver state switching times/costs), we formulate a power budget to guarantee perpetual ND. Finally, via testbed evaluation utilizing Commercial Off-The-Shelf (COTS) energy harvesting nodes, we demonstrate experimentally that Panda outperforms existing protocols by a factor of 2-3x.
We then consider Proportional Fair (PF) cellular scheduling algorithms for mobile users, These users experience slow-fading wireless channels while traversing roads, train tracks, bus routes, etc. We leverage the predicable mobility on these routes and present the Predictive Finite-horizon PF Scheduling ((PF)2S) Framework. We collect extensive channel measurement results from a 3G network and characterize mobility-induced channel state trends. We show that a user’s channel state is highly reproducible and leverage that to develop a data rate prediction mechanism. Our trace-based simulations of the (PF)2S Framework indicate that the framework can increase the throughput by 15%–55% compared to traditional PF schedulers, while improving fairness.
Finally, we study fragmentation within a probability model of combinatorial structures. Our model does not refer to any particular application. Yet, it is applicable to dynamic spectrum access networks which can be used as the wireless access technology for numerous IoT applications. In dynamic spectrum access networks, users share the wireless resource and compete to transmit and receive data, and accordingly have specific bandwidth and residence-time requirements. We prove that the spectrum tends towards states of complete fragmentation. That is, for every request for j > 1 sub-channels, nearly all size-j requests are allocated j mutually disjoint sub-channels. In a suite of four theorems, we show how this result specializes for certain classes of request-size distributions. We also show that the delays in reaching the inefficient states of complete fragmentation can be surprisingly long. The results of this chapter provide insights into the fragmentation process and, in turn, into those circumstances where defragmentation is worth the cost it incurs