AI gym for Networks

5G Networks are delivering better services and connecting more devices, but at the same time are becoming more complex. Problems like resource management and control optimization are increasingly dynamic and difficult to model making it very hard to use traditional model-based optimization techniques. Artificial Intelligence (AI) explores techniques such as Deep Reinforcement Learning (DRL), which uses the interaction between the agent and the environment to learn what action to take to obtain the best possible result. Researchers usually need to create and develop a simulation environment for their scenario of interest to be able to experiment with DRL algorithms. This takes a large amount of time from the research process, while the lack of a common environment makes it difficult to compare algorithms. The proposed solution aims to fill this gap by creating a tool that facilitates the setting up of DRL training environments for network scenarios. The developed tool uses three open source software, the Containernet to simulate the connections between devices, the Ryu Controller as the Software Defined Network Controller, and OpenAI Gym which is responsible for setting up the communication between the environment and the DRL agent. With the project developed during the thesis, the users will be capable of creating more scenarios in a short period, opening space to set up different environments, solving various problems as well as providing a common environment where other Agents can be compared. The developed software is used to compare the performance of several DRL agents in two different network control problems: routing and network slice admission control. A novel DRL based solution is used in the case of network slice admission control that jointly optimizes the admission and the placement of traffic of a network slice in the physical resources.As redes 5G oferecem melhores serviços e conectam mais dispositivos, fazendo com que se tornem mais complexas e difíceis de gerir. Problemas como a gestão de recursos e a otimização de controlo são cada vez mais dinâmicos e difíceis de modelar, o que torna difícil usar soluções de optimização basea- das em modelos tradicionais. A Inteligência Artificial (IA) explora técnicas como Deep Reinforcement Learning que utiliza a interação entre o agente e o ambiente para aprender qual a ação a ter para obter o melhor resultado possível. Normalmente, os investigadores precisam de criar e desenvolver um ambiente de simulação para poder estudar os algoritmos DRL e a sua interação com o cenário de interesse. A criação de ambientes a partir do zero retira tempo indispensável para a pesquisa em si, e a falta de ambientes de treino comuns torna difícil a comparação dos algoritmos. A solução proposta foca-se em preencher esta lacuna criando uma ferramenta que facilite a configuração de ambientes de treino DRL para cenários de rede. A ferramenta desenvolvida utiliza três softwares open source, o Containernet para simular as conexões entre os dispositivos, o Ryu Controller como Software Defined Network Controller e o OpenAI Gym que é responsável por configurar a comunicação entre o ambiente e o agente DRL. Através do projeto desenvolvido, os utilizadores serão capazes de criar mais cenários em um curto período, abrindo espaço para configurar diferentes ambientes e resolver diferentes problemas, bem como fornecer um ambiente comum onde diferentes Agentes podem ser comparados. O software desenvolvido foi usado para comparar o desempenho de vários agentes DRL em dois problemas diferentes de controlo de rede, nomeadamente, roteamento e controlo de admissão de slices na rede. Uma solução baseada em DRL é usada no caso do controlo de admissão de slices na rede que otimiza conjuntamente a admissão e a colocação de tráfego de uma slice na rede nos recursos físicos da mesma

Interference Aware Cognitive Femtocell Networks

Femtocells Access Points (FAP) are low power, plug and play home base stations which are designed to extend the cellular radio range in indoor environments where macrocell coverage is generally poor. They offer significant increases in data rates over a short range, enabling high speed wireless and mobile broadband services, with the femtocell network overlaid onto the macrocell in a dual-tier arrangement. In contrast to conventional cellular systems which are well planned, FAP are arbitrarily installed by the end users and this can create harmful interference to both collocated femtocell and macrocell users. The interference becomes particularly serious in high FAP density scenarios and compromises the ensuing data rate. Consequently, effective management of both cross and co-tier interference is a major design challenge in dual-tier networks. Since traditional radio resource management techniques and architectures for single-tier systems are either not applicable or operate inefficiently, innovative dual-tier approaches to intelligently manage interference are required. This thesis presents a number of original contributions to fulfill this objective including, a new hybrid cross-tier spectrum sharing model which builds upon an existing fractional frequency reuse technique to ensure minimal impact on the macro-tier resource allocation. A new flexible and adaptive virtual clustering framework is then formulated to alleviate co-tier interference in high FAP densities situations and finally, an intelligent coverage extension algorithm is developed to mitigate excessive femto-macrocell handovers, while upholding the required quality of service provision. This thesis contends that to exploit the undoubted potential of dual-tier, macro-femtocell architectures an interference awareness solution is necessary. Rigorous evidence confirms that noteworthy performance improvements can be achieved in the quality of the received signal and throughput by applying cognitive methods to manage interference

Congestion mitigation in LTE base stations using radio resource allocation techniques with TCP end to end transport

As of 2019, Long Term Evolution (LTE) is the chosen standard for most mobile and fixed wireless data communication. The next generation of standards known as 5G will encompass the Internet of Things (IoT) which will add more wireless devices to the network. Due to an exponential increase in the number of wireless subscriptions, in the next few years there is also an expected exponential increase in data traffic. Most of these devices will use Transmission Control Protocol (TCP) which is a type of network protocol for delivering internet data to users. Due to its reliability in delivering data payload to users and congestion management, TCP is the most common type of network protocol used. However, the ability for TCP to combat network congestion has certain limitations especially in a wireless network. This is due to wireless networks not being as reliable as fixed line networks for data delivery because of the use of last mile radio interface. LTE uses various error correction techniques for reliable data delivery over the air-interface. These cause other issues such as excessive latency and queuing in the base station leading to degradation in throughput for users and congestion in the network. Traditional methods of dealing with congestion such as tail-drop can be inefficient and cumbersome. Therefore, adequate congestion mitigation mechanisms are required. The LTE standard uses a technique to pre-empt network congestion by a mechanism known as Discard Timer. Additionally, there are other algorithms such as Random Early Detection (RED) that also are used for network congestion mitigation. However, these mechanisms rely on configured parameters and only work well within certain regions of operation. If the parameters are not set correctly then the TCP links can experience congestion collapse. In this thesis, the limitations of using existing LTE congestion mitigation mechanisms such as Discard Timer and RED have been explored. A different mechanism to analyse the effects of using control theory for congestion mitigation has been developed. Finally, congestion mitigation in LTE networks has been addresses using radio resource allocation techniques with non-cooperative game theory being an underlying mathematical framework. In doing so, two key end-to-end performance measurements considered for measuring congestion for the game theoretic models were identified which were the total end-to-end delay and the overall throughput of each individual TCP link. An end to end wireless simulator model with the radio access network using LTE and a TCP based backbone to the end server was developed using MATLAB. This simulator was used as a baseline for testing each of the congestion mitigation mechanisms. This thesis also provides a comparison and performance evaluation between the congestion mitigation models developed using existing techniques (such as Discard Timer and RED), control theory and game theory. As of 2019, Long Term Evolution (LTE) is the chosen standard for most mobile and fixed wireless data communication. The next generation of standards known as 5G will encompass the Internet of Things (IoT) which will add more wireless devices to the network. Due to an exponential increase in the number of wireless subscriptions, in the next few years there is also an expected exponential increase in data traffic. Most of these devices will use Transmission Control Protocol (TCP) which is a type of network protocol for delivering internet data to users. Due to its reliability in delivering data payload to users and congestion management, TCP is the most common type of network protocol used. However, the ability for TCP to combat network congestion has certain limitations especially in a wireless network. This is due to wireless networks not being as reliable as fixed line networks for data delivery because of the use of last mile radio interface. LTE uses various error correction techniques for reliable data delivery over the air-interface. These cause other issues such as excessive latency and queuing in the base station leading to degradation in throughput for users and congestion in the network. Traditional methods of dealing with congestion such as tail-drop can be inefficient and cumbersome. Therefore, adequate congestion mitigation mechanisms are required. The LTE standard uses a technique to pre-empt network congestion by a mechanism known as Discard Timer. Additionally, there are other algorithms such as Random Early Detection (RED) that also are used for network congestion mitigation. However, these mechanisms rely on configured parameters and only work well within certain regions of operation. If the parameters are not set correctly then the TCP links can experience congestion collapse. In this thesis, the limitations of using existing LTE congestion mitigation mechanisms such as Discard Timer and RED have been explored. A different mechanism to analyse the effects of using control theory for congestion mitigation has been developed. Finally, congestion mitigation in LTE networks has been addresses using radio resource allocation techniques with non-cooperative game theory being an underlying mathematical framework. In doing so, two key end-to-end performance measurements considered for measuring congestion for the game theoretic models were identified which were the total end-to-end delay and the overall throughput of each individual TCP link. An end to end wireless simulator model with the radio access network using LTE and a TCP based backbone to the end server was developed using MATLAB. This simulator was used as a baseline for testing each of the congestion mitigation mechanisms. This thesis also provides a comparison and performance evaluation between the congestion mitigation models developed using existing techniques (such as Discard Timer and RED), control theory and game theory

User Association in 5G Networks: A Survey and an Outlook

26 pages; accepted to appear in IEEE Communications Surveys and Tutorial

A Survey on Various Congestion Control Techniques in Wireless Sensor Networks

Wireless Sensor Networks (WSNs) are made up of small battery-powered sensors that can sense and monitor a variety of environmental conditions. These devices are self-contained and fault tolerant. The majority of WSNs are built to perform data collection tasks. These data are gathered and then sent to the sink node. Small packets are sent towards the sink node in such cases, and as a result, the areas near the sink node become congested, becoming the bottleneck of the entire network. In this paper, a survey of existing techniques or methods for detecting and eliminating congestions is conducted. Finally, a comparison in the form of a table based on various matrices is presented

Autonomic Overload Management For Large-Scale Virtualized Network Functions

The explosion of data traffic in telecommunication networks has been impressive in the last few years. To keep up with the high demand and staying profitable, Telcos are embracing the Network Function Virtualization (NFV) paradigm by shifting from hardware network appliances to software virtual network functions, which are expected to support extremely large scale architectures, providing both high performance and high reliability. The main objective of this dissertation is to provide frameworks and techniques to enable proper overload detection and mitigation for the emerging virtualized software-based network services. The thesis contribution is threefold. First, it proposes a novel approach to quickly detect performance anomalies in complex and large-scale VNF services. Second, it presents NFV-Throttle, an autonomic overload control framework to protect NFV services from overload within a short period of time, allowing to preserve the QoS of traffic flows admitted by network services in response to both traffic spikes (up to 10x the available capacity) and capacity reduction due to infrastructure problems (such as CPU contention). Third, it proposes DRACO, to manage overload problems arising in novel large-scale multi-tier applications, such as complex stateful network functions in which the state is spread across modern key-value stores to achieve both scalability and performance. DRACO performs a fine-grained admission control, by tuning the amount and type of traffic according to datastore node dependencies among the tiers (which are dynamically discovered at run-time), and to the current capacity of individual nodes, in order to mitigate overloads and preventing hot-spots. This thesis presents the implementation details and an extensive experimental evaluation for all the above overload management solutions, by means of a virtualized IP Multimedia Subsystem (IMS), which provides modern multimedia services for Telco operators, such as Videoconferencing and VoLTE, and which is one of the top use-cases of the NFV technology

A survey of self organisation in future cellular networks

This article surveys the literature over the period of the last decade on the emerging field of self organisation as applied to wireless cellular communication networks. Self organisation has been extensively studied and applied in adhoc networks, wireless sensor networks and autonomic computer networks; however in the context of wireless cellular networks, this is the first attempt to put in perspective the various efforts in form of a tutorial/survey. We provide a comprehensive survey of the existing literature, projects and standards in self organising cellular networks. Additionally, we also aim to present a clear understanding of this active research area, identifying a clear taxonomy and guidelines for design of self organising mechanisms. We compare strength and weakness of existing solutions and highlight the key research areas for further development. This paper serves as a guide and a starting point for anyone willing to delve into research on self organisation in wireless cellular communication networks