A comparison of reinforcement learning algorithms in fairness-oriented OFDMA schedulers

Due to large-scale control problems in 5G access networks, the complexity of radioresource management is expected to increase significantly. Reinforcement learning is seen as apromising solution that can enable intelligent decision-making and reduce the complexity of differentoptimization problems for radio resource management. The packet scheduler is an importantentity of radio resource management that allocates users’ data packets in the frequency domainaccording to the implemented scheduling rule. In this context, by making use of reinforcementlearning, we could actually determine, in each state, the most suitable scheduling rule to be employedthat could improve the quality of service provisioning. In this paper, we propose a reinforcementlearning-based framework to solve scheduling problems with the main focus on meeting the userfairness requirements. This framework makes use of feed forward neural networks to map momentarystates to proper parameterization decisions for the proportional fair scheduler. The simulation resultsshow that our reinforcement learning framework outperforms the conventional adaptive schedulersoriented on fairness objective. Discussions are also raised to determine the best reinforcement learningalgorithm to be implemented in the proposed framework based on various scheduler settings

Capacity Enhancement of Multiuser Wireless Communication System through Adaptive Non-Linear Pre coding

Multiuser multiple-input multiple-output (MIMO) nonlinear pre coding techniques face the issue of poor computational scalability of the size of the network. But by this nonlinear pre coding technique the interference is pre-cancelled automatically and also provides better capacity. So in order to reduce the computational burden in this paper, a definitive issue of MU-MIMO scalability is tackled through a non-linear adaptive optimum vector perturbation technique. Unlike the conventional (Vector Perturbation) VP methods, here a novel anterograde tracing is utilized which is usually recognized in the nervous system thus reducing complexity. The tracing of distance can be done through an iterative-optimization procedure. By this novel non-linear technique the capacity is improved to a greater extend which is explained practically. By means of this, the computational complexity is managed to be in the cubic order of the size of MUMIMO, and this mainly derives from the inverse of the channel matrix. The proposed signal processing system has been implemented in the working platform of MATLAB/SIMULINK. The simulation results of proposed communication system and comparison with existing systems shows the significance of the proposed work

Sustainable scheduling policies for radio access networks based on LTE technology

A thesis submitted to the University of Bedfordshire in partial fulfilment of the requirements for the degree of Doctor of PhilosophyIn the LTE access networks, the Radio Resource Management (RRM) is one of the most important modules which is responsible for handling the overall management of radio resources. The packet scheduler is a particular sub-module which assigns the existing radio resources to each user in order to deliver the requested services in the most efficient manner. Data packets are scheduled dynamically at every Transmission Time Interval (TTI), a time window used to take the user’s requests and to respond them accordingly. The scheduling procedure is conducted by using scheduling rules which select different users to be scheduled at each TTI based on some priority metrics. Various scheduling rules exist and they behave differently by balancing the scheduler performance in the direction imposed by one of the following objectives: increasing the system throughput, maintaining the user fairness, respecting the Guaranteed Bit Rate (GBR), Head of Line (HoL) packet delay, packet loss rate and queue stability requirements. Most of the static scheduling rules follow the sequential multi-objective optimization in the sense that when the first targeted objective is satisfied, then other objectives can be prioritized. When the targeted scheduling objective(s) can be satisfied at each TTI, the LTE scheduler is considered to be optimal or feasible. So, the scheduling performance depends on the exploited rule being focused on particular objectives. This study aims to increase the percentage of feasible TTIs for a given downlink transmission by applying a mixture of scheduling rules instead of using one discipline adopted across the entire scheduling session. Two types of optimization problems are proposed in this sense: Dynamic Scheduling Rule based Sequential Multi-Objective Optimization (DSR-SMOO) when the applied scheduling rules address the same objective and Dynamic Scheduling Rule based Concurrent Multi-Objective Optimization (DSR-CMOO) if the pool of rules addresses different scheduling objectives. The best way of solving such complex optimization problems is to adapt and to refine scheduling policies which are able to call different rules at each TTI based on the best matching scheduler conditions (states). The idea is to develop a set of non-linear functions which maps the scheduler state at each TTI in optimal distribution probabilities of selecting the best scheduling rule. Due to the multi-dimensional and continuous characteristics of the scheduler state space, the scheduling functions should be approximated. Moreover, the function approximations are learned through the interaction with the RRM environment. The Reinforcement Learning (RL) algorithms are used in this sense in order to evaluate and to refine the scheduling policies for the considered DSR-SMOO/CMOO optimization problems. The neural networks are used to train the non-linear mapping functions based on the interaction among the intelligent controller, the LTE packet scheduler and the RRM environment. In order to enhance the convergence in the feasible state and to reduce the scheduler state space dimension, meta-heuristic approaches are used for the channel statement aggregation. Simulation results show that the proposed aggregation scheme is able to outperform other heuristic methods. When the aggregation scheme of the channel statements is exploited, the proposed DSR-SMOO/CMOO problems focusing on different objectives which are solved by using various RL approaches are able to: increase the mean percentage of feasible TTIs, minimize the number of TTIs when the RL approaches punish the actions taken TTI-by-TTI, and minimize the variation of the performance indicators when different simulations are launched in parallel. This way, the obtained scheduling policies being focused on the multi-objective criteria are sustainable. Keywords: LTE, packet scheduling, scheduling rules, multi-objective optimization, reinforcement learning, channel, aggregation, scheduling policies, sustainable

Towards Wireless Virtualization for 5G Cellular Systems

Although it has been defined as one of the most promising key enabling technologies for the forthcoming fifth generation cellular networks, wireless virtualization still has several challenges remaining to be addressed. Amongst those, resource allocation, which decides how to embed the different wireless virtual networks on the physical relying infrastructure, is the one receiving maximum attention. This project aims at finding the optimal resource allocation for each virtual network, in terms of channel resources, power levels and radio access technologies so that the data rate requested by each virtual network can be guaranteed and the global throughput efficiency can be maximized.Aunque haya sido definida como una de las tecnologías clave para el desarrollo de la nueva generación de sistemas móviles, la virtualización del acceso radio aún tiene muchos retos a investigar. Entre ellos, la distribución de los recursos, que tiene por objetivo encontrar el mejor encaje de las distintas redes virtuales en la infraestructura física que comparten, es el que está recibiendo la mayor atención. Este proyecto, tiene por objetivo encontrar la repartición óptima de los recursos, tanto a nivel de canal como de potencia y de tecnologías de acceso radio, para que los requisitos de las redes virtuales puedan ser garantizadas y la eficiencia global sea maximizada.Malgrat ha estat definida com una de les tecnologies claus de cara al desenvolupament de la propera cinquena generació de xarxes mòbils, la virtualització de l'accés radio encara té molts reptes oberts a fer front. Entre ells, la distribució de recursos, que té per objectiu buscar el millor encaix de les diferents xarxes virtuals en la infraestructura física que comparteixen, és la que està centrant la màxima atenció. Aquest projecte té per objectiu aconseguir la repartició òptima de recursos, pel que fa al canal, als nivells de potència i a les tecnologies radio disponibles, de manera que els requisits de cada xarxa virtual puguin ser garantits i que l'eficiència global pugui ser maximitzada

Machine Learning for Unmanned Aerial System (UAS) Networking

Fueled by the advancement of 5G new radio (5G NR), rapid development has occurred in many fields. Compared with the conventional approaches, beamforming and network slicing enable 5G NR to have ten times decrease in latency, connection density, and experienced throughput than 4G long term evolution (4G LTE). These advantages pave the way for the evolution of Cyber-physical Systems (CPS) on a large scale. The reduction of consumption, the advancement of control engineering, and the simplification of Unmanned Aircraft System (UAS) enable the UAS networking deployment on a large scale to become feasible. The UAS networking can finish multiple complex missions simultaneously. However, the limitations of the conventional approaches are still a big challenge to make a trade-off between the massive management and efficient networking on a large scale. With 5G NR and machine learning, in this dissertation, my contributions can be summarized as the following: I proposed a novel Optimized Ad-hoc On-demand Distance Vector (OAODV) routing protocol to improve the throughput of Intra UAS networking. The novel routing protocol can reduce the system overhead and be efficient. To improve the security, I proposed a blockchain scheme to mitigate the malicious basestations for cellular connected UAS networking and a proof-of-traffic (PoT) to improve the efficiency of blockchain for UAS networking on a large scale. Inspired by the biological cell paradigm, I proposed the cell wall routing protocols for heterogeneous UAS networking. With 5G NR, the inter connections between UAS networking can strengthen the throughput and elasticity of UAS networking. With machine learning, the routing schedulings for intra- and inter- UAS networking can enhance the throughput of UAS networking on a large scale. The inter UAS networking can achieve the max-min throughput globally edge coloring. I leveraged the upper and lower bound to accelerate the optimization of edge coloring. This dissertation paves a way regarding UAS networking in the integration of CPS and machine learning. The UAS networking can achieve outstanding performance in a decentralized architecture. Concurrently, this dissertation gives insights into UAS networking on a large scale. These are fundamental to integrating UAS and National Aerial System (NAS), critical to aviation in the operated and unmanned fields. The dissertation provides novel approaches for the promotion of UAS networking on a large scale. The proposed approaches extend the state-of-the-art of UAS networking in a decentralized architecture. All the alterations can contribute to the establishment of UAS networking with CPS

Spectrally and Energy Efficient Radio Resource Management for Multi-Operator Shared Networks

Commercial mobile communication systems are mainly based on licensed frequency spectrum, and the license is very expensive as the spectrum is a sparse wireless resource. Therefore, sharing this wireless resource is an essential requirement not only at the present but also in the future considering trends like connectivity for everybody and everything. In this thesis, we study the sharing of wireless resources with different approaches for realizing fair, efficient, and predictable sharing solutions in a controlled manner. The efficient use of wireless channel resources is an important target to reduce the costs of network operation and deployment. To achieve this, we need practical scheduling algorithms for wireless resources, out of which several of them will be presented and analyzed in this work. Different optimization frameworks for the spectral efficiency utility are presented, with an individual focus on guaranteeing resource or rate fairness among the operators in a network with shared radio resources. Thus, the presented proposals will help the mobile network operators to overcome the issues of losing network control and traceability of used wireless resources in a shared environment. Besides this, emerging vertical industries, such as automotive, healthcare, industry 4.0, internet of things (IoT) industries will put a certain burden on the wireless networks asking for guaranteed service level requirement from the mobile network operators. In this regard, this thesis provides the necessary methods addressing these challenges with the help of scheduling methods which are based on the joint optimization of spectral and energy efficiency. Thus, wireless networks will be enabled as a service function in a controlled and scalable way for new emerging markets. Furthermore, the presented solutions t well with the requirements of fifth generation (5G) network slicing

Principles of Physical Layer Security in Multiuser Wireless Networks: A Survey

This paper provides a comprehensive review of the domain of physical layer security in multiuser wireless networks. The essential premise of physical-layer security is to enable the exchange of confidential messages over a wireless medium in the presence of unauthorized eavesdroppers without relying on higher-layer encryption. This can be achieved primarily in two ways: without the need for a secret key by intelligently designing transmit coding strategies, or by exploiting the wireless communication medium to develop secret keys over public channels. The survey begins with an overview of the foundations dating back to the pioneering work of Shannon and Wyner on information-theoretic security. We then describe the evolution of secure transmission strategies from point-to-point channels to multiple-antenna systems, followed by generalizations to multiuser broadcast, multiple-access, interference, and relay networks. Secret-key generation and establishment protocols based on physical layer mechanisms are subsequently covered. Approaches for secrecy based on channel coding design are then examined, along with a description of inter-disciplinary approaches based on game theory and stochastic geometry. The associated problem of physical-layer message authentication is also introduced briefly. The survey concludes with observations on potential research directions in this area.Comment: 23 pages, 10 figures, 303 refs. arXiv admin note: text overlap with arXiv:1303.1609 by other authors. IEEE Communications Surveys and Tutorials, 201