Load Balancing Models based on Reinforcement Learning for Self-Optimized Macro-Femto LTE-Advanced Heterogeneous Network

Heterogeneous Long Term Evolution-Advanced (LTE-A) network (HetNet) utilizes small cells to enhance its capacity and coverage. The intensive deployment of small cells such as pico- and femto-cells to complement macro-cells resulted in unbalanced distribution of traffic-load among cells. Machine learning techniques are employed in cooperation with Self-Organizing Network (SON) features to achieve load balancing between highly loaded Macro cells and underlay small cells such as Femto cells. In this paper, two algorithms have been proposed to balance the traffic load between Macro and Femto cells. The two proposed algorithms are named as Load Balancing based on Reinforcement Learning of end-user SINR (LBRL-SINR) and Load Balancing based on Reinforcement Learning of Macro cell-throughput (LBRL-T). Both of the proposed algorithms utilize Reinforcement Learning (RL) technique to control the reference signal power of each Femto cell that underlays a highly loaded Macro cell. At the same time, the algorithm monitors any degradation in the performance metrics of both Macro and its neighbor Femto cells and reacts to troubleshoot the degradation in real time. The simulation results showed that both of the proposed algorithms are able to off-load end-users from highly loaded Macro cell and redistribute the traffic load fairly with its neighbor Femto cells. As a result, both of call drop rate and call block rate of a highly loaded Macro cell are decreased

Implementation of cognitive radio networks to evaluate spectrum management strategies in real-time

This paper illustrates a Universal Software Radio Peripheral (USRP)-based real-time testbed that is able to evaluate different spectrum management solutions that exploit the Cognitive Radio (CR) paradigm. The main objective of this testbed is to provide an accurate and realistic platform by which the performance of innovative spectrum management solutions for a wide set of scenarios and use cases in the context of Opportunistic Networks (ONs) and Cognitive Radio Networks (CRNs) can be entirely validated and assessed before their implementation in real systems. Real-time platforms are essential to carry out significant studies and to accurately assess the performance of innovative solutions before their implementation in the real world. This work provides a comprehensive description of the testbed, highlighting many interesting implementation details and illustrating its applicability for different studies that rely on the CR paradigm. Then, a particular application in a realistic Digital Home (DH) scenario is also illustrated, which allows demonstrating the effectiveness of the real-time testbed and assessing its practicality in terms of user-perceived end-to-end Quality of Experience (QoE) in a realistic environment.Peer ReviewedPostprint (author's final draft

Packet Scheduling Algorithms in LTE/LTE-A cellular Networks: Multi-agent Q-learning Approach

Spectrum utilization is vital for mobile operators. It ensures an efficient use of spectrum bands, especially when obtaining their license is highly expensive. Long Term Evolution (LTE), and LTE-Advanced (LTE-A) spectrum bands license were auctioned by the Federal Communication Commission (FCC) to mobile operators with hundreds of millions of dollars. In the first part of this dissertation, we study, analyze, and compare the QoS performance of QoS-aware/Channel-aware packet scheduling algorithms while using CA over LTE, and LTE-A heterogeneous cellular networks. This included a detailed study of the LTE/LTE-A cellular network and its features, and the modification of an open source LTE simulator in order to perform these QoS performance tests. In the second part of this dissertation, we aim to solve spectrum underutilization by proposing, implementing, and testing two novel multi-agent Q-learning-based packet scheduling algorithms for LTE cellular network. The Collaborative Competitive scheduling algorithm, and the Competitive Competitive scheduling algorithm. These algorithms schedule licensed users over the available radio resources and un-licensed users over spectrum holes. In conclusion, our results show that the spectrum band could be utilized by deploying efficient packet scheduling algorithms for licensed users, and can be further utilized by allowing unlicensed users to be scheduled on spectrum holes whenever they occur

Sustainable scheduling policies for radio access networks based on LTE technology

A thesis submitted to the University of Bedfordshire in partial fulfilment of the requirements for the degree of Doctor of PhilosophyIn the LTE access networks, the Radio Resource Management (RRM) is one of the most important modules which is responsible for handling the overall management of radio resources. The packet scheduler is a particular sub-module which assigns the existing radio resources to each user in order to deliver the requested services in the most efficient manner. Data packets are scheduled dynamically at every Transmission Time Interval (TTI), a time window used to take the user’s requests and to respond them accordingly. The scheduling procedure is conducted by using scheduling rules which select different users to be scheduled at each TTI based on some priority metrics. Various scheduling rules exist and they behave differently by balancing the scheduler performance in the direction imposed by one of the following objectives: increasing the system throughput, maintaining the user fairness, respecting the Guaranteed Bit Rate (GBR), Head of Line (HoL) packet delay, packet loss rate and queue stability requirements. Most of the static scheduling rules follow the sequential multi-objective optimization in the sense that when the first targeted objective is satisfied, then other objectives can be prioritized. When the targeted scheduling objective(s) can be satisfied at each TTI, the LTE scheduler is considered to be optimal or feasible. So, the scheduling performance depends on the exploited rule being focused on particular objectives. This study aims to increase the percentage of feasible TTIs for a given downlink transmission by applying a mixture of scheduling rules instead of using one discipline adopted across the entire scheduling session. Two types of optimization problems are proposed in this sense: Dynamic Scheduling Rule based Sequential Multi-Objective Optimization (DSR-SMOO) when the applied scheduling rules address the same objective and Dynamic Scheduling Rule based Concurrent Multi-Objective Optimization (DSR-CMOO) if the pool of rules addresses different scheduling objectives. The best way of solving such complex optimization problems is to adapt and to refine scheduling policies which are able to call different rules at each TTI based on the best matching scheduler conditions (states). The idea is to develop a set of non-linear functions which maps the scheduler state at each TTI in optimal distribution probabilities of selecting the best scheduling rule. Due to the multi-dimensional and continuous characteristics of the scheduler state space, the scheduling functions should be approximated. Moreover, the function approximations are learned through the interaction with the RRM environment. The Reinforcement Learning (RL) algorithms are used in this sense in order to evaluate and to refine the scheduling policies for the considered DSR-SMOO/CMOO optimization problems. The neural networks are used to train the non-linear mapping functions based on the interaction among the intelligent controller, the LTE packet scheduler and the RRM environment. In order to enhance the convergence in the feasible state and to reduce the scheduler state space dimension, meta-heuristic approaches are used for the channel statement aggregation. Simulation results show that the proposed aggregation scheme is able to outperform other heuristic methods. When the aggregation scheme of the channel statements is exploited, the proposed DSR-SMOO/CMOO problems focusing on different objectives which are solved by using various RL approaches are able to: increase the mean percentage of feasible TTIs, minimize the number of TTIs when the RL approaches punish the actions taken TTI-by-TTI, and minimize the variation of the performance indicators when different simulations are launched in parallel. This way, the obtained scheduling policies being focused on the multi-objective criteria are sustainable. Keywords: LTE, packet scheduling, scheduling rules, multi-objective optimization, reinforcement learning, channel, aggregation, scheduling policies, sustainable