Deep Reinforcement Learning for URLLC data management on top of scheduled eMBB traffic

With the advent of 5G and the research into beyond 5G (B5G) networks, a novel and very relevant research issue is how to manage the coexistence of different types of traffic, each with very stringent but completely different requirements. In this paper we propose a deep reinforcement learning (DRL) algorithm to slice the available physical layer resources between ultra-reliable low-latency communications (URLLC) and enhanced Mobile BroadBand (eMBB) traffic. Specifically, in our setting the time-frequency resource grid is fully occupied by eMBB traffic and we train the DRL agent to employ proximal policy optimization (PPO), a state-of-the-art DRL algorithm, to dynamically allocate the incoming URLLC traffic by puncturing eMBB codewords. Assuming that each eMBB codeword can tolerate a certain limited amount of puncturing beyond which is in outage, we show that the policy devised by the DRL agent never violates the latency requirement of URLLC traffic and, at the same time, manages to keep the number of eMBB codewords in outage at minimum levels, when compared to other state-of-the-art schemes.Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Multiplexing eMBB and URLLC in wireless powered communication networks: a deep reinforcement learning-based approach

This paper investigates the multiplexing of enhanced mobile broadband (eMBB) and ultra-reliable low-latency communications (URLLC) services in a wireless powered communication network, where a hybrid access point coordinates the wireless energy transfer (WET) to users and receives information from them. The preemptive puncturing is adopted to multiplex URLLC traffic onto eMBB transmission. Apart from the energy used for wireless information transmission (WIT), the rest energy in user’s battery is reserved to avoid insufficient energy for future WIT. The problem is formulated to jointly allocate subcarriers, time, and energy to maximize the uplink eMBB sum rate under the constraints of URLLC latency, radio frequency to direct current (RF/DC) sensitivity, user’s battery capacity, and subcarriers availability. We propose a deep reinforcement learning-based approach named mixed deep deterministic policy gradient (Mixed-DDPG), which decomposes the optimization problem into a discrete subproblem for subcarriers allocation and a continuous subproblem for time and energy allocation, and solves them alternately. Numerical results show that the proposed algorithm achieves a higher eMBB sum rate than the existing schemes

A Survey of Scheduling in 5G URLLC and Outlook for Emerging 6G Systems

Future wireless communication is expected to be a paradigm shift from three basic service requirements of 5th Generation (5G) including enhanced Mobile Broadband (eMBB), Ultra Reliable and Low Latency communication (URLLC) and the massive Machine Type Communication (mMTC). Integration of the three heterogeneous services into a single system is a challenging task. The integration includes several design issues including scheduling network resources with various services. Specially, scheduling the URLLC packets with eMBB and mMTC packets need more attention as it is a promising service of 5G and beyond systems. It needs to meet stringent Quality of Service (QoS) requirements and is used in time-critical applications. Thus through understanding of packet scheduling issues in existing system and potential future challenges is necessary. This paper surveys the potential works that addresses the packet scheduling algorithms for 5G and beyond systems in recent years. It provides state of the art review covering three main perspectives such as decentralised, centralised and joint scheduling techniques. The conventional decentralised algorithms are discussed first followed by the centralised algorithms with specific focus on single and multi-connected network perspective. Joint scheduling algorithms are also discussed in details. In order to provide an in-depth understanding of the key scheduling approaches, the performances of some prominent scheduling algorithms are evaluated and analysed. This paper also provides an insight into the potential challenges and future research directions from the scheduling perspective

A Multi-Agent Reinforcement Learning Architecture for Network Slicing Orchestration

The Network Slicing (NS) paradigm is one of the pillars of the future 5G networks and is gathering great attention from both industry and scientific communities. In a NS scenario, physical and virtual resources are partitioned among multiple logical networks, named slices, with specific characteristics. The challenge consists in finding efficient strategies to dynamically allocate the network resources among the different slices according to the user requirements. In this paper, we tackle the target problem by exploiting a Deep Reinforcement Learning approach. Our framework is based on a distributed architecture, where multiple agents cooperate towards a common goal. The agent training is carried out following the Advantage Actor Critic algorithm, which makes it possible to handle continuous action spaces. By means of extensive simulations, we show that our strategy yields better performance than an efficient empirical algorithm, while ensuring high adaptability to different scenarios without the need for additional training.acceptedVersio

5g new radio access and core network slicing for next-generation network services and management

In recent years, fifth-generation New Radio (5G NR) has attracted much attention owing to its potential in enhancing mobile access networks and enabling better support for heterogeneous services and applications. Network slicing has garnered substantial focus as it promises to offer a higher degree of isolation between subscribers with diverse quality-of-service requirements. Integrating 5G NR technologies, specifically the mmWave waveform and numerology schemes, with network slicing can unlock unparalleled performance so crucial to meeting the demands of high throughput and sub-millisecond latency constraints. While conceding that optimizing next-generation access network performance is extremely important, it needs to be acknowledged that doing so for the core network is equally as significant. This is majorly due to the numerous core network functions that execute control tasks to establish end-to-end user sessions and route access network traffic. Consequently, the core network has a significant impact on the quality-of-experience of the radio access network customers. Currently, the core network lacks true end-to-end slicing isolation and reliability, and thus there is a dire need to examine more stringent configurations that offer the required levels of slicing isolation for the envisioned networking landscape. Considering the factors mentioned above, a sequential approach is adopted starting with the radio access network and progressing to the core network. First, to maximize the downlink average spectral efficiency of an enhanced mobile broadband slice in a time division duplex radio access network while meeting the quality-of-service requirements, an optimization problem is formulated to determine the duplex ratio, numerology scheme, power, and bandwidth allocation. Subsequently, to minimize the uplink transmission power of an ultra-reliable low latency communications slice while satisfying the quality-of-service constraints, a second optimization problem is formulated to determine the above-mentioned parameters and allocations. Because 5G NR supports dual-band transmissions, it also facilitates the usage of different numerology schemes and duplex ratios across bands simultaneously. Both problems, being mixed-integer non-linear programming problems, are relaxed into their respective convex equivalents and subsequently solved. Next, shifting attention to aerial networks, a priority-based 5G NR unmanned aerial vehicle network (UAV) is considered where the enhanced mobile broadband and ultra-reliable low latency communications services are considered as best-effort and high-priority slices, correspondingly. Following the application of a band access policy, an optimization problem is formulated. The goal is to minimize the downlink quality-of-service gap for the best-effort service, while still meeting the quality-of-service constraints of the high-priority service. This involves the allocation of transmission power and assignment of resource blocks. Given that this problem is a mixed-integer nonlinear programming problem, a low-complexity algorithm, PREDICT, i.e., PRiority BasED Resource AllocatIon in Adaptive SliCed NeTwork, which considers the channel quality on each individual resource block over both bands, is designed to solve the problem with a more accurate accounting for high-frequency channel conditions. Transitioning to minimizing the operational latency of the core network, an integer linear programming problem is formulated to instantiate network function instances, assign them to core network servers, assign slices and users to network function instances, and allocate computational resources while maintaining virtual network function isolation and physical separation of the core network control and user planes. The actor-critic method is employed to solve this problem for three proposed core network operation configurations, each offering an added degree of reliability and isolation over the default configuration that is currently standardized by the 3GPP. Looking ahead to potential future research directions, optimizing carrier aggregation-based resource allocation across triple-band sliced access networks emerges as a promising avenue. Additionally, the integration of coordinated multi-point techniques with carrier aggregation in multi-UAV NR aerial networks is especially challenging. The introduction of added carrier frequencies and channel bandwidths, while enhancing flexibility and robustness, complicates band-slice assignments and user-UAV associations. Another layer of intriguing yet complex research involves optimizing handovers in high-mobility UAV networks, where both users and UAVs are mobile. UAV trajectory planning, which is already NP-hard even in static-user scenarios, becomes even more intricate to obtain optimal solutions in high-mobility user cases

Scalable Multiuser Immersive Communications with Multi-numerology and Mini-slot

This paper studies multiuser immersive communications networks in which different user equipment may demand various extended reality (XR) services. In such heterogeneous networks, time-frequency resource allocation needs to be more adaptive since XR services are usually multi-modal and latency-sensitive. To this end, we develop a scalable time-frequency resource allocation method based on multi-numerology and mini-slot. To appropriately determining the discrete parameters of multi-numerology and mini-slot for multiuser immersive communications, the proposed method first presents a novel flexible time-frequency resource block configuration, then it leverages the deep reinforcement learning to maximize the total quality-of-experience (QoE) under different users' QoE constraints. The results confirm the efficiency and scalability of the proposed time-frequency resource allocation method