Video on Demand Streaming Using RL-based Edge Caching in 5G Networks

Edge caching can significantly improve the 5G networks' performance both in terms of delay and backhaul traffic. We use a reinforcement learning-based (RL-based) caching technique that can adapt to time-location-dependent popularity patterns for on-demand video contents. In a private 5G, we implement the proposed caching scheme as two virtual network functions (VNFs), edge and remote servers, and measure the cache hit ratio as a KPI. Combined with the HLS protocol, the proposed video-on-demand (VoD) streaming is a reliable and scalable service that can adapt to content popularity.Comment: 3 pages, 1 figure One page version of this paper has been accepted to 2022 IEEE Conference on Standards for Communications and Networking (CSCN) - Demo submission

R-Learning-based admission control for service federation in multi-domain 5G networks

Proceedings of: IEEE Global Communications Conference (GLOBECOM), 7-11 Dec. 2021, Madrid, Spain.Network service federation in 5G/B5G networks enables service providers to extend service offering by collaborating with peering providers. Realizing this vision requires interoperability among providers towards end-to-end service orchestration across multiple administrative domains. Smart admission control is fundamental to make such extended offering profitable. Without prior knowledge of service requests, the admission controller (AC) either determines the domain to deploy each demand or rejects it to maximize the long-term average profit. In this paper, we first obtain the optimal AC policy by formulating the problem as a Markov decision process, which is solved through the policy iteration method. This provides the theoretical performance bound under the assumption of known arrival and departure rates of demands. Then, for practical solutions to be deployed in real systems, where the rates are not known, we apply the Q-Learning and R-Learning algorithms to approximate the optimal policy. The extensive simulation results show that learning approaches outperform the greedy policy and are capable of getting close to optimal performance. More specifically, R-learning always outperformed the rest of practical solutions and achieved an optimality gap of 3-5% independent of the system configuration, while Q-Learning showed lower performance and depended on discount factor tuning.This work has been partially funded by the MINECO grant TEC2017-88373-R (5G-REFINE), the EC H2020 5Growth Project (grant no. 856709), and Generalitat de Catalunya grant 2017 SGR 1195

Explanation-Guided Deep Reinforcement Learning for Trustworthy 6G RAN Slicing

The complexity of emerging sixth-generation (6G) wireless networks has sparked an upsurge in adopting artificial intelligence (AI) to underpin the challenges in network management and resource allocation under strict service level agreements (SLAs). It inaugurates the era of massive network slicing as a distributive technology where tenancy would be extended to the final consumer through pervading the digitalization of vertical immersive use-cases. Despite the promising performance of deep reinforcement learning (DRL) in network slicing, lack of transparency, interpretability, and opaque model concerns impedes users from trusting the DRL agent decisions or predictions. This problem becomes even more pronounced when there is a need to provision highly reliable and secure services. Leveraging eXplainable AI (XAI) in conjunction with an explanation-guided approach, we propose an eXplainable reinforcement learning (XRL) scheme to surmount the opaqueness of black-box DRL. The core concept behind the proposed method is the intrinsic interpretability of the reward hypothesis aiming to encourage DRL agents to learn the best actions for specific network slice states while coping with conflict-prone and complex relations of state-action pairs. To validate the proposed framework, we target a resource allocation optimization problem where multi-agent XRL strives to allocate optimal available radio resources to meet the SLA requirements of slices. Finally, we present numerical results to showcase the superiority of the adopted XRL approach over the DRL baseline. As far as we know, this is the first work that studies the feasibility of an explanation-guided DRL approach in the context of 6G networks.Comment: 6 Pages, 6 figure

Intelligent service orchestration in edge cloud networks

The surge in data traffic is challenging for network infrastructure owners coping with stringent service requirements (e.g., high bandwidth, ultralow latency) as well as shrinking per-gigabyte revenues. Network softwarization and edge computing are powerful candidates to mitigate these issues. In parallel, there is an increasing demand for network virtualization and container-based services. In this study, we investigate the management of software defined networking (SDN)-based transport network and edge cloud service orchestration. To this end, we use a machine learning (ML)-based design to manage both transport and edge cloud resources of a mobile network effectively. To generate and use real-world data inside our ML platform, we use the Graphical Network Simulator-3 (GNS3) emulator environment. Our emulation results indicate that almost all of the trained ML models can accurately select the correct edge clouds (ECs) (i.e., with high test accuracy) under the considered two scenarios when transport and EC network parameters are considered in comparison to models trained via only transport or cloud-based parameters. At the end of the article, we also provide an evolved architecture where the proposed ML platform can be embedded in an end-to-end mobile network architecture and H2020 5Growth project's baseline management platform.This work has been partially funded by the EU H2020 5Growth Project (grant no. 856709), by MINECO grant TEC2017-88373-R (5G-REFINE), and Generalitat de Catalunya grant 2017 SGR, 1195

Deploying a containerized ns-3/LENA-based LTE mobile Network Service through the 5G-TRANSFORMER platform

This paper has been presented at: 2018 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN)This demo presents an ongoing prototype implementation of the Service Orchestrator (SO) building block of the 5G-TRANSFORMER (5GT) architecture. Within the 5GT-SO, we define the Service Manager (SM), which hosts the intelligence of the 5GT-SO and interacts with the other architectural blocks of the 5GT architecture through the defined APIs. The aim of defining the SM is to decouple the 5GT-SO implementation from the associated MANO platform, allowing the interoperability with other MANO platforms, hence increasing the scope of the 5GT solution. In this demo, we will show how the current ongoing implementation of the 5GT-SO, using the SM, is able to automate the orchestration of both computing and networking resources to deploy a virtualized mobile network service based on ns-3/LENA network simulator/emulator in minutes over an emulated environment consisting of a multi-point of presence infrastructure connected by a custom transport network.This work was supported by the 5G-TRANSFORMER project (H2020-761536), by MINECO grant TEC2017-88373-R (5G-REFINE) and Generalitat de Catalunya grant 2017 SGR 1195

Intent-Based Orchestration for Application Relocation in a 5G Cloud-native Platform

The need of mobile network operators for cost-effectiveness is driving 5G and beyond networks towards highly flexible and agile deployments to adapt to dynamic and resource-constrained scenarios while meeting a myriad of user network stakeholders' requirements. In this setting, we consider that zero-touch orchestration schemes based on cloud-native deployments equipped with end-to-end monitoring capabilities provide the necessary technology mix to be a solution candidate. This demonstration, built on top of an end-to-end cloud-native 5G experimental platform with over-the-air transmissions, shows how dynamic orchestration can relocate container-based end-user applications to fulfil intent-based requirements. Accordingly, we provide an experimental validation to showcase how the platform enables the desired flexible and agile 5G deployments

Wireless Interface Agent for SDN mmwave multi-hop networks: design and experimental evaluation

2nd ACM Workshop on Millimeter Wave Networks and Sensing Systems (mmNets)Millimeter wave (mmwave) communications will likely be an enabler for 5G due to its multi-gigabit per second throughput capabilities. Furthermore,mmWave communications will have to be integrated in a new redesigned network required by 5G to fulfill its ambitious targets. In this paper, we present the design and implementation of a management agent for wireless devices deployed in a heterogeneous SDN wireless multi-hop research platform featuring mmwave communications for crosshauling (backhaul and fronthaul) purposes. The performance of the deployed mmwave network, based on the IEEE 802.11ad standard, is measured employing this agent. We measure the downtime in the presence of link up/down events, with obtained response times in the order of 10s-to-100s of milliseconds depending on the case. Furthermore, the TCP performance over the multi-hop 802.11ad mmwave network is also experimentally evaluated. In fact, TCP throughput up to around 800Mbps are obtained for single and multi-hop scenarios despite neighboring links using the same channel. Finally, one can also observe the impact of MTU size on TCP throughput, which may hinder the full exploitation of the mmWave link capacity when combined with other transport technologies, since the advantages of big MTUs (much bigger than the typical 1500 bytes) offered by mmwave devices may not be reaped.Thiswork was supported by MINECO grants TEC2017-88373-R (5G-REFINE), Generalitat de Catalunya grant 2017 SGR 1195, and by the 5G-TRANSFORMER project (H2020-761536)

Decentralized Energy Marketplace via NFTs and AI-based Agents

The paper introduces an advanced Decentralized Energy Marketplace (DEM) integrating blockchain technology and artificial intelligence to manage energy exchanges among smart homes with energy storage systems. The proposed framework uses Non-Fungible Tokens (NFTs) to represent unique energy profiles in a transparent and secure trading environment. Leveraging Federated Deep Reinforcement Learning (FDRL), the system promotes collaborative and adaptive energy management strategies, maintaining user privacy. A notable innovation is the use of smart contracts, ensuring high efficiency and integrity in energy transactions. Extensive evaluations demonstrate the system's scalability and the effectiveness of the FDRL method in optimizing energy distribution. This research significantly contributes to developing sophisticated decentralized smart grid infrastructures. Our approach broadens potential blockchain and AI applications in sustainable energy systems and addresses incentive alignment and transparency challenges in traditional energy trading mechanisms. The implementation of this paper is publicly accessible at \url{https://github.com/RasoulNik/DEM}.Comment: 6 page

Experimental validation of compute and network resource abstraction and allocation mechanisms within an NFV infrastructure

Proceedings of: IFIP/IEEE International Symposium on Integrated Network Management (IM), 17-21 May 2021, Bordeaux, France.5G supported capabilities (e.g., slicing) enable accommodating heterogeneous vertical services having their own requirements over a common cloud and transport infrastructure. In this context, the EU-H2020 5Growth project defines a service and infrastructure orchestration architecture to automatically deploy network services (NSes) fulfilling vertical demands. In this architecture, the Service Orchestrator (5Gr-SO), as a service provider, maps the vertical service needs into NS requirements (e.g., CPU, RAM, bandwidth, etc.). The 5Gr-SO interacts with an underlying infrastructure orchestrator referred to as 5Gr-RL. The 5Gr-RL, as an infrastructure provider, handles two main functions: i) abstraction of the resources exposed to the 5GrSO, and ii) fine-grained resource selection. Different interaction forms between both 5Gr-SO and 5Gr-RL arise differing in the exchanged abstracted information and resource allocation. We present two 5Gr-SO and 5Gr-RL interaction solutions stemming from two 5Gr-RL operational modes: Infrastructure Abstraction (InA) and Connectivity Service Abstraction (CSA). In the InA approach, the 5Gr-SO is granted with an aggregated view of the computing resources and a set of transport logical links between the cloud locations. In the CSA strategy, besides the aggregated view of the cloud resources, the logical links are associated to potential connectivity service types. Both InA and CSA strategies are presented describing their pros and cons. Moreover, the designed workflows (involving the devised abstraction and allocation algorithms) between the 5Gr-SO and 5Gr-RL entities are experimentally validated. Scalability studies are conducted upon two different cloud and transport infrastructure sizes in terms of the abstraction composition time, the expansion computation time, and total NS deployment time.Work supported in part by EU Commission H2020 5Growth project (Grant No. 856709), Spanish MICINN AURORAS (RTI2018-099178-B-I00) and Spanish MINECO 5G-REFINE (TEC2017-88373-R) projects and Generalitat de Catalunya grant 2017 SGR 1195