Resource Allocation in SDN/NFV-Enabled Core Networks

For next generation core networks, it is anticipated to integrate communication, storage and computing resources into one unified, programmable and flexible infrastructure. Software-defined networking (SDN) and network function virtualization (NFV) become two enablers. SDN decouples the network control and forwarding functions, which facilitates network management and enables network programmability. NFV allows the network functions to be virtualized and placed on high capacity servers located anywhere in the network, not only on dedicated devices in current networks. Driven by SDN and NFV platforms, the future network architecture is expected to feature centralized network management, virtualized function chaining, reduced capital and operational costs, and enhanced service quality. The combination of SDN and NFV provides a potential technical route to promote the future communication networks. It is imperative to efficiently manage, allocate and optimize the heterogeneous resources, including computing, storage, and communication resources, to the customized services to achieve better quality-of-service (QoS) provisioning. This thesis makes some in-depth researches on efficient resource allocation for SDN/NFV-enabled core networks in multiple aspects and dimensionality. Typically, the resource allocation task is implemented in three aspects. Given the traffic metrics, QoS requirements, and resource constraints of the substrate network, we first need to compose a virtual network function (VNF) chain to form a virtual network (VN) topology. Then, virtual resources allocated to each VNF or virtual link need to be optimized in order to minimize the provisioning cost while satisfying the QoS requirements. Next, we need to embed the virtual network (i.e., VNF chain) onto the substrate network, in which we need to assign the physical resources in an economical way to meet the resource demands of VNFs and links. This involves determining the locations of NFV nodes to host the VNFs and the routing from source to destination. Finally, we need to schedule the VNFs for multiple services to minimize the service completion time and maximize the network performance. In this thesis, we study resource allocation in SDN/NFV-enabled core networks from the aforementioned three aspects. First, we jointly study how to design the topology of a VN and embed the resultant VN onto a substrate network with the objective of minimizing the embedding cost while satisfying the QoS requirements. In VN topology design, optimizing the resource requirement for each virtual node and link is necessary. Without topology optimization, the resources assigned to the virtual network may be insufficient or redundant, leading to degraded service quality or increased embedding cost. The joint problem is formulated as a Mixed Integer Nonlinear Programming (MINLP), where queueing theory is utilized as the methodology to analyze the network delay and help to define the optimal set of physical resource requirements at network elements. Two algorithms are proposed to obtain the optimal/near-optimal solutions of the MINLP model. Second, we address the multi-SFC embedding problem by a game theoretical approach, considering the heterogeneity of NFV nodes, the effect of processing-resource sharing among various VNFs, and the capacity constraints of NFV nodes. In the proposed resource constrained multi-SFC embedding game (RC-MSEG), each SFC is treated as a player whose objective is to minimize the overall latency experienced by the supported service flow, while satisfying the capacity constraints of all its NFV nodes. Due to processing-resource sharing, additional delay is incurred and integrated into the overall latency for each SFC. The capacity constraints of NFV nodes are considered by adding a penalty term into the cost function of each player, and are guaranteed by a prioritized admission control mechanism. We first prove that the proposed game RC-MSEG is an exact potential game admitting at least one pure Nash Equilibrium (NE) and has the finite improvement property (FIP). Then, we design two iterative algorithms, namely, the best response (BR) algorithm with fast convergence and the spatial adaptive play (SAP) algorithm with great potential to obtain the best NE of the proposed game. Third, the VNF scheduling problem is investigated to minimize the makespan (i.e., overall completion time) of all services, while satisfying their different end-to-end (E2E) delay requirements. The problem is formulated as a mixed integer linear program (MILP) which is NP-hard with exponentially increasing computational complexity as the network size expands. To solve the MILP with high efficiency and accuracy, the original problem is reformulated as a Markov decision process (MDP) problem with variable action set. Then, a reinforcement learning (RL) algorithm is developed to learn the best scheduling policy by continuously interacting with the network environment. The proposed learning algorithm determines the variable action set at each decision-making state and accommodates different execution time of the actions. The reward function in the proposed algorithm is carefully designed to realize delay-aware VNF scheduling. To sum up, it is of great importance to integrate SDN and NFV in the same network to accelerate the evolution toward software-enabled network services. We have studied VN topology design, multi-VNF chain embedding, and delay-aware VNF scheduling to achieve efficient resource allocation in different dimensions. The proposed approaches pave the way for exploiting network slicing to improve resource utilization and facilitate QoS-guaranteed service provisioning in SDN/NFV-enabled networks

A reinforcement learning approach for Virtual Network Function Chaining and sharing in softwarized networks

​© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Cognizant of the ease with which softwarized functions can be dynamically scaled according to real time resource requirements, and the fact that multiple services can have common VNFs in their chaining, this paper tackles the problem of cost effective deployment of online services from the perspective of sharing their VNF instances. First, we formally formulate the deployment problem under VNFs sharing. Secondly, given the NP-hard nature of the above problem, we propose a reinforcement learning (RL) algorithm capable of making intelligent placement decisions while considering multiple conflicting costs. Costs of transmission, VNF instantiation or energy consumption, among others. Thanks to the intelligence of the RL algorithm, simulation results show that the performance of the proposed algorithm is within a 14% margin and similar to an optimal solution in terms of request provisioning cost and acceptance ratio, respectively. Moreover, the algorithm results in more than a 20% and a 70% improvement in terms of request deployment cost and time compared to a state-of-the-art algorithm, and up to more than a 40% improvement in terms of cost compared to an algorithm that greedily minimizes the transmission or VNF activation costs.Postprint (author's final draft

A deep reinforcement learning-based algorithm for reliability-aware multi-domain service deployment in smart ecosystems

The final publication is available at Springer via http://dx.doi.org/10.1007/s00521-020-05372-xThe transition towards full network virtualization will see services for smart ecosystems including smart metering, healthcare and transportation among others, being deployed as Service Function Chains (SFCs) comprised of an ordered set of virtual network functions. However, since such services are usually deployed in remote cloud networks, the SFCs may transcend multiple domains belonging to different Infrastructure Providers (InPs), possibly with differing policies regarding billing and Quality-of-service (QoS) guarantees. Therefore, efficiently allocating the exhaustible network resources to the different SFCs while meeting the stringent requirements of the services such as delay and QoS among others, remains a complex challenge, especially under limited information disclosure by the InPs. In this work, we formulate the SFC deployment problem across multiple domains focusing on delay constraints, and propose a framework for SFC orchestration which adheres to the privacy requirements of the InPs. Then, we propose a reinforcement learning (RL)-based algorithm for partitioning the SFC request across the different InPs while considering service reliability across the participating InPs. Such RL-based algorithms have the intelligence to infer undisclosed InP information from historical data obtained from past experiences. Simulation results, considering both online and offline scenarios, reveal that the proposed algorithm results in up to 10% improvement in terms of acceptance ratio and provisioning cost compared to the benchmark algorithms, with up to more than 90% saving in execution time for large networks. In addition, the paper proposes an enhancement to a state-of-the-art algorithm which results in up to 5% improvement in terms of provisioning cost.This work has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 777067 (NECOS project) and the national project TEC2015-71329-C2-2-R (MINECO/FEDER). This work is also supported by the " Fundamental Research Funds for the Central Universities " of China University of Petroleum (East China) under Grant 18CX02139APeer ReviewedPostprint (author's final draft

Latency and energy-aware provisioning of network slices in cloud networks

n/

Machine Learning Meets Communication Networks: Current Trends and Future Challenges

The growing network density and unprecedented increase in network traffic, caused by the massively expanding number of connected devices and online services, require intelligent network operations. Machine Learning (ML) has been applied in this regard in different types of networks and networking technologies to meet the requirements of future communicating devices and services. In this article, we provide a detailed account of current research on the application of ML in communication networks and shed light on future research challenges. Research on the application of ML in communication networks is described in: i) the three layers, i.e., physical, access, and network layers; and ii) novel computing and networking concepts such as Multi-access Edge Computing (MEC), Software Defined Networking (SDN), Network Functions Virtualization (NFV), and a brief overview of ML-based network security. Important future research challenges are identified and presented to help stir further research in key areas in this direction

Definition and specification of connectivity and QoE/QoS management mechanisms – final report

This document summarizes the WP5 work throughout the project, describing its functional architecture and the solutions that implement the WP5 concepts on network control and orchestration. For this purpose, we defined 3 innovative controllers that embody the network slicing and multi tenancy: SDM-C, SDM-X and SDM-O. The functionalities of each block are detailed with the interfaces connecting them and validated through exemplary network processes, highlighting thus 5G NORMA innovations. All the proposed modules are designed to implement the functionality needed to provide the challenging KPIs required by future 5G networks while keeping the largest possible compatibility with the state of the art

Neural combinatorial optimization as an enabler technology to design real-time virtual network function placement decision systems

158 p.The Fifth Generation of the mobile network (5G) represents a breakthrough technology for thetelecommunications industry. 5G provides a unified infrastructure capable of integrating over thesame physical network heterogeneous services with different requirements. This is achieved thanksto the recent advances in network virtualization, specifically in Network Function Virtualization(NFV) and Software Defining Networks (SDN) technologies. This cloud-based architecture not onlybrings new possibilities to vertical sectors but also entails new challenges that have to be solvedaccordingly. In this sense, it enables to automate operations within the infrastructure, allowing toperform network optimization at operational time (e.g., spectrum optimization, service optimization,traffic optimization). Nevertheless, designing optimization algorithms for this purpose entails somedifficulties. Solving the underlying Combinatorial Optimization (CO) problems that these problemspresent is usually intractable due to their NP-Hard nature. In addition, solutions to these problems arerequired in close to real-time due to the tight time requirements on this dynamic environment. Forthis reason, handwritten heuristic algorithms have been widely used in the literature for achievingfast approximate solutions on this context.However, particularizing heuristics to address CO problems can be a daunting task that requiresexpertise. The ability to automate this resolution processes would be of utmost importance forachieving an intelligent network orchestration. In this sense, Artificial Intelligence (AI) is envisionedas the key technology for autonomously inferring intelligent solutions to these problems. Combining AI with network virtualization can truly transform this industry. Particularly, this Thesis aims at using Neural Combinatorial Optimization (NCO) for inferring endsolutions on CO problems. NCO has proven to be able to learn near optimal solutions on classicalcombinatorial problems (e.g., the Traveler Salesman Problem (TSP), Bin Packing Problem (BPP),Vehicle Routing Problem (VRP)). Specifically, NCO relies on Reinforcement Learning (RL) toestimate a Neural Network (NN) model that describes the relation between the space of instances ofthe problem and the solutions for each of them. In other words, this model for a new instance is ableto infer a solution generalizing from the problem space where it has been trained. To this end, duringthe learning process the model takes instances from the learning space, and uses the reward obtainedfrom evaluating the solution to improve its accuracy.The work here presented, contributes to the NCO theory in two main directions. First, this workargues that the performance obtained by sequence-to-sequence models used for NCO in the literatureis improved presenting combinatorial problems as Constrained Markov Decision Processes (CMDP).Such property can be exploited for building a Markovian model that constructs solutionsincrementally based on interactions with the problem. And second, this formulation enables toaddress general constrained combinatorial problems under this framework. In this context, the modelin addition to the reward signal, relies on penalty signals generated from constraint dissatisfactionthat direct the model toward a competitive policy even in highly constrained environments. Thisstrategy allows to extend the number of problems that can be addressed using this technology.The presented approach is validated in the scope of intelligent network management, specifically inthe Virtual Network Function (VNF) placement problem. This problem consists of efficientlymapping a set of network service requests on top of the physical network infrastructure. Particularly,we seek to obtain the optimal placement for a network service chain considering the state of thevirtual environment, so that a specific resource objective is accomplished, in this case theminimization of the overall power consumption. Conducted experiments prove the capability of theproposal for learning competitive solutions when compared to classical heuristic, metaheuristic, andConstraint Programming (CP) solvers

Network slice reconfiguration by exploiting deep reinforcement learning with large action space

It is widely acknowledged that network slicing can tackle the diverse usage scenarios and connectivity services that the 5G-and-beyond system needs to support. To guarantee performance isolation while maximizing network resource utilization under dynamic traffic load, network slice needs to be reconfigured adaptively. However, it is commonly believed that the fine-grained resource reconfiguration problem is intractable due to the extremely high computational complexity caused by numerous variables. In this paper, we investigate the reconfiguration within a core network slice with aim of minimizing long-term resource consumption by exploiting Deep Reinforcement Learning (DRL). This problem is also intractable by using conventional Deep Q Network (DQN), as it has a multi-dimensional discrete action space which is difficult to explore efficiently. To address the curse of dimensionality, we propose a discrete Branching Dueling Q-network (discrete BDQ) by incorporating the action branching architecture into DQN, for drastically decreasing the number of estimated actions. Based on the discrete BDQ network, we develop an intelligent network slice reconfiguration algorithm (INSRA). Extensive simulation experiments are conducted to evaluate the performance of INSRA and the numerical results reveal that INSRA can minimize the long-term resource consumption and achieve high resource efficiency compared with several benchmark algorithms