The Optical RL-Gym: an open-source toolkit for applying reinforcement learning in optical networks

Reinforcement Learning (RL) is leading to important breakthroughs in several areas (e.g., self-driving vehicles, robotics, and network automation). Part of its success is due to the existence of toolkits (e.g., OpenAI Gym) to implement standard RL tasks. On the one hand, they allow for the quick implementation and testing of new ideas. On the other, these toolkits ensure easy reproducibility via quick and fair benchmarking. RL is also gaining traction in the optical networks research community, showing promising results while solving several use cases. However, there are many scenarios where the benefits of RL-based solutions remain still unclear. A possible reason for this is the steep learning curve required to tailor RL-based frameworks to each specific use case. This, in turn, might delay or even prevent the development of new ideas. This paper introduces the Optical Network Reinforcement-Learning-Gym (Optical RL-Gym), an open-source toolkit that can be used to apply RL to problems related to optical networks. The Optical RL-Gym follows the principles established by the OpenAI Gym, the de-facto standard for RL environments. Optical RL-Gym allows for the quick integration with existing RL agents, as well as the possibility to build upon several already available environments to implement and solve more elaborated use cases related to the optical networks research area. The capabilities and the benefits of the proposed toolkit are illustrated by using the Optical RL-Gym to solve two different service provisioning problems

Network automation: challenges, enablers, and benefits

Communication infrastructures are evolving towards an ad-hoc service provisioning scenario where programmability and flexibility are fundamental concepts. Network automation is expected to play a vital role in streamlining all aspects of the service provisioning process (i.e., deployment, maintenance, and tear down). However, to fully realize this autonomous operation vision, closed-loop automation procedures need to be developed.This tutorial will present the main motivations and challenges behind designing and operating closed-loop autonomous decision-making processes, including a brief overview of current standardization initiatives. The tutorial will then address several use cases showcasing how network automation can alleviate the complexity of the service provisioning processes and the benefits brought in by the introduction of network automation

Storage Protection with Connectivity and Processing Restoration for Survivable Cloud Services

The operation and management of software-based communication systems and services is a big challenge for infrastructure and service providers.The challenge is mainly associated with the larger number of configurable elements and the higher dynamicity in the software-based systems compared to the classical ones. On the other hand, the modularity and programmability in software-based networks enabled by technologies like Software Defined Networking (SDN) and Network Function Virtualization (NFV) provide new opportunities for operators to realize advanced network and service management strategies beyond the classical techniques.In our work, we elaborate on these new opportunities and propose a novel strategy for the management of survivable cloud services.In particular, we leverage the flexibility of SDN and NFV to combine proactive protection and reactive restoration mechanisms and we put forward a novel strategy for enhancing the survivability of cloud services. Through comprehensive evaluations, we demonstrate that the proposed strategy offers significant benefits in terms of availability and restorability of services while reducing, at the same time, the overhead caused by the relocation of cloud services in case of failures

Machine-Learning-as-a-Service for Optical Networks: Use Cases and Benefits

Machine Learning (ML) models have been a valuable tool to assist on the design and operation of optical networks. Several use cases have benefited from ML models, such as Quality-of-Transmission (QoT) estimation, device modeling, constellation shaping, and attack/anomaly prediction/detection. ML models are expected to be ubiquitous in optical network management and operations thereof. However, the amount of human intervention and empirical decisions needed to select the exact ML model, train and evaluate its performance, and ultimately deploy and use the model, may become a bottleneck for widespread ML use in optical networks. Machine-Learning-as-a-Service (MLaaS) has the potential to greatly reduce human intervention and empirical decisions during the creation, evaluation, and deployment of ML models. In this talk, we will firstly discuss optical network use cases that can benefit from MLaaS. Then, we detail our proposed architecture for MLaaS. Finally, performance results for two use cases will be presented

P4-based Telemetry Processing for Fast Soft Failure Recovery in Packet-Optical Networks

A novel framework for in-network P4 processing of distributed multi-layer telemetry data is presented, enabling effective soft failure detection and recovery strategies enforced in just a few microseconds

Network Slicing Automation: Challenges and Benefits

Network slicing is a technique widely used in 5G networks where multiple logical networks (i.e., slices) run over a single shared physical infrastructure. Each slice may realize one or multiple services, whose specific requirements are negotiated beforehand and regulated through Service Level Agreements (SLAs).\ua0 In Beyond 5G (B5G) networks it is envisioned that slices should be created, deployed, and managed in an automated fashion (i.e., without human intervention) irrespective of the technological and administrative domains over which a slice may span.\ua0Achieving this vision requires a combination of novel physical layer technologies, artificial intelligence tools, standard interfaces, network function virtualization, and software-defined networking principles. This paper provides an overview of the challenges facing network slicing automation with a focus on transport networks. Results from a selected group of use cases show the benefits of applying conventional optimization tools and machine-learning-based techniques while addressing some slicing design and provisioning problems

Reinforcement Learning for Slicing in a 5G Flexible RAN

Network slicing enables an infrastructure provider (InP) to support heterogeneous 5G services over a common platform (i.e., by creating a customized slice for each service). Once in operation, slices can be dynamically scaled up/down to match the variation of their service requirements. An InP generates revenue by accepting a slice request. If a slice cannot be scaled up when required, an InP has to also pay a penalty (proportional to the level of service degradation). It becomes then crucial for an InP to decide which slice requests should be accepted/rejected in order to increase its net profit. \ua0This paper presents a slice admission strategy based on reinforcement learning (RL) in the presence of services with different priorities. The use case considered is a 5G flexible radio access network (RAN), where slices of different mobile service providers are virtualized over the same RAN infrastructure. The proposed policy learns which are the services with the potential to bring high profit (i.e., high revenue with low degradation penalty), and hence should be accepted.\ua0The performance of the RL-based admission policy is compared against two deterministic heuristics. Results show that in the considered scenario, the proposed strategy outperforms the benchmark heuristics by at least 55%. Moreover, this paper shows how the policy is able to adapt to different conditions in terms of: (i)slice degradation penalty vs. slice revenue factors, and (ii)proportion of high vs. low priority services

Cost Benefits of Centralizing Service Processing in 5G Network Infrastructures

We assess the benefits of centralizing service processing in a few high-scale data center locations within an operator infrastructure. Results show up to 74% less cost while provisioning latency and availability constrained services

Benefits of Pod dimensioning with best-effort resources in bare metal cloud native deployments

Container orchestration platforms automatically adjust resources to evolving traffic conditions. However, these scaling mechanisms are reactive and may lead to service degradation. Traditionally, resource dimensioning has been performed considering guaranteed (or request) resources. Recently, container orchestration platforms included the possibility of allocating idle (or limit) resources for a short time in a best-effort fashion. This paper analyzes the potential of using limit resources as a way to mitigate degradation while reducing the number of allocated request resources. Results show that a 25% CPU reduction can be achieved by relying on limit resources

Scalable Physical Layer Security Components for Microservice-Based Optical SDN Controllers

We propose and demonstrate a set of microservice-based security components able to perform physical layer security assessment and mitigation in optical networks. Results illustrate the scalability of the attack detection mechanism and the agility in mitigating attacks