Toward multilayer disaggregated node telemetry and local decision making

A generic node agent supporting disaggregated node telemetry is presented. Data collection close to devices enable making local decisions, leveraging SDN controllers for network-wide operations. The agent is demonstrated in a BER-triggered transponder reconfiguration scenario.Peer ReviewedPostprint (published version

Autonomic disaggregated multilayer networking

Focused on reducing capital expenditures by opening the data plane to multiple vendors without impacting performance, node disaggregation is attracting the interest of network operators. Although the software-defined networking (SDN) paradigm is key for the control of such networks, the increased complexity of multilayer networks strictly requires monitoring/telemetry and data analytics capabilities to assist in creating and operating self-managed (autonomic) networks. Such autonomicity greatly reduces operational expenditures, while improving network performance. In this context, a monitoring and data analytics (MDA) architecture consisting of centralized data storage with data analytics capabilities, together with a generic node agent for monitoring/telemetry supporting disaggregation, is presented. A YANG data model that allows one to clearly separate responsibilities for monitoring configuration from node configuration is also proposed. The MDA architecture and YANG data models are experimentally demonstrated through three different use cases: i) virtual link creation supported by an optical connection, where monitoring is automatically activated; ii) multilayer self-configuration after bit error rate (BER) degradation detection, where a modulation format adaptation is recommended for the SDN controller to minimize errors (this entails reducing the capacity of both the virtual link and supported multiprotocol label switching-transport profile (MPLS-TP) paths); and iii) optical layer selfhealing, including failure localization at the optical layer to find the cause of BER degradation. A combination of active and passive monitoring procedures allows one to localize the cause of the failure, leading to lightpath rerouting recommendations toward the SDN controller avoiding the failing element(s).Peer ReviewedPostprint (author's final draft

Optical network automation [Invited tutorial]

© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Increased levels of automation will be necessary in view of more stringent performance requirements that next generation optical transport networks need to support in a near term, not only for high capacity, but, even more importantly, for dynamicity, latency, and availability. All these aspects will become more relevant with the growing complexity of modern networks. Network automation targets resource reoptimization to rapidly adapt the network to the expected conditions, quick degradation detection to improve the quality of the connections, as well as failure detection and identification to facilitate maintenance. Network automation requires and implies the collection of data for performance monitoring, being then elaborated by data analytics algorithms to produce meaningful inputs for the network controller, which will finally program the underlying devices. In this paper, we analyze alternative architectures for monitoring and data analytics (MDA) and illustrative control loops are presented aiming at validating the usefulness of MDA to automate optical networks operation.The research leading to these results has received funding from the Spanish MINECO TWINS project (TEC2017-90097-R), and from the Catalan Institution for Research and Advanced Studies (ICREA).Peer ReviewedPostprint (author's final draft

Monitoring and Data Analytics for Optical Networking:Benefits, Architectures, and Use Cases

Operators' network management continuously measures network health by collecting data from the deployed network devices; data is used mainly for performance reporting and diagnosing network problems after failures, as well as by human capacity planners to predict future traffic growth. Typically, these network management tools are generally reactive and require significant human effort and skills to operate effectively. As optical networks evolve to fulfil highly flexible connectivity and dynamicity requirements, and supporting ultra-low latency services, they must also provide reliable connectivity and increased network resource efficiency. Therefore, reactive human-based network measurement and management will be a limiting factor in the size and scale of these new networks. Future optical networks must support fully automated management, providing dynamic resource re-optimization to rapidly adapt network resources based on predicted conditions and events; identify service degradation conditions that will eventually impact connectivity and highlight critical devices and links for further inspection; and augment rapid protection schemes if a failure is predicted or detected, and facilitate resource optimization after restoration events. Applying automation techniques to network management requires both the collection of data from a variety of sources at various time frequencies, but it must also support the capability to extract knowledge and derive insight for performance monitoring, troubleshooting, and maintain network service continuity. Innovative analytics algorithms must be developed to derive meaningful input to the entities that orchestrate and control network resources; these control elements must also be capable of proactively programming the underlying optical infrastructure. In this article, we review the emerging requirements for optical network management automation, the capabilities of current optical systems, and the development and standardization status of data models and protocols to facilitate automated network monitoring. Finally, we propose an architecture to provide Monitoring and Data Analytics (MDA) capabilities, we present illustrative control loops for advanced network monitoring use cases, and the findings that validate the usefulness of MDA to provide automated optical network management

A control and management architecture supporting autonomic NFV services

The proposed control, orchestration and management (COM) architecture is presented from a high-level point of view; it enables the dynamic provisioning of services such as network data connectivity or generic network slicing instances based on virtual network functions (VNF). The COM is based on Software Defined Networking (SDN) principles and is hierarchical, with a dedicated controller per technology domain. Along with the SDN control plane for the provisioning of connectivity, an ETSI NFV management and orchestration system is responsible for the instantiation of Network Services, understood in this context as interconnected VNFs. A key, novel component of the COM architecture is the monitoring and data analytics (MDA) system, able to collect monitoring data from the network, datacenters and applications which outputs can be used to proactively reconfigure resources thus adapting to future conditions, like load or degradations. To illustrate the COM architecture, a use case of a Content Delivery Network service taking advantage of the MDA ability to collect and deliver monitoring data is experimentally demonstrated.Peer ReviewedPostprint (author's final draft

Autonomous operations in optical networks

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Machine Learning (ML) has already proven its benefits for network operation, being a sub-domain of artificial intelligence, it is highly suitable for complex system representation. In this paper, basic ML concepts are reviewed, as well as its integration into existing network control and management planes. Then, a use case focused on soft-failure detection is presented in detail covering optical spectrum analysis and ML algorithms; the technique relies on the widespread deployment of cost-effective optical spectrum analyzer (OSA). Finally, the retrieved optical parameters are analyzed using ML algorithms giving rise to illustrative results.The research leading to these results has received funding from the Spanish MINECO TWINS project (TEC2017-90097-R), from the EC through the METRO-HAUL project (G.A. no 761727), and from the Catalan Institution for Research and Advanced Studies (ICREA).Peer ReviewedPostprint (author's final draft

Knowledge management in optical networks: architecture, methods, and use cases [Invited]

© [2019 Optical Society of America]. Users may use, reuse, and build upon the article, or use the article for text or data mining, so long as such uses are for non-commercial purposes and appropriate attribution is maintained. All other rights are reserved.Autonomous network operation realized by means of control loops, where prediction from machine learning (ML) models is used as input to proactively reconfigure individual optical devices or the whole optical network, has been recently proposed to minimize human intervention. A general issue in this approach is the limited accuracy of ML models due to the lack of real data for training the models. Although the training dataset can be complemented with data from lab experiments and simulation, it is probable that once in operation, events not considered during the training phase appear and thus lead to model inaccuracies. A feasible solution is to implement self-learning approaches, where model inaccuracies are used to re-train the models in the field and to spread such data for training models being used for devices of the same type in other nodes in the network. In this paper, we develop the concept of collective self-learning aiming at improving the model’s error convergence time as well as at minimizing the amount of data being shared and stored. To this end, we propose a knowledge management (KM) process and an architecture to support it. Besides knowledge usage, the KM process entails knowledge discovery, knowledge sharing, and knowledge assimilation. Specifically, knowledge sharing and assimilation are based on distributing and combining ML models, so specific methods are proposed for combining models. Two use cases are used to evaluate the proposed KM architecture and methods. Exhaustive simulation results show that model-based KM provides the best error convergence time with reduced data being shared.Peer ReviewedPostprint (author's final draft

Distributed collaborative knowledge management for optical network

Network automation has been long time envisioned. In fact, the Telecommunications Management Network (TMN), defined by the International Telecommunication Union (ITU), is a hierarchy of management layers (network element, network, service, and business management), where high-level operational goals propagate from upper to lower layers. The network management architecture has evolved with the development of the Software Defined Networking (SDN) concept that brings programmability to simplify configuration (it breaks down high-level service abstraction into lower-level device abstractions), orchestrates operation, and automatically reacts to changes or events. Besides, the development and deployment of solutions based on Artificial Intelligence (AI) and Machine Learning (ML) for making decisions (control loop) based on the collected monitoring data enables network automation, which targets at reducing operational costs. AI/ML approaches usually require large datasets for training purposes, which are difficult to obtain. The lack of data can be compensated with a collective self-learning approach. In this thesis, we go beyond the aforementioned traditional control loop to achieve an efficient knowledge management (KM) process that enhances network intelligence while bringing down complexity. In this PhD thesis, we propose a general architecture to support KM process based on four main pillars, which enable creating, sharing, assimilating and using knowledge. Next, two alternative strategies based on model inaccuracies and combining model are proposed. To highlight the capacity of KM to adapt to different applications, two use cases are considered to implement KM in a purely centralized and distributed optical network architecture. Along with them, various policies are considered for evaluating KM in data- and model- based strategies. The results target to minimize the amount of data that need to be shared and reduce the convergence error. We apply KM to multilayer networks and propose the PILOT methodology for modeling connectivity services in a sandbox domain. PILOT uses active probes deployed in Central Offices (COs) to obtain real measurements that are used to tune a simulation scenario reproducing the real deployment with high accuracy. A simulator is eventually used to generate large amounts of realistic synthetic data for ML training and validation. We apply KM process also to a more complex network system that consists of several domains, where intra-domain controllers assist a broker plane in estimating accurate inter-domain delay. In addition, the broker identifies and corrects intra-domain model inaccuracies, as well as it computes an accurate compound model. Such models can be used for quality of service (QoS) and accurate end-to-end delay estimations. Finally, we investigate the application on KM in the context of Intent-based Networking (IBN). Knowledge in terms of traffic model and/or traffic perturbation is transferred among agents in a hierarchical architecture. This architecture can support autonomous network operation, like capacity management.La automatización de la red se ha concebido desde hace mucho tiempo. De hecho, la red de gestión de telecomunicaciones (TMN), definida por la Unión Internacional de Telecomunicaciones (ITU), es una jerarquía de capas de gestión (elemento de red, red, servicio y gestión de negocio), donde los objetivos operativos de alto nivel se propagan desde las capas superiores a las inferiores. La arquitectura de gestión de red ha evolucionado con el desarrollo del concepto de redes definidas por software (SDN) que brinda capacidad de programación para simplificar la configuración (descompone la abstracción de servicios de alto nivel en abstracciones de dispositivos de nivel inferior), organiza la operación y reacciona automáticamente a los cambios o eventos. Además, el desarrollo y despliegue de soluciones basadas en inteligencia artificial (IA) y aprendizaje automático (ML) para la toma de decisiones (bucle de control) en base a los datos de monitorización recopilados permite la automatización de la red, que tiene como objetivo reducir costes operativos. AI/ML generalmente requieren un gran conjunto de datos para entrenamiento, los cuales son difíciles de obtener. La falta de datos se puede compensar con un enfoque de autoaprendizaje colectivo. En esta tesis, vamos más allá del bucle de control tradicional antes mencionado para lograr un proceso eficiente de gestión del conocimiento (KM) que mejora la inteligencia de la red al tiempo que reduce la complejidad. En esta tesis doctoral, proponemos una arquitectura general para apoyar el proceso de KM basada en cuatro pilares principales que permiten crear, compartir, asimilar y utilizar el conocimiento. A continuación, se proponen dos estrategias alternativas basadas en inexactitudes del modelo y modelo de combinación. Para resaltar la capacidad de KM para adaptarse a diferentes aplicaciones, se consideran dos casos de uso para implementar KM en una arquitectura de red óptica puramente centralizada y distribuida. Junto a ellos, se consideran diversas políticas para evaluar KM en estrategias basadas en datos y modelos. Los resultados apuntan a minimizar la cantidad de datos que deben compartirse y reducir el error de convergencia. Aplicamos KM a redes multicapa y proponemos la metodología PILOT para modelar servicios de conectividad en un entorno aislado. PILOT utiliza sondas activas desplegadas en centrales de telecomunicación (CO) para obtener medidas reales que se utilizan para ajustar un escenario de simulación que reproducen un despliegue real con alta precisión. Un simulador se utiliza finalmente para generar grandes cantidades de datos sintéticos realistas para el entrenamiento y la validación de ML. Aplicamos el proceso de KM también a un sistema de red más complejo que consta de varios dominios, donde los controladores intra-dominio ayudan a un plano de bróker a estimar el retardo entre dominios de forma precisa. Además, el bróker identifica y corrige las inexactitudes de los modelos intra-dominio, así como también calcula un modelo compuesto preciso. Estos modelos se pueden utilizar para estimar la calidad de servicio (QoS) y el retardo extremo a extremo de forma precisa. Finalmente, investigamos la aplicación en KM en el contexto de red basada en intención (IBN). El conocimiento en términos de modelo de tráfico y/o perturbación del tráfico se transfiere entre agentes en una arquitectura jerárquica. Esta arquitectura puede soportar el funcionamiento autónomo de la red, como la gestión de la capacidad.Postprint (published version

Resource Management in Softwarized Networks

Communication networks are undergoing a major transformation through softwarization, which is changing the way networks are designed, operated, and managed. Network Softwarization is an emerging paradigm where software controls the treatment of network flows, adds value to these flows by software processing, and orchestrates the on-demand creation of customized networks to meet the needs of customer applications. Software-Defined Networking (SDN), Network Function Virtualization (NFV), and Network Virtualization are three cornerstones of the overall transformation trend toward network softwarization. Together, they are empowering network operators to accelerate time-to-market for new services, diversify the supply chain for networking hardware and software, bringing the benefits of agility, economies of scale, and flexibility of cloud computing to networks. The enhanced programmability enabled by softwarization creates unique opportunities for adapting network resources in support of applications and users with diverse requirements. To effectively leverage the flexibility provided by softwarization and realize its full potential, it is of paramount importance to devise proper mechanisms for allocating resources to different applications and users and for monitoring their usage over time. The overarching goal of this dissertation is to advance state-of-the-art in how resources are allocated and monitored and build the foundation for effective resource management in softwarized networks. Specifically, we address four resource management challenges in three key enablers of network softwarization, namely SDN, NFV, and network virtualization. First, we challenge the current practice of realizing network services with monolithic software network functions and propose a microservice-based disaggregated architecture enabling finer-grained resource allocation and scaling. Then, we devise optimal solutions and scalable heuristics for establishing virtual networks with guaranteed bandwidth and guaranteed survivability against failure on multi-layer IP-over-Optical and single-layer IP substrate network, respectively. Finally, we propose adaptive sampling mechanisms for balancing the overhead of softwarized network monitoring and the accuracy of the network view constructed from monitoring data

Learning life cycle to speed up autonomic optical transmission and networking adoption

Autonomic optical transmission and networking requires machine learning (ML) models to be trained with large datasets. However, the availability of enough real data to produce accurate ML models is rarely ensured since new optical equipment and techniques are continuously being deployed in the network. One option is to generate data from simulations and lab experiments, but such data could not cover the whole features space and would translate into inaccuracies in the ML models. In this paper, we propose an ML-based algorithm life cycle to facilitate ML deployment in real operator networks. The dataset for ML training can be initially populated based on the results from simulations and lab experiments. Once ML models are generated, ML retraining can be performed after inaccuracies are detected to improve their precision. Illustrative numerical results show the benefits of the proposed learning cycle for general use cases. In addition, two specific use cases are proposed and demonstrated that implement different learning strategies: (i) a two-phase strategy performing out-of-field training using data from simulations and lab experiments with generic equipment, followed by an in-field adaptation to support heterogeneous equipment (the accuracy of this strategy is shown for a use case of failure detection and identification), and (ii) in-field retraining, where ML models are retrained after detecting model inaccuracies. Different approaches are analyzed and evaluated for a use case of autonomic transmission, where results show the significant benefits of collective learning.Peer ReviewedPostprint (published version