Demonstration of monitoring and data analytics-triggered reconfiguration in partially disaggregated optical networks

©2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.We demonstrate a novel agent for optical disaggregated optical networks. When the Monitoring and Data Analytics detects a degradation, it recommends the SDN controller to trigger a network reconfiguration computed by a novel planning tool.The research leading to these results has received funding from the EC through the METRO-HAUL (G.A. nº 761727).Peer ReviewedPostprint (author's final draft

Monitoring and data analytics-triggered reconfiguration in partially disaggregated optical networks

©2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.We present ADONIS, which stands for Aggregator/Disaggregator for Optical Network equIpmentS, a novel open device agent able to construct logical network devices from (dis)aggregation of physical components in order to expose meaningful network devices to the SDN controller. We experimentally assess it by means of a control closed-loop involving ADONIS, a Software Defined Network controller, a Monitoring and Data Analytics system, and a novel reconfiguration tool, SMART-A.The research leading to these results has received funding from the EC through the METRO-HAUL (G.A. nº 761727).Peer ReviewedPostprint (author's final draft

Autonomic disaggregated multilayer networking

Focused on reducing capital expenditures by opening the data plane to multiple vendors without impacting performance, node disaggregation is attracting the interest of network operators. Although the software-defined networking (SDN) paradigm is key for the control of such networks, the increased complexity of multilayer networks strictly requires monitoring/telemetry and data analytics capabilities to assist in creating and operating self-managed (autonomic) networks. Such autonomicity greatly reduces operational expenditures, while improving network performance. In this context, a monitoring and data analytics (MDA) architecture consisting of centralized data storage with data analytics capabilities, together with a generic node agent for monitoring/telemetry supporting disaggregation, is presented. A YANG data model that allows one to clearly separate responsibilities for monitoring configuration from node configuration is also proposed. The MDA architecture and YANG data models are experimentally demonstrated through three different use cases: i) virtual link creation supported by an optical connection, where monitoring is automatically activated; ii) multilayer self-configuration after bit error rate (BER) degradation detection, where a modulation format adaptation is recommended for the SDN controller to minimize errors (this entails reducing the capacity of both the virtual link and supported multiprotocol label switching-transport profile (MPLS-TP) paths); and iii) optical layer selfhealing, including failure localization at the optical layer to find the cause of BER degradation. A combination of active and passive monitoring procedures allows one to localize the cause of the failure, leading to lightpath rerouting recommendations toward the SDN controller avoiding the failing element(s).Peer ReviewedPostprint (author's final draft

Toward multilayer disaggregated node telemetry and local decision making

A generic node agent supporting disaggregated node telemetry is presented. Data collection close to devices enable making local decisions, leveraging SDN controllers for network-wide operations. The agent is demonstrated in a BER-triggered transponder reconfiguration scenario.Peer ReviewedPostprint (published version

Demonstration of latency-aware 5G network slicing on optical metro networks

The H2020 METRO-HAUL European project has architected a latency-aware, cost-effective, agile, and programmable optical metro network. This includes the design of semi-disaggregated metro nodes with compute and storage capabilities, which interface effectively with both 5G access and multi-Tbit/s elastic optical networks in the core. In this paper, we report the automated deployment of 5G services, in particular, a public safety video surveillance use case employing low-latency object detection and tracking using on-camera and on-the-edge analytics. The demonstration features flexible deployment of network slice instances, implemented in terms of ETSI NFV Network Services. We summarize the key findings in a detailed analysis of end-to-end quality of service, service setup time, and soft-failure detection time. The results show that the round-trip-time over an 80 km link is under 800 µs and the service deployment time under 180 seconds.Horizon 2020 Framework Programme (761727); Bundesministerium für Bildung und Forschung (16KIS0979K).Peer ReviewedArticle signat per 25 autors/es: B. Shariati, Fraunhofer HHI, Berlin, Germany / L. Velasco, Universitat Politècnica de Catalunya, Barcelona, Spain / J.-J. Pedreno-Manresa, ADVA, Munich, Germany / A. Dochhan, ADVA, Munich, Germany / R. Casellas, Centre Tecnològic Telecomunicacions Catalunya, Castelldefels, Spain / A. Muqaddas, University of Bristol, Bristol, UK / O. Gonzalez de Dios, Telefónica, Madrid, Spain / L. Luque Canto, Telefónica, Madrid, Spain / B. Lent, Qognify GmbH, Bruchsal, Germany / J. E. Lopez de Vergara, Naudit HPCN, Madrid, Spain / S. Lopez-Buedo, Naudit HPCN, Madrid, Spain / F. Moreno, Universidad Politécnica de Cartagena, Cartagena, Spain / P. Pavon, Universidad Politécnica de Cartagena, Cartagena, Spain / M. Ruiz, Universitat Politècnica de Catalunya, Barcelona, Spain / S. K. Patri, ADVA, Munich, Germany / A. Giorgetti, CNIT, Pisa, Italy / F. Cugini, CNIT, Pisa, Italy / A. Sgambelluri, CNIT, Pisa, Italy / R. Nejabati, University of Bristol, Bristol, UK / D. Simeonidou, University of Bristol, Bristol, UK / R.-P. Braun, Deutsche Telekom, Germany / A. Autenrieth, ADVA, Munich, Germany / J.-P. Elbers, ADVA, Munich, Germany / J. K. Fischer, Fraunhofer HHI, Berlin, Germany / R. Freund, Fraunhofer HHI, Berlin, GermanyPostprint (author's final draft

An Overview on Application of Machine Learning Techniques in Optical Networks

Today's telecommunication networks have become sources of enormous amounts of widely heterogeneous data. This information can be retrieved from network traffic traces, network alarms, signal quality indicators, users' behavioral data, etc. Advanced mathematical tools are required to extract meaningful information from these data and take decisions pertaining to the proper functioning of the networks from the network-generated data. Among these mathematical tools, Machine Learning (ML) is regarded as one of the most promising methodological approaches to perform network-data analysis and enable automated network self-configuration and fault management. The adoption of ML techniques in the field of optical communication networks is motivated by the unprecedented growth of network complexity faced by optical networks in the last few years. Such complexity increase is due to the introduction of a huge number of adjustable and interdependent system parameters (e.g., routing configurations, modulation format, symbol rate, coding schemes, etc.) that are enabled by the usage of coherent transmission/reception technologies, advanced digital signal processing and compensation of nonlinear effects in optical fiber propagation. In this paper we provide an overview of the application of ML to optical communications and networking. We classify and survey relevant literature dealing with the topic, and we also provide an introductory tutorial on ML for researchers and practitioners interested in this field. Although a good number of research papers have recently appeared, the application of ML to optical networks is still in its infancy: to stimulate further work in this area, we conclude the paper proposing new possible research directions

Enabling P4 Network Telemetry in Edge Micro Data Centers With Kubernetes Orchestration

Integrating computation resources with networking technologies is an hot research topic targeting the optimization of containers deployment on a set of host machines interconnected by a network infrastructure. Particularly, next generation edge nodes will offer significant advantages leveraging on integrated computation resources and networking awareness, enabling configurable, granular and monitorable quality of service to different micro-services, applications and tenants, especially in terms of bounded end-to-end latency. In this regard, SDN is a key technology enabling network telemetry and traffic switching with the granularity of the single traffic flow. However, currently available solutions are based on legacy SDN techniques, not enabling the matching of tunneled traffic, and thus require a tricky integration inside the hosts where containers are deployed. This work considers Kubernetes clusters deployed on next generation edge micro data center platforms and proposes an innovative SDN solution exploiting the P4 technology to gain visibility inside tunnelled traffic exchanged among pods. This way, the integration is achieved at the control plane level through the communication between Kubernetes and the SDN controller. The proposed solution is experimentally validated including a comprehensive framework enabling effective traffic switching and in-band telemetry at pod level. The major paper contributions consist in the design and the development of: (i) the networking applications at SDN control plane level; (ii) the P4 switch pipeline at the data plane level; (iii) the monitoring system used to collect, aggregate and elaborate the telemetry data

Distributed collaborative knowledge management for optical network

Network automation has been long time envisioned. In fact, the Telecommunications Management Network (TMN), defined by the International Telecommunication Union (ITU), is a hierarchy of management layers (network element, network, service, and business management), where high-level operational goals propagate from upper to lower layers. The network management architecture has evolved with the development of the Software Defined Networking (SDN) concept that brings programmability to simplify configuration (it breaks down high-level service abstraction into lower-level device abstractions), orchestrates operation, and automatically reacts to changes or events. Besides, the development and deployment of solutions based on Artificial Intelligence (AI) and Machine Learning (ML) for making decisions (control loop) based on the collected monitoring data enables network automation, which targets at reducing operational costs. AI/ML approaches usually require large datasets for training purposes, which are difficult to obtain. The lack of data can be compensated with a collective self-learning approach. In this thesis, we go beyond the aforementioned traditional control loop to achieve an efficient knowledge management (KM) process that enhances network intelligence while bringing down complexity. In this PhD thesis, we propose a general architecture to support KM process based on four main pillars, which enable creating, sharing, assimilating and using knowledge. Next, two alternative strategies based on model inaccuracies and combining model are proposed. To highlight the capacity of KM to adapt to different applications, two use cases are considered to implement KM in a purely centralized and distributed optical network architecture. Along with them, various policies are considered for evaluating KM in data- and model- based strategies. The results target to minimize the amount of data that need to be shared and reduce the convergence error. We apply KM to multilayer networks and propose the PILOT methodology for modeling connectivity services in a sandbox domain. PILOT uses active probes deployed in Central Offices (COs) to obtain real measurements that are used to tune a simulation scenario reproducing the real deployment with high accuracy. A simulator is eventually used to generate large amounts of realistic synthetic data for ML training and validation. We apply KM process also to a more complex network system that consists of several domains, where intra-domain controllers assist a broker plane in estimating accurate inter-domain delay. In addition, the broker identifies and corrects intra-domain model inaccuracies, as well as it computes an accurate compound model. Such models can be used for quality of service (QoS) and accurate end-to-end delay estimations. Finally, we investigate the application on KM in the context of Intent-based Networking (IBN). Knowledge in terms of traffic model and/or traffic perturbation is transferred among agents in a hierarchical architecture. This architecture can support autonomous network operation, like capacity management.La automatización de la red se ha concebido desde hace mucho tiempo. De hecho, la red de gestión de telecomunicaciones (TMN), definida por la Unión Internacional de Telecomunicaciones (ITU), es una jerarquía de capas de gestión (elemento de red, red, servicio y gestión de negocio), donde los objetivos operativos de alto nivel se propagan desde las capas superiores a las inferiores. La arquitectura de gestión de red ha evolucionado con el desarrollo del concepto de redes definidas por software (SDN) que brinda capacidad de programación para simplificar la configuración (descompone la abstracción de servicios de alto nivel en abstracciones de dispositivos de nivel inferior), organiza la operación y reacciona automáticamente a los cambios o eventos. Además, el desarrollo y despliegue de soluciones basadas en inteligencia artificial (IA) y aprendizaje automático (ML) para la toma de decisiones (bucle de control) en base a los datos de monitorización recopilados permite la automatización de la red, que tiene como objetivo reducir costes operativos. AI/ML generalmente requieren un gran conjunto de datos para entrenamiento, los cuales son difíciles de obtener. La falta de datos se puede compensar con un enfoque de autoaprendizaje colectivo. En esta tesis, vamos más allá del bucle de control tradicional antes mencionado para lograr un proceso eficiente de gestión del conocimiento (KM) que mejora la inteligencia de la red al tiempo que reduce la complejidad. En esta tesis doctoral, proponemos una arquitectura general para apoyar el proceso de KM basada en cuatro pilares principales que permiten crear, compartir, asimilar y utilizar el conocimiento. A continuación, se proponen dos estrategias alternativas basadas en inexactitudes del modelo y modelo de combinación. Para resaltar la capacidad de KM para adaptarse a diferentes aplicaciones, se consideran dos casos de uso para implementar KM en una arquitectura de red óptica puramente centralizada y distribuida. Junto a ellos, se consideran diversas políticas para evaluar KM en estrategias basadas en datos y modelos. Los resultados apuntan a minimizar la cantidad de datos que deben compartirse y reducir el error de convergencia. Aplicamos KM a redes multicapa y proponemos la metodología PILOT para modelar servicios de conectividad en un entorno aislado. PILOT utiliza sondas activas desplegadas en centrales de telecomunicación (CO) para obtener medidas reales que se utilizan para ajustar un escenario de simulación que reproducen un despliegue real con alta precisión. Un simulador se utiliza finalmente para generar grandes cantidades de datos sintéticos realistas para el entrenamiento y la validación de ML. Aplicamos el proceso de KM también a un sistema de red más complejo que consta de varios dominios, donde los controladores intra-dominio ayudan a un plano de bróker a estimar el retardo entre dominios de forma precisa. Además, el bróker identifica y corrige las inexactitudes de los modelos intra-dominio, así como también calcula un modelo compuesto preciso. Estos modelos se pueden utilizar para estimar la calidad de servicio (QoS) y el retardo extremo a extremo de forma precisa. Finalmente, investigamos la aplicación en KM en el contexto de red basada en intención (IBN). El conocimiento en términos de modelo de tráfico y/o perturbación del tráfico se transfiere entre agentes en una arquitectura jerárquica. Esta arquitectura puede soportar el funcionamiento autónomo de la red, como la gestión de la capacidad.Postprint (published version

Towards cognitive in-operation network planning

Next-generation internet services such as live TV and video on demand require high bandwidth and ultra-low latency. The ever-increasing volume, dynamicity and stringent requirements of these services’ demands are generating new challenges to nowadays telecom networks. To decrease expenses, service-layer content providers are delivering their content near the end users, thus allowing a low latency and tailored content delivery. As a consequence of this, unseen metro and even core traffic dynamicity is arising with changes in the volume and direction of the traffic along the day. A tremendous effort to efficiently manage networks is currently ongoing towards the realisation of 5G networks. This translates in looking for network architectures supporting dynamic resource allocation, fulfilling strict service requirements and minimising the total cost of ownership (TCO). In this regard, in-operation network planning was recently proven to successfully support various network reconfiguration use cases in prospective scenarios. Nevertheless, additional research to extend in-operation planning capabilities from typical reactive optimization schemes to proactive and predictive schemes based on the analysis of network monitoring data is required. A hot topic raising increasing attention is cognitive networking, where an elevated knowledge about the network could be obtained as a result of introducing data analytics in the telecom operator’s infrastructure. By using predictive knowledge about the network traffic, in-operation network planning mechanisms could be enhanced to efficiently adapt the network by means of future traffic prediction, thus achieving cognitive in-operation network planning. In this thesis, we focus on studying mechanisms to enable cognitive in-operation network planning in core networks. In particular, we focus on dynamically reconfiguring virtual network topologies (VNT) at the MPLS layer, covering a number of detailed objectives. First, we start studying mechanisms to allow network traffic flow modelling, from monitoring and data transformation to the estimation of predictive traffic model based on this data. By means of these traffic models, then we tackle a cognitive approach to periodically adapt the core VNT to current and future traffic, using predicted traffic matrices based on origin-destination (OD) predictive models. This optimization approach, named VENTURE, is efficiently solved using dedicated heuristic algorithms and its feasibility is demonstrated in an experimental in-operation network planning environment. Finally, we extend VENTURE to consider core flows dynamicity as a result of metro flows re-routing, which represents a meaningful dynamic traffic scenario. This extension, which entails enhancements to coordinate metro and core network controllers with the aim of allowing fast adaption of core OD traffic models, is evaluated and validated in terms of traffic models accuracy and experimental feasibility.Els serveis d’internet de nova generació tals com la televisió en viu o el vídeo sota demanda requereixen d’un gran ample de banda i d’ultra-baixa latència. L’increment continu del volum, dinamicitat i requeriments d’aquests serveis està generant nous reptes pels teleoperadors de xarxa. Per reduir costs, els proveïdors de contingut estan disposant aquests més a prop dels usuaris finals, aconseguint així una entrega de contingut feta a mida. Conseqüentment, estem presenciant una dinamicitat mai vista en el tràfic de xarxes de metro amb canvis en la direcció i el volum del tràfic al llarg del dia. Actualment, s’està duent a terme un gran esforç cap a la realització de xarxes 5G. Aquest esforç es tradueix en cercar noves arquitectures de xarxa que suportin l’assignació dinàmica de recursos, complint requeriments de servei estrictes i minimitzant el cost total de la propietat. En aquest sentit, recentment s’ha demostrat com l’aplicació de “in-operation network planning” permet exitosament suportar diversos casos d’ús de reconfiguració de xarxa en escenaris prospectius. No obstant, és necessari dur a terme més recerca per tal d’estendre “in-operation network planning” des d’un esquema reactiu d’optimització cap a un nou esquema proactiu basat en l’analítica de dades provinents del monitoritzat de la xarxa. El concepte de xarxes cognitives es també troba al centre d’atenció, on un elevat coneixement de la xarxa s’obtindria com a resultat d’introduir analítica de dades en la infraestructura del teleoperador. Mitjançant un coneixement predictiu sobre el tràfic de xarxa, els mecanismes de in-operation network planning es podrien millorar per adaptar la xarxa eficientment basant-se en predicció de tràfic, assolint així el que anomenem com a “cognitive in-operation network Planning”. En aquesta tesi ens centrem en l’estudi de mecanismes que permetin establir “el cognitive in-operation network Planning” en xarxes de core. En particular, ens centrem en reconfigurar dinàmicament topologies de xarxa virtual (VNT) a la capa MPLS, cobrint una sèrie d’objectius detallats. Primer comencem estudiant mecanismes pel modelat de fluxos de tràfic de xarxa, des del seu monitoritzat i transformació fins a l’estimació de models predictius de tràfic. Posteriorment, i mitjançant aquests models predictius, tractem un esquema cognitiu per adaptar periòdicament la VNT utilitzant matrius de tràfic basades en predicció de parells origen-destí (OD). Aquesta optimització, anomenada VENTURE, és resolta eficientment fent servir heurístiques dedicades i és posteriorment avaluada sota escenaris de tràfic de xarxa dinàmics. A continuació, estenem VENTURE considerant la dinamicitat dels fluxos de tràfic de xarxes de metro, el qual representa un escenari rellevant de dinamicitat de tràfic. Aquesta extensió involucra millores per coordinar els operadors de metro i core amb l’objectiu d’aconseguir una ràpida adaptació de models de tràfic OD. Finalment, proposem dues arquitectures de xarxa necessàries per aplicar els mecanismes anteriors en entorns experimentals, emprant protocols estat-de-l’art com són OpenFlow i IPFIX. La metodologia emprada per avaluar el treball anterior consisteix en una primera avaluació numèrica fent servir un simulador de xarxes íntegrament dissenyat i desenvolupat per a aquesta tesi. Després d’aquesta validació basada en simulació, la factibilitat experimental de les arquitectures de xarxa proposades és avaluada en un entorn de proves distribuït