Centralized and Distributed Machine Learning-Based QoT Estimation for Sliceable Optical Networks

Dynamic network slicing has emerged as a promising and fundamental framework for meeting 5G's diverse use cases. As machine learning (ML) is expected to play a pivotal role in the efficient control and management of these networks, in this work we examine the ML-based Quality-of-Transmission (QoT) estimation problem under the dynamic network slicing context, where each slice has to meet a different QoT requirement. We examine ML-based QoT frameworks with the aim of finding QoT model/s that are fine-tuned according to the diverse QoT requirements. Centralized and distributed frameworks are examined and compared according to their accuracy and training time. We show that the distributed QoT models outperform the centralized QoT model, especially as the number of diverse QoT requirements increases.Comment: accepted for presentation at the IEEE GLOBECOM 201

Artificial intelligence (AI) methods in optical networks: A comprehensive survey

Producción CientíficaArtificial intelligence (AI) is an extensive scientific discipline which enables computer systems to solve problems by emulating complex biological processes such as learning, reasoning and self-correction. This paper presents a comprehensive review of the application of AI techniques for improving performance of optical communication systems and networks. The use of AI-based techniques is first studied in applications related to optical transmission, ranging from the characterization and operation of network components to performance monitoring, mitigation of nonlinearities, and quality of transmission estimation. Then, applications related to optical network control and management are also reviewed, including topics like optical network planning and operation in both transport and access networks. Finally, the paper also presents a summary of opportunities and challenges in optical networking where AI is expected to play a key role in the near future.Ministerio de Economía, Industria y Competitividad (Project EC2014-53071-C3-2-P, TEC2015-71932-REDT

An Overview on Application of Machine Learning Techniques in Optical Networks

Today's telecommunication networks have become sources of enormous amounts of widely heterogeneous data. This information can be retrieved from network traffic traces, network alarms, signal quality indicators, users' behavioral data, etc. Advanced mathematical tools are required to extract meaningful information from these data and take decisions pertaining to the proper functioning of the networks from the network-generated data. Among these mathematical tools, Machine Learning (ML) is regarded as one of the most promising methodological approaches to perform network-data analysis and enable automated network self-configuration and fault management. The adoption of ML techniques in the field of optical communication networks is motivated by the unprecedented growth of network complexity faced by optical networks in the last few years. Such complexity increase is due to the introduction of a huge number of adjustable and interdependent system parameters (e.g., routing configurations, modulation format, symbol rate, coding schemes, etc.) that are enabled by the usage of coherent transmission/reception technologies, advanced digital signal processing and compensation of nonlinear effects in optical fiber propagation. In this paper we provide an overview of the application of ML to optical communications and networking. We classify and survey relevant literature dealing with the topic, and we also provide an introductory tutorial on ML for researchers and practitioners interested in this field. Although a good number of research papers have recently appeared, the application of ML to optical networks is still in its infancy: to stimulate further work in this area, we conclude the paper proposing new possible research directions

Autonomous and reliable operation of multilayer optical networks

This Ph.D. thesis focuses on the reliable autonomous operation of multilayer optical networks. The first objective focuses on the reliability of the optical network and proposes methods for health analysis related to Quality of Transmission (QoT) degradation. Such degradation is produced by soft-failures in optical devices and fibers in core and metro segments of the operators’ transport networks. Here, we compare estimated and measured QoT in the optical transponder by using a QoT tool based on GNPy. We show that the changes in the values of input parameters of the QoT model representing optical devices can explain the deviations and degradation in performance of such devices. We use reverse engineering to estimate the value of those parameters that explain the observed QoT. We show by simulation a large anticipation in soft-failure detection, localization and identification of degradation before affecting the network. Finally, for validating our approach, we experimentally observe the high accuracy in the estimation of the modeling parameters. The second objective focuses on multilayer optical networks, where lightpaths are used to connect packet nodes thus creating virtual links (vLink). Specifically, we study how lightpaths can be managed to provide enough capacity to the packet layer without detrimental effects in their Quality of Service (QoS), like added delays or packet losses, and at the same time minimize energy consumption. Such management must be as autonomous as possible to minimize human intervention. We study the autonomous operation of optical connections based on digital subcarrier multiplexing (DSCM). We propose several solutions for the autonomous operation of DSCM systems. In particular, the combination of two modules running in the optical node and in the optical transponder activate and deactivate subcarriers to adapt the capacity of the optical connection to the upper layer packet traffic. The module running in the optical node is part of our Intent-based Networking (IBN) solution and implements prediction to anticipate traffic changes. Our comprehensive study demonstrates the feasibility of DSCM autonomous operation and shows large cost savings in terms of energy consumption. In addition, our study provides a guideline to help vendors and operators to adopt the proposed solutions. The final objective targets at automating packet layer connections (PkC). Automating the capacity required by PkCs can bring further cost reduction to network operators, as it can limit the resources used at the optical layer. However, such automation requires careful design to avoid any QoS degradation, which would impact Service Level Agreement (SLA) in the case that the packet flow is related to some customer connection. We study autonomous packet flow capacity management. We apply RL techniques and propose a management lifecycle consisting of three different phases: 1) a self-tuned threshold-based approach for setting up the connection until enough data is collected, which enables understanding the traffic characteristics; 2) RL operation based on models pre-trained with generic traffic profiles; and 3) RL operation based on models trained with the observed traffic. We show that RL algorithms provide poor performance until they learn optimal policies, as well as when the traffic characteristics change over time. The proposed lifecycle provides remarkable performance from the starting of the connection and it shows the robustness while facing changes in traffic. The contribution is twofold: 1) and on the one hand, we propose a solution based on RL, which shows superior performance with respect to the solution based on prediction; and 2) because vLinks support packet connections, coordination between the intents of both layers is proposed. In this case, the actions taken by the individual PkCs are used by the vLink intent. The results show noticeable performance compared to independent vLink operation.Esta tesis doctoral se centra en la operación autónoma y confiable de redes ópticas multicapa. El primer objetivo se centra en la fiabilidad de la red óptica y propone métodos para el análisis del estado relacionados con la degradación de la calidad de la transmisión (QoT). Dicha degradación se produce por fallos en dispositivos ópticos y fibras en las redes de transporte de los operadores que no causan el corte de la señal. Comparamos el QoT estimado y medido en el transpondedor óptico mediante el uso de una herramienta de QoT basada en GNPy. Mostramos que los cambios en los valores de los parámetros de entrada del modelo QoT que representan los dispositivos ópticos pueden explicar las desviaciones y la degradación en el rendimiento de dichos dispositivos. Usamos ingeniería inversa para estimar el valor de aquellos parámetros que explican el QoT observado. Mostramos, mediante simulación, una gran anticipación en la detección, localización e identificación de fallas leves antes de afectar la red. Finalmente, validamos nuestro método de forma experimental y comprobamos la alta precisión en la estimación de los parámetros de los modelos. El segundo objetivo se centra en las redes ópticas multicapa, donde se utilizan conexiones ópticas (lightpaths) para conectar nodos de paquetes creando así enlaces virtuales (vLink). Específicamente, estudiamos cómo se pueden gestionar los lightpaths para proporcionar suficiente capacidad a la capa de paquetes sin efectos perjudiciales en su calidad de servicio (QoS), como retrasos adicionales o pérdidas de paquetes, y al mismo tiempo minimizar el consumo de energía. Estudiamos el funcionamiento autónomo de conexiones ópticas basadas en multiplexación de subportadoras digitales (DSCM) y proponemos soluciones para su funcionamiento autónomo. En particular, la combinación de dos módulos que se ejecutan en el nodo óptico y en el transpondedor óptico activan y desactivan subportadoras para adaptar la capacidad de la conexión óptica al tráfico de paquetes. El módulo que se ejecuta en el nodo óptico implementa la predicción para anticipar los cambios de tráfico. Nuestro estudio demuestra la viabilidad de la operación autónoma de DSCM y muestra un gran ahorro de consumo de energía. El objetivo final es la automatización de conexiones de capa de paquete (PkC). La automatización de la capacidad requerida por las PkC puede generar una mayor reducción de costes, ya que puede limitar los recursos utilizados en la capa óptica. Sin embargo, dicha automatización requiere un diseño cuidadoso para evitar cualquier degradación de QoS, lo que afectaría acuerdos de nivel de servicio (SLA) en el caso de que el flujo de paquetes esté relacionado con alguna conexión del cliente. Estudiamos la gestión autónoma de la capacidad del flujo de paquetes. Aplicamos RL y proponemos un ciclo de vida de gestión con tres fases: 1) un enfoque basado en umbrales auto ajustados para configurar la conexión hasta que se recopilen suficientes datos, lo que permite comprender las características del tráfico; 2) operación RL basada en modelos pre-entrenados con perfiles de tráfico genéricos; y 3) operación de RL en base a modelos entrenados con el tránsito observado. Mostramos que los algoritmos de RL ofrecen un desempeño deficiente hasta que aprenden las políticas óptimas, así cuando las características del tráfico cambian con el tiempo. El ciclo de vida propuesto proporciona un rendimiento notable desde el inicio de la conexión y muestra la robustez frente a cambios en el tráfico. La contribución es doble: 1) proponemos una solución basada en RL que muestra un rendimiento superior que la solución basada en predicción; y 2) debido a que los vLinks admiten conexiones de paquetes, se propone la coordinación entre las intenciones de ambas capas. En este caso, la intención de vLink utiliza las acciones realizadas por los PkC individuales. Los resultados muestran un rendimiento notable en comparación con la operación independiente de vLink.Postprint (published version

Machine Learning for Multi-Layer Open and Disaggregated Optical Networks

L'abstract è presente nell'allegato / the abstract is in the attachmen

Study and application of spectral monitoring techniques for optical network optimization

One of the possible ways to address the constantly increasing amount of heterogeneous and variable internet traffic is the evolution of the current optical networks towards a more flexible, open, and disaggregated paradigm. In such scenarios, the role played by Optical Performance Monitoring (OPM) is fundamental. In fact, OPM allows to balance performance and specification mismatches resulting from the disaggregation adoption and provides the control plane with the necessary feedback to grant the optical networks an adequate automation level. Therefore, new flexible and cost-effective OPM solutions are needed, as well as novel techniques to extract the desired information from the monitored data and process and apply them. In this dissertation, we focus on three aspects related to OPM. We first study a monitoring data plane scheme to acquire the high resolution signal optical spectra in a nonintrusive way. In particular, we propose a coherent detection based Optical Spectrum Analyzer (OSA) enhanced with specific Digital Signal Processing (DSP) to detect spectral slices of the considered optical signals. Then, we identify two main placement strategies for such monitoring solutions, enhancing them using two spectral processing techniques to estimate signal- and optical filter-related parameters. Specifically, we propose a way to estimate the Amplified Spontaneous Emission (ASE) noise or its related Optical Signal-to-Noise (OSNR) using optical spectra acquired at the egress ports of the network nodes and the filter central frequency and 3/6 dB bandwidth, using spectra captured at the ingress ports of the network nodes. To do so, we leverage Machine Learning (ML) algorithms and the function fitting principle, according to the considered scenario. We validate both the monitoring strategies and their related processing techniques through simulations and experiments. The obtained results confirm the validity of the two proposed estimation approaches. In particular, we are able to estimate in-band the OSNR/ASE noise within an egress monitor placement scenario, with a Maximum Absolute Error (MAE) lower than 0.4 dB. Moreover, we are able to estimate the filter central frequency and 3/6 dB bandwidth, within an ingress optical monitor placement scenario, with a MAE lower than 0.5 GHz and 0.98 GHz, respectively. Based on such evaluations, we also compare the two placement scenarios and provide guidelines on their implementation. According to the analysis of specific figures of merit, such as the estimation of the Signal-to-Noise Ratio (SNR) penalty introduced by an optical filter, we identify the ingress monitoring strategy as the most promising. In fact, when compared to scenarios where no monitoring strategy is adopted, the ingress one reduced the SNR penalty estimation by 92%. Finally, we identify a potential application for the monitored information. Specifically, we propose a solution for the optimization of the subchannel spectral spacing in a superchannel. Leveraging convex optimization methods, we implement a closed control loop process for the dynamical reconfiguration of the subchannel central frequencies to optimize specific Quality of Transmission (QoT)-related metrics. Such a solution is based on the information monitored at the superchannel receiver side. In particular, to make all the subchannels feasible, we consider the maximization of the total superchannel capacity and the maximization of the minimum superchannel subchannel SNR value. We validate the proposed approach using simulations, assuming scenarios with different subchannel numbers, signal characteristics, and starting frequency values. The obtained results confirm the effectiveness of our solution. Specifically, compared with the equally spaced subchannel scenario, we are able to improve the total and the minimum subchannel SNR values of a four subchannel superchannel, of 1.45 dB and 1.19 dB, respectively.Una de las posibles formas de hacer frente a la creciente cantidad de tráfico heterogéneo y variable de Internet es la evolución de las actuales redes ópticas hacia un paradigma más flexible, abierto y desagregado. En estos escenarios, el papel que desempeña el modulo óptico de monitorización de prestaciones (OPM) es fundamental. De hecho, el OPM permite equilibrar los desajustes de rendimiento y especificación, los cuales surgen con la adopción de la desagregación; del mismo modo el OPM también proporciona al plano de control la realimentación necesaria para otorgar un nivel de automatización adecuado a las redes ópticas. En esta tesis, nos centramos en tres aspectos relacionados con el OPM. En primer lugar, estudiamos un esquema de monitorización para adquirir, de forma no intrusiva, los espectros ópticos de señales de alta resolución. En concreto, proponemos un analizador de espectro óptico (OSA) basado en detección coherente y mejorado con un específico procesado digital de señal (DSP) para detectar cortes espectrales de las señales ópticas consideradas. A continuación, presentamos dos técnicas de colocación para dichas soluciones de monitorización, mejorándolas mediante dos técnicas de procesamiento espectral para estimar los parámetros relacionados con la señal y el filtro óptico. Específicamente, proponemos un método para estimar el ruido de emisión espontánea amplificada (ASE), o la relación de señal-ruido óptica (OSNR), utilizando espectros ópticos adquiridos en los puertos de salida de los nodos de la red. Del mismo modo, estimamos la frecuencia central del filtro y el ancho de banda de 3/6 dB, utilizando espectros capturados en los puertos de entrada de los nodos de la red. Para ello, aprovechamos los algoritmos de Machine Learning (ML) y el principio de function fitting, según el escenario considerado. Validamos tanto las estrategias de monitorización como las técnicas de procesamiento mediante simulaciones y experimentos. Se puede estimar en banda el ruido ASE/OSNR en un escenario de colocación de monitores de salida, con un Maximum Absolute Error (MAE) inferior a 0.4 dB. Además, se puede estimar la frecuencia central del filtro y el ancho de banda de 3/6 dB, dentro de un escenario de colocación de monitores ópticos de entrada, con un MAE inferior a 0.5 GHz y 0.98 GHz, respectivamente. A partir de estas evaluaciones, también comparamos los dos escenarios de colocación y proporcionamos directrices sobre su aplicación. Según el análisis de específicas figuras de mérito, como la estimación de la penalización de la relación señal-ruido (SNR) introducida por un filtro óptico, demostramos que la estrategia de monitorización de entrada es la más prometedora. De hecho, utilizar un sistema de monitorización de entrada redujo la estimación de la penalización del SNR en un 92%. Por último, identificamos una posible aplicación para la información monitorizada. En concreto, proponemos una solución para la optimización del espaciado espectral de los subcanales en un supercanal. Aprovechando los métodos de optimización convexa, implementamos un proceso cíclico de control cerrado para la reconfiguración dinámica de las frecuencias centrales de los subcanales con el fin de optimizar métricas específicas relacionadas con la calidad de la transmisión (QoT). Esta solución se basa en la información monitorizada en el lado del receptor del supercanal. Validamos el enfoque propuesto mediante simulaciones, asumiendo escenarios con un diferente número de subcanales, distintas características de la señal, y diversos valores de la frecuencia inicial. Los resultados obtenidos confirman la eficacia de nuestra solución. Más específicatamente, en comparación con el escenario de subcanales igualmente espaciados, se pueden mejorar los valores totales y minimos de SNR de los subcanales de un supercanal de cuatro subcanales, de 1.45 dB y 1.19 dB, respectivamentePostprint (published version

An Overview on Application of Machine Learning Techniques in Optical Networks

Today's telecommunication networks have become sources of enormous amounts of widely heterogeneous data. This information can be retrieved from network traffic traces, network alarms, signal quality indicators, users' behavioral data, etc. Advanced mathematical tools are required to extract meaningful information from these data and take decisions pertaining to the proper functioning of the networks from the network-generated data. Among these mathematical tools, machine learning (ML) is regarded as one of the most promising methodological approaches to perform network-data analysis and enable automated network self-configuration and fault management. The adoption of ML techniques in the field of optical communication networks is motivated by the unprecedented growth of network complexity faced by optical networks in the last few years. Such complexity increase is due to the introduction of a huge number of adjustable and interdependent system parameters (e.g., routing configurations, modulation format, symbol rate, coding schemes, etc.) that are enabled by the usage of coherent transmission/reception technologies, advanced digital signal processing, and compensation of nonlinear effects in optical fiber propagation. In this paper we provide an overview of the application of ML to optical communications and networking. We classify and survey relevant literature dealing with the topic, and we also provide an introductory tutorial on ML for researchers and practitioners interested in this field. Although a good number of research papers have recently appeared, the application of ML to optical networks is still in its infancy: to stimulate further work in this area, we conclude this paper proposing new possible research directions

Towards cognitive in-operation network planning

Next-generation internet services such as live TV and video on demand require high bandwidth and ultra-low latency. The ever-increasing volume, dynamicity and stringent requirements of these services’ demands are generating new challenges to nowadays telecom networks. To decrease expenses, service-layer content providers are delivering their content near the end users, thus allowing a low latency and tailored content delivery. As a consequence of this, unseen metro and even core traffic dynamicity is arising with changes in the volume and direction of the traffic along the day. A tremendous effort to efficiently manage networks is currently ongoing towards the realisation of 5G networks. This translates in looking for network architectures supporting dynamic resource allocation, fulfilling strict service requirements and minimising the total cost of ownership (TCO). In this regard, in-operation network planning was recently proven to successfully support various network reconfiguration use cases in prospective scenarios. Nevertheless, additional research to extend in-operation planning capabilities from typical reactive optimization schemes to proactive and predictive schemes based on the analysis of network monitoring data is required. A hot topic raising increasing attention is cognitive networking, where an elevated knowledge about the network could be obtained as a result of introducing data analytics in the telecom operator’s infrastructure. By using predictive knowledge about the network traffic, in-operation network planning mechanisms could be enhanced to efficiently adapt the network by means of future traffic prediction, thus achieving cognitive in-operation network planning. In this thesis, we focus on studying mechanisms to enable cognitive in-operation network planning in core networks. In particular, we focus on dynamically reconfiguring virtual network topologies (VNT) at the MPLS layer, covering a number of detailed objectives. First, we start studying mechanisms to allow network traffic flow modelling, from monitoring and data transformation to the estimation of predictive traffic model based on this data. By means of these traffic models, then we tackle a cognitive approach to periodically adapt the core VNT to current and future traffic, using predicted traffic matrices based on origin-destination (OD) predictive models. This optimization approach, named VENTURE, is efficiently solved using dedicated heuristic algorithms and its feasibility is demonstrated in an experimental in-operation network planning environment. Finally, we extend VENTURE to consider core flows dynamicity as a result of metro flows re-routing, which represents a meaningful dynamic traffic scenario. This extension, which entails enhancements to coordinate metro and core network controllers with the aim of allowing fast adaption of core OD traffic models, is evaluated and validated in terms of traffic models accuracy and experimental feasibility.Els serveis d’internet de nova generació tals com la televisió en viu o el vídeo sota demanda requereixen d’un gran ample de banda i d’ultra-baixa latència. L’increment continu del volum, dinamicitat i requeriments d’aquests serveis està generant nous reptes pels teleoperadors de xarxa. Per reduir costs, els proveïdors de contingut estan disposant aquests més a prop dels usuaris finals, aconseguint així una entrega de contingut feta a mida. Conseqüentment, estem presenciant una dinamicitat mai vista en el tràfic de xarxes de metro amb canvis en la direcció i el volum del tràfic al llarg del dia. Actualment, s’està duent a terme un gran esforç cap a la realització de xarxes 5G. Aquest esforç es tradueix en cercar noves arquitectures de xarxa que suportin l’assignació dinàmica de recursos, complint requeriments de servei estrictes i minimitzant el cost total de la propietat. En aquest sentit, recentment s’ha demostrat com l’aplicació de “in-operation network planning” permet exitosament suportar diversos casos d’ús de reconfiguració de xarxa en escenaris prospectius. No obstant, és necessari dur a terme més recerca per tal d’estendre “in-operation network planning” des d’un esquema reactiu d’optimització cap a un nou esquema proactiu basat en l’analítica de dades provinents del monitoritzat de la xarxa. El concepte de xarxes cognitives es també troba al centre d’atenció, on un elevat coneixement de la xarxa s’obtindria com a resultat d’introduir analítica de dades en la infraestructura del teleoperador. Mitjançant un coneixement predictiu sobre el tràfic de xarxa, els mecanismes de in-operation network planning es podrien millorar per adaptar la xarxa eficientment basant-se en predicció de tràfic, assolint així el que anomenem com a “cognitive in-operation network Planning”. En aquesta tesi ens centrem en l’estudi de mecanismes que permetin establir “el cognitive in-operation network Planning” en xarxes de core. En particular, ens centrem en reconfigurar dinàmicament topologies de xarxa virtual (VNT) a la capa MPLS, cobrint una sèrie d’objectius detallats. Primer comencem estudiant mecanismes pel modelat de fluxos de tràfic de xarxa, des del seu monitoritzat i transformació fins a l’estimació de models predictius de tràfic. Posteriorment, i mitjançant aquests models predictius, tractem un esquema cognitiu per adaptar periòdicament la VNT utilitzant matrius de tràfic basades en predicció de parells origen-destí (OD). Aquesta optimització, anomenada VENTURE, és resolta eficientment fent servir heurístiques dedicades i és posteriorment avaluada sota escenaris de tràfic de xarxa dinàmics. A continuació, estenem VENTURE considerant la dinamicitat dels fluxos de tràfic de xarxes de metro, el qual representa un escenari rellevant de dinamicitat de tràfic. Aquesta extensió involucra millores per coordinar els operadors de metro i core amb l’objectiu d’aconseguir una ràpida adaptació de models de tràfic OD. Finalment, proposem dues arquitectures de xarxa necessàries per aplicar els mecanismes anteriors en entorns experimentals, emprant protocols estat-de-l’art com són OpenFlow i IPFIX. La metodologia emprada per avaluar el treball anterior consisteix en una primera avaluació numèrica fent servir un simulador de xarxes íntegrament dissenyat i desenvolupat per a aquesta tesi. Després d’aquesta validació basada en simulació, la factibilitat experimental de les arquitectures de xarxa proposades és avaluada en un entorn de proves distribuït