Spectrum Anomaly Detection for Optical Network Monitoring using Deep Unsupervised Learning

Accurate and efficient anomaly detection is a key enabler for the cognitive management of optical networks, but traditional anomaly detection algorithms are computationally complex and do not scale well with the amount of monitoring data. Therefore, we propose an optical spectrum anomaly detection scheme that exploits computer vision and deep unsupervised learning to perform optical network monitoring relying only on constellation diagrams of received signals. The proposed scheme achieves 100% detection accuracy even without prior knowledge of the anomalies. Furthermore, operation with encoded images of constellation diagrams reduces the runtime by up to 200 times

Experiment-based detection of service disruption attacks in optical networks using data analytics and unsupervised learning

The paper addresses the detection of malicious attacks targeting service disruption at the optical layer as a key prerequisite for fast and effective attack response and network recovery. We experimentally demonstrate the effects of signal insertion attacks with varying intensity in a real-life scenario. By applying data analytics tools, we analyze the properties of the obtained dataset to determine how the relationships among different optical performance monitoring (OPM) parameters of the signal change in the presence of an attack as opposed to the normal operating conditions. In addition, we evaluate the performance of an unsupervised learning technique, i.e., a clustering algorithm for anomaly detection, which can detect attacks as anomalies without prior knowledge of the attacks. We demonstrate the potential and the challenges of unsupervised learning for attack detection, propose guidelines for attack signature identification needed for the detection of the considered attack methods, and discuss remaining challenges related to optical network security

Learning from the optical spectrum: failure detection and identification [Invited]

© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksThe availability of coarse-resolution cost-effective Optical Spectrum Analyzers (OSA) allows its widespread deployment in operators’ networks. In this paper, we explore several machine learning approaches for soft-failure detection, identification and localization that take advantage of OSAs. In particular, we present three different solutions for the two most common filter-related soft-failures; filter shift and tight filtering which noticeably deform the expected shape of the optical spectrum. However, filter cascading is a key challenge as it affects the shape of the optical spectrum similarly to tight filtering; the approaches are specifically designed to avoid the misclassification of properly operating signals when normal filter cascading effects are present. The proposed solutions are: $i) multi-classifier$ approach, which uses features extracted directly from the optical spectrum, $ii) single-classifier$ approach, which uses pre-processed features to compensate for filter cascading, and $iii) residual-based$ approach, which uses a residual signal computed from subtracting the acquired single by OSAs from an expected signal synthetically generated. Extensive numerical results are ultimately presented to compare the performance of the proposed approaches in terms of accuracy and robustness.Peer ReviewedPostprint (author's final draft

A control and management architecture supporting autonomic NFV services

The proposed control, orchestration and management (COM) architecture is presented from a high-level point of view; it enables the dynamic provisioning of services such as network data connectivity or generic network slicing instances based on virtual network functions (VNF). The COM is based on Software Defined Networking (SDN) principles and is hierarchical, with a dedicated controller per technology domain. Along with the SDN control plane for the provisioning of connectivity, an ETSI NFV management and orchestration system is responsible for the instantiation of Network Services, understood in this context as interconnected VNFs. A key, novel component of the COM architecture is the monitoring and data analytics (MDA) system, able to collect monitoring data from the network, datacenters and applications which outputs can be used to proactively reconfigure resources thus adapting to future conditions, like load or degradations. To illustrate the COM architecture, a use case of a Content Delivery Network service taking advantage of the MDA ability to collect and deliver monitoring data is experimentally demonstrated.Peer ReviewedPostprint (author's final draft

Demonstration of latency-aware 5G network slicing on optical metro networks

The H2020 METRO-HAUL European project has architected a latency-aware, cost-effective, agile, and programmable optical metro network. This includes the design of semi-disaggregated metro nodes with compute and storage capabilities, which interface effectively with both 5G access and multi-Tbit/s elastic optical networks in the core. In this paper, we report the automated deployment of 5G services, in particular, a public safety video surveillance use case employing low-latency object detection and tracking using on-camera and on-the-edge analytics. The demonstration features flexible deployment of network slice instances, implemented in terms of ETSI NFV Network Services. We summarize the key findings in a detailed analysis of end-to-end quality of service, service setup time, and soft-failure detection time. The results show that the round-trip-time over an 80 km link is under 800 µs and the service deployment time under 180 seconds.Horizon 2020 Framework Programme (761727); Bundesministerium für Bildung und Forschung (16KIS0979K).Peer ReviewedArticle signat per 25 autors/es: B. Shariati, Fraunhofer HHI, Berlin, Germany / L. Velasco, Universitat Politècnica de Catalunya, Barcelona, Spain / J.-J. Pedreno-Manresa, ADVA, Munich, Germany / A. Dochhan, ADVA, Munich, Germany / R. Casellas, Centre Tecnològic Telecomunicacions Catalunya, Castelldefels, Spain / A. Muqaddas, University of Bristol, Bristol, UK / O. Gonzalez de Dios, Telefónica, Madrid, Spain / L. Luque Canto, Telefónica, Madrid, Spain / B. Lent, Qognify GmbH, Bruchsal, Germany / J. E. Lopez de Vergara, Naudit HPCN, Madrid, Spain / S. Lopez-Buedo, Naudit HPCN, Madrid, Spain / F. Moreno, Universidad Politécnica de Cartagena, Cartagena, Spain / P. Pavon, Universidad Politécnica de Cartagena, Cartagena, Spain / M. Ruiz, Universitat Politècnica de Catalunya, Barcelona, Spain / S. K. Patri, ADVA, Munich, Germany / A. Giorgetti, CNIT, Pisa, Italy / F. Cugini, CNIT, Pisa, Italy / A. Sgambelluri, CNIT, Pisa, Italy / R. Nejabati, University of Bristol, Bristol, UK / D. Simeonidou, University of Bristol, Bristol, UK / R.-P. Braun, Deutsche Telekom, Germany / A. Autenrieth, ADVA, Munich, Germany / J.-P. Elbers, ADVA, Munich, Germany / J. K. Fischer, Fraunhofer HHI, Berlin, Germany / R. Freund, Fraunhofer HHI, Berlin, GermanyPostprint (author's final draft

Soft failure localization during commissioning testing and lightpath operation

In elastic optical networks (EONs), effective soft failure localization is of paramount importance to early detection of service level agreement violations while anticipating possible hard failure events. So far, failure localization techniques have been proposed and deployed mainly for hard failures, while significant work is still required to provide effective and automated solutions for soft failures, both during commissioning testing and in-operation phases. In this paper, we focus on soft failure localization in EONs by proposing two techniques for active monitoring during commissioning testing and for passive in-operation monitoring. The techniques rely on specifically designed low-cost optical testing channel (OTC) modules and on the widespread deployment of cost-effective optical spectrum analyzers (OSAs). The retrieved optical parameters are elaborated by machine learning-based algorithms running in the agent’s node and in the network controller. In particular, the Testing optIcal Switching at connection SetUp timE (TISSUE) algorithm is proposed to localize soft failures by elaborating the estimated bit-error rate (BER) values provided by the OTC module. In addition, the FailurE causE Localization for optIcal NetworkinG (FEELING) algorithm is proposed to localize failures affecting a lightpath using OSAs. Extensive simulation results are presented, showing the effectiveness of the TISSUE algorithm in properly exploiting OTC information to assess BER performance of quadrature-phase-shift-keying-modulated signals, and the high accuracy of the FEELING algorithm to correctly detect soft failures as laser drift, filter shift, and tight filtering.Peer ReviewedPostprint (published version

Dynamic core VNT adaptability based on predictive metro-flow traffic models

© 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.MPLS-over-optical virtual network topologies (VNTs) can be adapted to near-future traffic matrices based on predictive models that are estimated by applying data analytics on monitored origin-destination (OD) traffic. However, the deployment of independent SDN controllers for core and metro segments can bring large inefficiencies to this core network reconfiguration based on traffic prediction when traffic flows from metro areas are rerouted to different ingress nodes in the core. In such cases, OD traffic patterns in the core might severely change, thus affecting the quality of the predictive OD models. New traffic model re-estimation usually takes a long time, during which no predictive capabilities are available for the network operator. To alleviate this problem, we propose to extend data analytics to metro networks to obtain predictive models for the metro flows; by knowing how these flows are aggregated into OD pairs in the core, we can also aggregate their predictive models, thus accurately predicting OD traffic and therefore enabling core VNT reconfiguration. To obtain quality metro-flow models, we propose an estimation algorithmthat processes monitored data and returns a predictive model. In addition, a flow controller is proposed for the control architecture to allow metro and core controllers to exchange metro-flow model information. The proposed model aggregation is evaluated through exhaustive simulation, and eventually experimentally assessed together with the flow controller in a testbed connecting premises in CNIT (Pisa, Italy) and UPC (Barcelona, Spain).Peer ReviewedPostprint (author's final draft

Predictive autonomic transmission for low-cost low-margin metro optical networks

Low-cost low-margin implementation plays an essential role in upgrading optical metro networks required for future 5G ecosystem. In this regard, low-resolution analog-to-digital converters can be used in coherent optical transponders to reduce cost and power consumption. However, the resulting transmission systems become more sensitive to physical layer fluctuations like the events caused by fiber stressing. Such fluctuations might have a strong impact on the quality of transmission (QoT) of the signals. To guarantee robust operation, soft decision forward error correction (FEC) techniques are required to guarantee zero post-FEC bit error rate (BER) transmission, which could increase the power consumption of the receiver and thus operational expenses. In this paper, we aim at minimizing power consumption while keeping zero post-FEC errors by means of a predictive autonomic transmission agent (ATA) based on machine learning. We present a sophisticated ATA model that, taking advantage of real-time monitoring of state of polarization traces and the corresponding pre-FEC BER, predicts the right FEC configuration for short-term operation, thus requiring minimum power consumption. In addition, we propose a complementary long-term prediction of excessive pre-FEC BER to enable remote reconfiguration at the transmitter side through the network controller. A set of experimental measurements is used to train and validate the proposed ATA system. Exhaustive numerical analysis allows concluding that ATA based on artificial neural network predictors achieves the maximum QoT robustness with 80% power consumption reductions compared to static FEC configuration.The research leading to these results has received funding from the European Commission for the H2020-ICT-2016-2 METRO-HAUL project (G.A. 761727), from the AEI/FEDER TWINS project (TEC2017-90097-R), and from the Catalan Institution for Research and Advanced Studies (ICREA).Peer ReviewedPostprint (author's final draft

Autonomic disaggregated multilayer networking

Focused on reducing capital expenditures by opening the data plane to multiple vendors without impacting performance, node disaggregation is attracting the interest of network operators. Although the software-defined networking (SDN) paradigm is key for the control of such networks, the increased complexity of multilayer networks strictly requires monitoring/telemetry and data analytics capabilities to assist in creating and operating self-managed (autonomic) networks. Such autonomicity greatly reduces operational expenditures, while improving network performance. In this context, a monitoring and data analytics (MDA) architecture consisting of centralized data storage with data analytics capabilities, together with a generic node agent for monitoring/telemetry supporting disaggregation, is presented. A YANG data model that allows one to clearly separate responsibilities for monitoring configuration from node configuration is also proposed. The MDA architecture and YANG data models are experimentally demonstrated through three different use cases: i) virtual link creation supported by an optical connection, where monitoring is automatically activated; ii) multilayer self-configuration after bit error rate (BER) degradation detection, where a modulation format adaptation is recommended for the SDN controller to minimize errors (this entails reducing the capacity of both the virtual link and supported multiprotocol label switching-transport profile (MPLS-TP) paths); and iii) optical layer selfhealing, including failure localization at the optical layer to find the cause of BER degradation. A combination of active and passive monitoring procedures allows one to localize the cause of the failure, leading to lightpath rerouting recommendations toward the SDN controller avoiding the failing element(s).Peer ReviewedPostprint (author's final draft

Near real-time estimation of end-to-end performance in converged fixed-mobile networks

© Elsevier. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/The independent operation of mobile and fixed network segments is one of the main barriers that prevents improving network performance while reducing capital expenditures coming from overprovisioning. In particular, a coordinated dynamic network operation of both network segments is essential to guarantee end-to-end Key Performance Indicators (KPI), on which new network services rely on. To achieve such dynamic operation, accurate estimation of end-to-end KPIs is needed to trigger network reconfiguration before performance degrades. In this paper, we present a methodology to achieve an accurate, scalable, and predictive estimation of end-to-end KPIs with sub-second granularity near real-time in converged fixed-mobile networks. Specifically, we extend our CURSA-SQ methodology for mobile network traffic analysis, to enable converged fixed-mobile network operation. CURSA-SQ combines simulation and machine learning fueled with real network monitoring data. Numerical results validate the accuracy, robustness, and usability of the proposed CURSA-SQ methodology for converged fixed-mobile network scenarios.Peer ReviewedPostprint (author's final draft