An Overview on Application of Machine Learning Techniques in Optical Networks

Today's telecommunication networks have become sources of enormous amounts of widely heterogeneous data. This information can be retrieved from network traffic traces, network alarms, signal quality indicators, users' behavioral data, etc. Advanced mathematical tools are required to extract meaningful information from these data and take decisions pertaining to the proper functioning of the networks from the network-generated data. Among these mathematical tools, Machine Learning (ML) is regarded as one of the most promising methodological approaches to perform network-data analysis and enable automated network self-configuration and fault management. The adoption of ML techniques in the field of optical communication networks is motivated by the unprecedented growth of network complexity faced by optical networks in the last few years. Such complexity increase is due to the introduction of a huge number of adjustable and interdependent system parameters (e.g., routing configurations, modulation format, symbol rate, coding schemes, etc.) that are enabled by the usage of coherent transmission/reception technologies, advanced digital signal processing and compensation of nonlinear effects in optical fiber propagation. In this paper we provide an overview of the application of ML to optical communications and networking. We classify and survey relevant literature dealing with the topic, and we also provide an introductory tutorial on ML for researchers and practitioners interested in this field. Although a good number of research papers have recently appeared, the application of ML to optical networks is still in its infancy: to stimulate further work in this area, we conclude the paper proposing new possible research directions

On Cooperative Fault Management in Multi-Domain Optical Networks Using Hybrid Learning

This paper presents a hybrid learning approach for cooperative fault management in multi-domain optical networks (MD-ONs). The proposed approach relies on a broker-based MD-ON architecture for coordination of inter-domain service provisioning. We first propose a self-supervised learning design for soft failure detection. The self-supervised learning design makes use of a clustering algorithm for extracting normal and abnormal patterns from optical performance monitoring data and a supervised learning-based classifier trained with the learned patterns for online detection. To facilitate high soft failure detection accuracy in the absence of sufficient abnormal data for training, the proposed design estimates model uncertainties during predictions and identifies instances associated with high uncertainties as also soft failures. Then, we extend the self-supervised learning design and present a federated learning framework for the broker plane and DMs to learn cooperatively while complying with the privacy constraints of each domain. Finally, a data-driven soft failure localization scheme that operates by analyzing the patterns of data is proposed as a complement to the existing approaches. Performance evaluations indicate that the self-supervised learning design can achieve soft failure detection accuracy of up to ∼ 97% with 0.01%-0.04% false alarm rate, while federated learning enables DMs to realize >90% soft failure detection rates in the cases of highly unbalanced data distribution (two of the three domains possess zero abnormal data for training)

A Family of Joint Sparse PCA Algorithms for Anomaly Localization in Network Data Streams

Determining anomalies in data streams that are collected and transformed from various types of networks has recently attracted significant research interest. Principal Component Analysis (PCA) is arguably the most widely applied unsupervised anomaly detection technique for networked data streams due to its simplicity and efficiency. However, none of existing PCA based approaches addresses the problem of identifying the sources that contribute most to the observed anomaly, or anomaly localization. In this paper, we first proposed a novel joint sparse PCA method to perform anomaly detection and localization for network data streams. Our key observation is that we can detect anomalies and localize anomalous sources by identifying a low dimensional abnormal subspace that captures the abnormal behavior of data. To better capture the sources of anomalies, we incorporated the structure of the network stream data in our anomaly localization framework. Also, an extended version of PCA, multidimensional KLE, was introduced to stabilize the localization performance. We performed comprehensive experimental studies on four real-world data sets from different application domains and compared our proposed techniques with several state-of-the-arts. Our experimental studies demonstrate the utility of the proposed methods

Performance studies of evolutionary transfer learning for end-to-end QoT estimation in multi-domain optical networks [Invited]

This paper proposes an evolutionary transfer learning approach (Evol-TL) for scalable quality-of-transmission (QoT) estimation in multi-domain elastic optical networks (MD-EONs). Evol-TL exploits a broker-based MD-EON architecture that enables cooperative learning between the broker plane (end-to-end) and domain-level (local) machine learning functions while securing the autonomy of each domain. We designed a genetic algorithm to optimize the neural network architectures and the sets of weights to be transferred between the source and destination tasks. We evaluated the performance of Evol-TL with three case studies considering the QoT estimation task for lightpaths with (i) different path lengths (in terms of the numbers of fiber links traversed), (ii) different modulation formats, and (iii) different device conditions (emulated by introducing different levels of wavelength-specific attenuation to the amplifiers). The results show that the proposed approach can reduce the average amount of required training data by up to 13× while achieving an estimation accuracy above 95%

Demonstration of Machine-Learning-Assisted Security Monitoring in Optical Networks

We report on the first demonstration of machine-learning-assisted detection, identification and localisation of optical-layer attacks integrated into network management system and verified on real-life experimental attack traces from a network operator testbed