Experimental demonstration of machine-learning-aided QoT estimation in multi-domain elastic optical networks with alien wavelengths

In multi-domain elastic optical networks with alien wavelengths, each domain needs to consider intradomain and interdomain alien traffic to estimate and guarantee the required quality of transmission (QoT) for each lightpath and perform provisioning operations. This paper experimentally demonstrates an alien wavelength performance monitoring technique and machine-learning-aided QoT estimation for lightpath provisioning of intradomain/interdomain traffic. Testbed experiments demonstrate modulation format recognition, QoT monitoring, and cognitive routing for a 160 Gbaud alien multi-wavelength lightpath. By using experimental training datasets from the testbed and an artificial neural network, we demonstrated an accurate optical-signal-to-noise ratio prediction with an accuracy of ∼95% when using 1200 data points

Demonstration of distributed collaborative learning with end-to-end QoT estimation in multi-domain elastic optical networks

This paper proposes a distributed collaborative learning approach for cognitive and autonomous multi-domain elastic optical networking (EON). The proposed approach exploits a knowledge-defined networking framework which leverages a broker plane to coordinate the operations of multiple EON domains and applies machine learning (ML) to support autonomous and cognitive inter-domain service provisioning. By employing multiple distributed ML blocks learning domain-level features and working with broker plane aggregation ML blocks (through the chain rule-based training), the proposed approach enables to develop cognitive networking applications that can fully exploit the multi-domain EON states while obviating the need for the raw and confidential intra-domain data. In particular, we investigate end-to-end quality-of-transmission estimation application using the distributed learning approach and propose three estimator designs incorporating the concepts of multi-task learning (MTL) and transfer learning (TL). Evaluations with experimental data demonstrate that the proposed designs can achieve estimation accuracies very close to (with differences less than 0.5%) or even higher than (with MTL/TL) those of the baseline models assuming full domain visibility

Hierarchical Learning for Cognitive End-to-End Service Provisioning in Multi-Domain Autonomous Optical Networks

This paper demonstrates, for the first time to our knowledge, hierarchical learning framework for inter-domain service provisioning in software-defined elastic optical networking (EON). By using a broker-based hierarchical architecture, the broker collaborates with the domain managers to realize efficient global service provisioning without violating the privacy constrains of each domain. In the proposed hierarchical learning scheme, machine learning-based cognition agents exist in the domain managers as well as in the broker. The proposed system is experimentally demonstrated on a two-domain seven-node EON testbed for with real-time optical performance monitors (OPMs). By using over 42000 datasets collected from OPM units, the cognition agents can be trained to accurately infer the Q-factor of an unestablished or established lightpath, enabling an impairment-aware end-to-end service provisioning with an prediction Q-factor deviation less than 0.6 dB

Leveraging Semantic Web Technologies for Managing Resources in a Multi-Domain Infrastructure-as-a-Service Environment

This paper reports on experience with using semantically-enabled network resource models to construct an operational multi-domain networked infrastructure-as-a-service (NIaaS) testbed called ExoGENI, recently funded through NSF's GENI project. A defining property of NIaaS is the deep integration of network provisioning functions alongside the more common storage and computation provisioning functions. Resource provider topologies and user requests can be described using network resource models with common base classes for fundamental cyber-resources (links, nodes, interfaces) specialized via virtualization and adaptations between networking layers to specific technologies. This problem space gives rise to a number of application areas where semantic web technologies become highly useful - common information models and resource class hierarchies simplify resource descriptions from multiple providers, pathfinding and topology embedding algorithms rely on query abstractions as building blocks. The paper describes how the semantic resource description models enable ExoGENI to autonomously instantiate on-demand virtual topologies of virtual machines provisioned from cloud providers and are linked by on-demand virtual connections acquired from multiple autonomous network providers to serve a variety of applications ranging from distributed system experiments to high-performance computing

Self-Taught Anomaly Detection With Hybrid Unsupervised/Supervised Machine Learning in Optical Networks

This paper proposes a self-taught anomaly detection framework for optical networks. The proposed framework makes use of a hybrid unsupervised and supervised machine learning scheme. First, it employs an unsupervised data clustering module (DCM) to analyze the patterns of monitoring data. The DCM enables a self-learning capability that eliminates the requirement of prior knowledge of abnormal network behaviors and therefore can potentially detect unforeseen anomalies. Second, we introduce a self-taught mechanism that transfers the patterns learned by the DCM to a supervised data regression and classification module (DRCM). The DRCM, whose complexity is mainly related to the scale of the applied supervised learning model, can potentially facilitate more scalable and time-efficient online anomaly detection by avoiding excessively traversing the original dataset. We designed the DCM and DRCM based on the density-based clustering algorithm and the deep neural network structure, respectively. Evaluations with experimental data from two use cases (i.e., single-point detection and end-to-end detection) demonstrate that up to 99% anomaly detection accuracy can be achieved with a false positive rate below 1%

Performance studies of evolutionary transfer learning for end-to-end QoT estimation in multi-domain optical networks [Invited]

This paper proposes an evolutionary transfer learning approach (Evol-TL) for scalable quality-of-transmission (QoT) estimation in multi-domain elastic optical networks (MD-EONs). Evol-TL exploits a broker-based MD-EON architecture that enables cooperative learning between the broker plane (end-to-end) and domain-level (local) machine learning functions while securing the autonomy of each domain. We designed a genetic algorithm to optimize the neural network architectures and the sets of weights to be transferred between the source and destination tasks. We evaluated the performance of Evol-TL with three case studies considering the QoT estimation task for lightpaths with (i) different path lengths (in terms of the numbers of fiber links traversed), (ii) different modulation formats, and (iii) different device conditions (emulated by introducing different levels of wavelength-specific attenuation to the amplifiers). The results show that the proposed approach can reduce the average amount of required training data by up to 13× while achieving an estimation accuracy above 95%

Network automation: challenges, enablers, and benefits

Communication infrastructures are evolving towards an ad-hoc service provisioning scenario where programmability and flexibility are fundamental concepts. Network automation is expected to play a vital role in streamlining all aspects of the service provisioning process (i.e., deployment, maintenance, and tear down). However, to fully realize this autonomous operation vision, closed-loop automation procedures need to be developed.This tutorial will present the main motivations and challenges behind designing and operating closed-loop autonomous decision-making processes, including a brief overview of current standardization initiatives. The tutorial will then address several use cases showcasing how network automation can alleviate the complexity of the service provisioning processes and the benefits brought in by the introduction of network automation