Optical Network Virtualisation using Multi-technology Monitoring and SDN-enabled Optical Transceiver

We introduce the real-time multi-technology transport layer monitoring to facilitate the coordinated virtualisation of optical and Ethernet networks supported by optical virtualise-able transceivers (V-BVT). A monitoring and network resource configuration scheme is proposed to include the hardware monitoring in both Ethernet and Optical layers. The scheme depicts the data and control interactions among multiple network layers under the software defined network (SDN) background, as well as the application that analyses the monitored data obtained from the database. We also present a re-configuration algorithm to adaptively modify the composition of virtual optical networks based on two criteria. The proposed monitoring scheme is experimentally demonstrated with OpenFlow (OF) extensions for a holistic (re-)configuration across both layers in Ethernet switches and V-BVTs

Optimisation for Optical Data Centre Switching and Networking with Artificial Intelligence

Cloud and cluster computing platforms have become standard across almost every domain of business, and their scale quickly approaches $\mathbf{O}(10^6)$ servers in a single warehouse. However, the tier-based opto-electronically packet switched network infrastructure that is standard across these systems gives way to several scalability bottlenecks including resource fragmentation and high energy requirements. Experimental results show that optical circuit switched networks pose a promising alternative that could avoid these. However, optimality challenges are encountered at realistic commercial scales. Where exhaustive optimisation techniques are not applicable for problems at the scale of Cloud-scale computer networks, and expert-designed heuristics are performance-limited and typically biased in their design, artificial intelligence can discover more scalable and better performing optimisation strategies. This thesis demonstrates these benefits through experimental and theoretical work spanning all of component, system and commercial optimisation problems which stand in the way of practical Cloud-scale computer network systems. Firstly, optical components are optimised to gate in $\approx 500 ps$ and are demonstrated in a proof-of-concept switching architecture for optical data centres with better wavelength and component scalability than previous demonstrations. Secondly, network-aware resource allocation schemes for optically composable data centres are learnt end-to-end with deep reinforcement learning and graph neural networks, where $3\times$ less networking resources are required to achieve the same resource efficiency compared to conventional methods. Finally, a deep reinforcement learning based method for optimising PID-control parameters is presented which generates tailored parameters for unseen devices in $\mathbf{O}(10^{-3}) s$. This method is demonstrated on a market leading optical switching product based on piezoelectric actuation, where switching speed is improved $>20\%$ with no compromise to optical loss and the manufacturing yield of actuators is improved. This method was licensed to and integrated within the manufacturing pipeline of this company. As such, crucial public and private infrastructure utilising these products will benefit from this work

Traffic and task allocation in networks and the cloud

Communication services such as telephony, broadband and TV are increasingly migrating into Internet Protocol(IP) based networks because of the consolidation of telephone and data networks. Meanwhile, the increasingly wide application of Cloud Computing enables the accommodation of tens of thousands of applications from the general public or enterprise users which make use of Cloud services on-demand through IP networks such as the Internet. Real-Time services over IP (RTIP) have also been increasingly significant due to the convergence of network services, and the real-time needs of the Internet of Things (IoT) will strengthen this trend. Such Real-Time applications have strict Quality of Service (QoS) constraints, posing a major challenge for IP networks. The Cognitive Packet Network (CPN) has been designed as a QoS-driven protocol that addresses user-oriented QoS demands by adaptively routing packets based on online sensing and measurement. Thus in this thesis we first describe our design for a novel ``Real-Time (RT) traffic over CPN'' protocol which uses QoS goals that match the needs of voice packet delivery in the presence of other background traffic under varied traffic conditions; we present its experimental evaluation via measurements of key QoS metrics such as packet delay, delay variation (jitter) and packet loss ratio. Pursuing our investigation of packet routing in the Internet, we then propose a novel Big Data and Machine Learning approach for real-time Internet scale Route Optimisation based on Quality-of-Service using an overlay network, and evaluate is performance. Based on the collection of data sampled each $2$ minutes over a large number of source-destinations pairs, we observe that intercontinental Internet Protocol (IP) paths are far from optimal with respect to metrics such as end-to-end round-trip delay. On the other hand, our machine learning based overlay network routing scheme exploits large scale data collected from communicating node pairs to select overlay paths, while it uses IP between neighbouring overlay nodes. We report measurements over a week long experiment with several million data points shows substantially better end-to-end QoS than is observed with pure IP routing. Pursuing the machine learning approach, we then address the challenging problem of dispatching incoming tasks to servers in Cloud systems so as to offer the best QoS and reliable job execution; an experimental system (the Task Allocation Platform) that we have developed is presented and used to compare several task allocation schemes, including a model driven algorithm, a reinforcement learning based scheme, and a ``sensible’’ allocation algorithm that assigns tasks to sub-systems that are observed to provide lower response time. These schemes are compared via measurements both among themselves and against a standard round-robin scheduler, with two architectures (with homogenous and heterogenous hosts having different processing capacities) and the conditions under which the different schemes offer better QoS are discussed. Since Cloud systems include both locally based servers at user premises and remote servers and multiple Clouds that can be reached over the Internet, we also describe a smart distributed system that combines local and remote Cloud facilities, allocating tasks dynamically to the service that offers the best overall QoS, and it includes a routing overlay which minimizes network delay for data transfer between Clouds. Internet-scale experiments that we report exhibit the effectiveness of our approach in adaptively distributing workload across multiple Clouds.Open Acces

Reinforcement learning for efficient network penetration testing

Penetration testing (also known as pentesting or PT) is a common practice for actively assessing the defenses of a computer network by planning and executing all possible attacks to discover and exploit existing vulnerabilities. Current penetration testing methods are increasingly becoming non-standard, composite and resource-consuming despite the use of evolving tools. In this paper, we propose and evaluate an AI-based pentesting system which makes use of machine learning techniques, namely reinforcement learning (RL) to learn and reproduce average and complex pentesting activities. The proposed system is named Intelligent Automated Penetration Testing System (IAPTS) consisting of a module that integrates with industrial PT frameworks to enable them to capture information, learn from experience, and reproduce tests in future similar testing cases. IAPTS aims to save human resources while producing much-enhanced results in terms of time consumption, reliability and frequency of testing. IAPTS takes the approach of modeling PT environments and tasks as a partially observed Markov decision process (POMDP) problem which is solved by POMDP-solver. Although the scope of this paper is limited to network infrastructures PT planning and not the entire practice, the obtained results support the hypothesis that RL can enhance PT beyond the capabilities of any human PT expert in terms of time consumed, covered attacking vectors, accuracy and reliability of the outputs. In addition, this work tackles the complex problem of expertise capturing and re-use by allowing the IAPTS learning module to store and re-use PT policies in the same way that a human PT expert would learn but in a more efficient way

A survey of self organisation in future cellular networks

This article surveys the literature over the period of the last decade on the emerging field of self organisation as applied to wireless cellular communication networks. Self organisation has been extensively studied and applied in adhoc networks, wireless sensor networks and autonomic computer networks; however in the context of wireless cellular networks, this is the first attempt to put in perspective the various efforts in form of a tutorial/survey. We provide a comprehensive survey of the existing literature, projects and standards in self organising cellular networks. Additionally, we also aim to present a clear understanding of this active research area, identifying a clear taxonomy and guidelines for design of self organising mechanisms. We compare strength and weakness of existing solutions and highlight the key research areas for further development. This paper serves as a guide and a starting point for anyone willing to delve into research on self organisation in wireless cellular communication networks

Protecting valuable resources using optimal control theory and feedback strategies for plant disease management

Mathematical models of tree diseases often have little to say about how to manage established epidemics. Models often show that it is too late for successful disease eradication, but few study what management could still be beneficial. This study focusses on finding effective control strategies for managing sudden oak death, a tree disease caused by Phytophthora ramorum. Sudden oak death is a devastating disease spreading through forests in California and southwestern Oregon. The disease is well established and eradication is no longer possible. The ongoing spread of sudden oak death is threatening high value tree resources, including national parks, and culturally and ecologically important species like tanoak. In this thesis we show how the allocation of limited resources for controlling sudden oak death can be optimised to protect these valuable trees. We use simple, approximate models of sudden oak death dynamics, to which we apply the mathematical framework of optimal control theory. Applying the optimised controls from the approximate model to a complex, spatial simulation model, we demonstrate that the framework finds effective strategies for protecting tanoak, whilst also conserving biodiversity. When applied to the problem of protecting Redwood National Park, which is under threat from a nearby outbreak of sudden oak death, the framework finds spatial strategies that balance protective barriers with control at the epidemic wavefront. Because of the number of variables in the system, computational and numerical limitations restrict the control optimisation to relatively simple approximate models. We show how a lack of accuracy in the approximate model can be accounted for by using model predictive control, from control systems engineering: an approach coupling feedback with optimal control theory. Continued surveillance of the complex system, and re-optimisation of the control strategy, ensures that the result remains close to optimal, and leads to highly effective disease management. In this thesis we show how the machinery of optimal control theory can inform plant disease management, protecting valuable resources from sudden oak death. Incorporating feedback into the application of the resulting strategies ensures control remains effective over long timescales, and is robust to uncertainties and stochasticity in the system. Local management of sudden oak death is still possible, and our results show how this can be achieved

Optimised Design and Analysis of All-Optical Networks

This PhD thesis presents a suite of methods for optimising design and for analysing blocking probabilities of all-optical networks. It thus contributes methodical knowledge to the field of computer assisted planning of optical networks. A two-stage greenfield optical network design optimiser is developed, based on shortest-path algorithms and a comparatively new metaheuristic called simulated allocation. It is able to handle design of all-optical mesh networks with optical cross-connects, considers duct as well as fibre and node costs, and can also design protected networks. The method is assessed through various experiments and is shown to produce good results and to be able to scale up to networks of realistic sizes. A novel method, subpath wavelength grouping, for routing connections in a multigranular all-optical network where several wavelengths can be grouped and switched at band and fibre level is presented. The method uses an unorthodox routing strategy focusing on common subpaths rather than individual connections, and strives to minimise switch port count as well as fibre usage. It is shown to produce cheaper network designs than previous methods when fibre costs are comparatively high. A new optical network concept, the synchronous optical hierarchy, is proposed, in which wavelengths are subdivided into timeslots to match the traffic granularity. Various theoretical properties of this concept are investigated and compared in simulation studies. An integer linear programming model for optical ring network design is presented. Manually designed real world ring networks are studied and it is found that the model can lead to cheaper network design. Moreover, ring and mesh network architectures are compared using real world costs, and it is found that optical cros..

Survivable mesh-network design & optimization to support multiple QoP service classes

Every second, vast amounts of data are transferred over communication systems around the world, and as a result, the demands on optical infrastructures are extending beyond the traditional, ring-based architecture. The range of content and services available from the Internet is increasing, and network operations are constantly under pressure to expand their optical networks in order to keep pace with the ever increasing demand for higher speed and more reliable links