An Overview on Application of Machine Learning Techniques in Optical Networks

Today's telecommunication networks have become sources of enormous amounts of widely heterogeneous data. This information can be retrieved from network traffic traces, network alarms, signal quality indicators, users' behavioral data, etc. Advanced mathematical tools are required to extract meaningful information from these data and take decisions pertaining to the proper functioning of the networks from the network-generated data. Among these mathematical tools, Machine Learning (ML) is regarded as one of the most promising methodological approaches to perform network-data analysis and enable automated network self-configuration and fault management. The adoption of ML techniques in the field of optical communication networks is motivated by the unprecedented growth of network complexity faced by optical networks in the last few years. Such complexity increase is due to the introduction of a huge number of adjustable and interdependent system parameters (e.g., routing configurations, modulation format, symbol rate, coding schemes, etc.) that are enabled by the usage of coherent transmission/reception technologies, advanced digital signal processing and compensation of nonlinear effects in optical fiber propagation. In this paper we provide an overview of the application of ML to optical communications and networking. We classify and survey relevant literature dealing with the topic, and we also provide an introductory tutorial on ML for researchers and practitioners interested in this field. Although a good number of research papers have recently appeared, the application of ML to optical networks is still in its infancy: to stimulate further work in this area, we conclude the paper proposing new possible research directions

Towards cognitive in-operation network planning

Next-generation internet services such as live TV and video on demand require high bandwidth and ultra-low latency. The ever-increasing volume, dynamicity and stringent requirements of these services’ demands are generating new challenges to nowadays telecom networks. To decrease expenses, service-layer content providers are delivering their content near the end users, thus allowing a low latency and tailored content delivery. As a consequence of this, unseen metro and even core traffic dynamicity is arising with changes in the volume and direction of the traffic along the day. A tremendous effort to efficiently manage networks is currently ongoing towards the realisation of 5G networks. This translates in looking for network architectures supporting dynamic resource allocation, fulfilling strict service requirements and minimising the total cost of ownership (TCO). In this regard, in-operation network planning was recently proven to successfully support various network reconfiguration use cases in prospective scenarios. Nevertheless, additional research to extend in-operation planning capabilities from typical reactive optimization schemes to proactive and predictive schemes based on the analysis of network monitoring data is required. A hot topic raising increasing attention is cognitive networking, where an elevated knowledge about the network could be obtained as a result of introducing data analytics in the telecom operator’s infrastructure. By using predictive knowledge about the network traffic, in-operation network planning mechanisms could be enhanced to efficiently adapt the network by means of future traffic prediction, thus achieving cognitive in-operation network planning. In this thesis, we focus on studying mechanisms to enable cognitive in-operation network planning in core networks. In particular, we focus on dynamically reconfiguring virtual network topologies (VNT) at the MPLS layer, covering a number of detailed objectives. First, we start studying mechanisms to allow network traffic flow modelling, from monitoring and data transformation to the estimation of predictive traffic model based on this data. By means of these traffic models, then we tackle a cognitive approach to periodically adapt the core VNT to current and future traffic, using predicted traffic matrices based on origin-destination (OD) predictive models. This optimization approach, named VENTURE, is efficiently solved using dedicated heuristic algorithms and its feasibility is demonstrated in an experimental in-operation network planning environment. Finally, we extend VENTURE to consider core flows dynamicity as a result of metro flows re-routing, which represents a meaningful dynamic traffic scenario. This extension, which entails enhancements to coordinate metro and core network controllers with the aim of allowing fast adaption of core OD traffic models, is evaluated and validated in terms of traffic models accuracy and experimental feasibility.Els serveis d’internet de nova generació tals com la televisió en viu o el vídeo sota demanda requereixen d’un gran ample de banda i d’ultra-baixa latència. L’increment continu del volum, dinamicitat i requeriments d’aquests serveis està generant nous reptes pels teleoperadors de xarxa. Per reduir costs, els proveïdors de contingut estan disposant aquests més a prop dels usuaris finals, aconseguint així una entrega de contingut feta a mida. Conseqüentment, estem presenciant una dinamicitat mai vista en el tràfic de xarxes de metro amb canvis en la direcció i el volum del tràfic al llarg del dia. Actualment, s’està duent a terme un gran esforç cap a la realització de xarxes 5G. Aquest esforç es tradueix en cercar noves arquitectures de xarxa que suportin l’assignació dinàmica de recursos, complint requeriments de servei estrictes i minimitzant el cost total de la propietat. En aquest sentit, recentment s’ha demostrat com l’aplicació de “in-operation network planning” permet exitosament suportar diversos casos d’ús de reconfiguració de xarxa en escenaris prospectius. No obstant, és necessari dur a terme més recerca per tal d’estendre “in-operation network planning” des d’un esquema reactiu d’optimització cap a un nou esquema proactiu basat en l’analítica de dades provinents del monitoritzat de la xarxa. El concepte de xarxes cognitives es també troba al centre d’atenció, on un elevat coneixement de la xarxa s’obtindria com a resultat d’introduir analítica de dades en la infraestructura del teleoperador. Mitjançant un coneixement predictiu sobre el tràfic de xarxa, els mecanismes de in-operation network planning es podrien millorar per adaptar la xarxa eficientment basant-se en predicció de tràfic, assolint així el que anomenem com a “cognitive in-operation network Planning”. En aquesta tesi ens centrem en l’estudi de mecanismes que permetin establir “el cognitive in-operation network Planning” en xarxes de core. En particular, ens centrem en reconfigurar dinàmicament topologies de xarxa virtual (VNT) a la capa MPLS, cobrint una sèrie d’objectius detallats. Primer comencem estudiant mecanismes pel modelat de fluxos de tràfic de xarxa, des del seu monitoritzat i transformació fins a l’estimació de models predictius de tràfic. Posteriorment, i mitjançant aquests models predictius, tractem un esquema cognitiu per adaptar periòdicament la VNT utilitzant matrius de tràfic basades en predicció de parells origen-destí (OD). Aquesta optimització, anomenada VENTURE, és resolta eficientment fent servir heurístiques dedicades i és posteriorment avaluada sota escenaris de tràfic de xarxa dinàmics. A continuació, estenem VENTURE considerant la dinamicitat dels fluxos de tràfic de xarxes de metro, el qual representa un escenari rellevant de dinamicitat de tràfic. Aquesta extensió involucra millores per coordinar els operadors de metro i core amb l’objectiu d’aconseguir una ràpida adaptació de models de tràfic OD. Finalment, proposem dues arquitectures de xarxa necessàries per aplicar els mecanismes anteriors en entorns experimentals, emprant protocols estat-de-l’art com són OpenFlow i IPFIX. La metodologia emprada per avaluar el treball anterior consisteix en una primera avaluació numèrica fent servir un simulador de xarxes íntegrament dissenyat i desenvolupat per a aquesta tesi. Després d’aquesta validació basada en simulació, la factibilitat experimental de les arquitectures de xarxa proposades és avaluada en un entorn de proves distribuït.Postprint (published version

Towards cognitive in-operation network planning

Next-generation internet services such as live TV and video on demand require high bandwidth and ultra-low latency. The ever-increasing volume, dynamicity and stringent requirements of these services’ demands are generating new challenges to nowadays telecom networks. To decrease expenses, service-layer content providers are delivering their content near the end users, thus allowing a low latency and tailored content delivery. As a consequence of this, unseen metro and even core traffic dynamicity is arising with changes in the volume and direction of the traffic along the day. A tremendous effort to efficiently manage networks is currently ongoing towards the realisation of 5G networks. This translates in looking for network architectures supporting dynamic resource allocation, fulfilling strict service requirements and minimising the total cost of ownership (TCO). In this regard, in-operation network planning was recently proven to successfully support various network reconfiguration use cases in prospective scenarios. Nevertheless, additional research to extend in-operation planning capabilities from typical reactive optimization schemes to proactive and predictive schemes based on the analysis of network monitoring data is required. A hot topic raising increasing attention is cognitive networking, where an elevated knowledge about the network could be obtained as a result of introducing data analytics in the telecom operator’s infrastructure. By using predictive knowledge about the network traffic, in-operation network planning mechanisms could be enhanced to efficiently adapt the network by means of future traffic prediction, thus achieving cognitive in-operation network planning. In this thesis, we focus on studying mechanisms to enable cognitive in-operation network planning in core networks. In particular, we focus on dynamically reconfiguring virtual network topologies (VNT) at the MPLS layer, covering a number of detailed objectives. First, we start studying mechanisms to allow network traffic flow modelling, from monitoring and data transformation to the estimation of predictive traffic model based on this data. By means of these traffic models, then we tackle a cognitive approach to periodically adapt the core VNT to current and future traffic, using predicted traffic matrices based on origin-destination (OD) predictive models. This optimization approach, named VENTURE, is efficiently solved using dedicated heuristic algorithms and its feasibility is demonstrated in an experimental in-operation network planning environment. Finally, we extend VENTURE to consider core flows dynamicity as a result of metro flows re-routing, which represents a meaningful dynamic traffic scenario. This extension, which entails enhancements to coordinate metro and core network controllers with the aim of allowing fast adaption of core OD traffic models, is evaluated and validated in terms of traffic models accuracy and experimental feasibility.Els serveis d’internet de nova generació tals com la televisió en viu o el vídeo sota demanda requereixen d’un gran ample de banda i d’ultra-baixa latència. L’increment continu del volum, dinamicitat i requeriments d’aquests serveis està generant nous reptes pels teleoperadors de xarxa. Per reduir costs, els proveïdors de contingut estan disposant aquests més a prop dels usuaris finals, aconseguint així una entrega de contingut feta a mida. Conseqüentment, estem presenciant una dinamicitat mai vista en el tràfic de xarxes de metro amb canvis en la direcció i el volum del tràfic al llarg del dia. Actualment, s’està duent a terme un gran esforç cap a la realització de xarxes 5G. Aquest esforç es tradueix en cercar noves arquitectures de xarxa que suportin l’assignació dinàmica de recursos, complint requeriments de servei estrictes i minimitzant el cost total de la propietat. En aquest sentit, recentment s’ha demostrat com l’aplicació de “in-operation network planning” permet exitosament suportar diversos casos d’ús de reconfiguració de xarxa en escenaris prospectius. No obstant, és necessari dur a terme més recerca per tal d’estendre “in-operation network planning” des d’un esquema reactiu d’optimització cap a un nou esquema proactiu basat en l’analítica de dades provinents del monitoritzat de la xarxa. El concepte de xarxes cognitives es també troba al centre d’atenció, on un elevat coneixement de la xarxa s’obtindria com a resultat d’introduir analítica de dades en la infraestructura del teleoperador. Mitjançant un coneixement predictiu sobre el tràfic de xarxa, els mecanismes de in-operation network planning es podrien millorar per adaptar la xarxa eficientment basant-se en predicció de tràfic, assolint així el que anomenem com a “cognitive in-operation network Planning”. En aquesta tesi ens centrem en l’estudi de mecanismes que permetin establir “el cognitive in-operation network Planning” en xarxes de core. En particular, ens centrem en reconfigurar dinàmicament topologies de xarxa virtual (VNT) a la capa MPLS, cobrint una sèrie d’objectius detallats. Primer comencem estudiant mecanismes pel modelat de fluxos de tràfic de xarxa, des del seu monitoritzat i transformació fins a l’estimació de models predictius de tràfic. Posteriorment, i mitjançant aquests models predictius, tractem un esquema cognitiu per adaptar periòdicament la VNT utilitzant matrius de tràfic basades en predicció de parells origen-destí (OD). Aquesta optimització, anomenada VENTURE, és resolta eficientment fent servir heurístiques dedicades i és posteriorment avaluada sota escenaris de tràfic de xarxa dinàmics. A continuació, estenem VENTURE considerant la dinamicitat dels fluxos de tràfic de xarxes de metro, el qual representa un escenari rellevant de dinamicitat de tràfic. Aquesta extensió involucra millores per coordinar els operadors de metro i core amb l’objectiu d’aconseguir una ràpida adaptació de models de tràfic OD. Finalment, proposem dues arquitectures de xarxa necessàries per aplicar els mecanismes anteriors en entorns experimentals, emprant protocols estat-de-l’art com són OpenFlow i IPFIX. La metodologia emprada per avaluar el treball anterior consisteix en una primera avaluació numèrica fent servir un simulador de xarxes íntegrament dissenyat i desenvolupat per a aquesta tesi. Després d’aquesta validació basada en simulació, la factibilitat experimental de les arquitectures de xarxa proposades és avaluada en un entorn de proves distribuït

Green IT - dynamic network topologies

All engineering disciplines are influenced by the global focus on energy consumption reduction and sustainability. Due to its resident inefficiency, The ICT sector is of particular concern, and there has been extensive work to develop sustainability enhancements to networks and/or network devices. Previous work presented dynamic topology concepts in which the behaviour and topology of the devices and the network react dynamically in response to traffic demands, with the intent of placing devices into standby states to reduce energy consumption. The key aim of this study is to develop a dynamic topology mechanism implementation; it proposes a testbed environment and corresponding dynamic topology mechanism that makes use of two programs: one running on a centralised controller, and one running on the network nodes. The former determines the optimal topology based on energy consumption reductions and network traffic, while the latter uses MPLS to implement the topology. The testbed is used to determine the dynamic topology mechanism’s effectiveness and impact on network performance, and does so by subjecting it to controlled variations in network traffic. Quantitative measurements of the dynamic topology mechanism’s network performance metrics are presented and analysed relative to baseline measurements. The analysis shows that the dynamic topology mechanism is quite effective, as the effect on network performance is mostly minimal and the reaction to network traffic variations is sufficiently swift. The system takes approximately 30 seconds to react to traffic variations and implement topology changes, and has negligible effect on jitter, packet loss, and the number of out of order packets. However, it produces an average increase in delay of 8 ms, the source of which requires further investigation. This study proves the feasibility of dynamic topology mechanism implementation, and provides a framework for further development and eventual widespread deployment

Traffic engineering in dynamic optical networks

Traffic Engineering (TE) refers to all the techniques a Service Provider employs to improve the efficiency and reliability of network operations. In IP over Optical (IPO) networks, traffic coming from upper layers is carried over the logical topology defined by the set of established lightpaths. Within this framework then, TE techniques allow to optimize the configuration of optical resources with respect to an highly dynamic traffic demand. TE can be performed with two main methods: if the demand is known only in terms of an aggregated traffic matrix, the problem of automatically updating the configuration of an optical network to accommodate traffic changes is called Virtual Topology Reconfiguration (VTR). If instead the traffic demand is known in terms of data-level connection requests with sub-wavelength granularity, arriving dynamically from some source node to any destination node, the problem is called Dynamic Traffic Grooming (DTG). In this dissertation new VTR algorithms for load balancing in optical networks based on Local Search (LS) techniques are presented. The main advantage of using LS is the minimization of network disruption, since the reconfiguration involves only a small part of the network. A comparison between the proposed schemes and the optimal solutions found via an ILP solver shows calculation time savings for comparable results of network congestion. A similar load balancing technique has been applied to alleviate congestion in an MPLS network, based on the efficient rerouting of Label-Switched Paths (LSP) from the most congested links to allow a better usage of network resources. Many algorithms have been developed to deal with DTG in IPO networks, where most of the attention is focused on optimizing the physical resources utilization by considering specific constraints on the optical node architecture, while very few attention has been put so far on the Quality of Service (QoS) guarantees for the carried traffic. In this thesis a novel Traffic Engineering scheme is proposed to guarantee QoS from both the viewpoint of service differentiation and transmission quality. Another contribution in this thesis is a formal framework for the definition of dynamic grooming policies in IPO networks. The framework is then specialized for an overlay architecture, where the control plane of the IP and optical level are separated, and no information is shared between the two. A family of grooming policies based on constraints on the number of hops and on the bandwidth sharing degree at the IP level is defined, and its performance analyzed in both regular and irregular topologies. While most of the literature on DTG problem implicitly considers the grooming of low-speed connections onto optical channels using a TDM approach, the proposed grooming policies are evaluated here by considering a realistic traffic model which consider a Dynamic Statistical Multiplexing (DSM) approach, i.e. a single wavelength channel is shared between multiple IP elastic traffic flows

Traffic matrix estimation with enhanced origin destination generator algorithm using simulation of real network

The rapid growth of the Internet has made the issue of ensuring reliability and redundancy a big challenge. Studies of these issues using Traffic Engineering and simulation have been extensively done. In Traffic Matrix Estimation (TME), the Origin–Destination Generator algorithm (ODGen) is limited to the number of hops, where the Expectation Maximization (EM) accuracy is 92%. Most studies have not taken into account real traffic parameters and integration of TME models with routing protocols in their simulation models. Also, there is no a comprehensive model consisting of TME, Border Gateway Protocol (BGP) and Hot Potato (HP) routing in the NS-2 network simulator based on real networks. In this research, Integrated Simulated Model (ISM) is introduced consisting of ODGen-HP algorithm and BGP integrated into the NS-2 network simulator. ISM is then used to simulate the infrastructure of a real production network using actual captured traffic data parameters. Validation is then done against the changes in network topology based on packet loss, delay and throughput. Results gave the average error for packet sent by simulated and production networks of 0% and the average error for packet received by simulation and production networks of 3.61%. The network is modelled with a baseline topology where 5 main nodes were connected together, with redundant links for some nodes. The simulations were repeated for link failures, node addition, and node removal. TME used in ISM is based on ODGen, that is optimized with unlimited number of hops, the accuracy of EM increases to 97% and Central Processing Unit complexity is reduced. HP helps in improving the node which experiences a link failure to select shorter distance route to egress router. In the case of a link failure, HP switching time between the links is 0.05 seconds. ISM performance was evaluated by comparing trace file before and after link failure or by adding nodes (up to 32) or removing nodes. The parameters used for comparison are the packets loss, delay and throughput. The ISM error percentage obtained for packets loss is 0.025%, delay 0.013% and throughput 0.003%

ENERGY EFFICIENT WIRED NETWORKING

This research proposes a new dynamic energy management framework for a backbone Internet Protocol over Dense Wavelength Division Multiplexing (IP over DWDM) network. Maintaining the logical IP-layer topology is a key constraint of our architecture whilst saving energy by infrastructure sleeping and virtual router migration. The traffic demand in a Tier 2/3 network typically has a regular diurnal pattern based on people‟s activities, which is high in working hours and much lighter during hours associated with sleep. When the traffic demand is light, virtual router instances can be consolidated to a smaller set of physical platforms and the unneeded physical platforms can be put to sleep to save energy. As the traffic demand increases the sleeping physical platforms can be re-awoken in order to host virtual router instances and so maintain quality of service. Since the IP-layer topology remains unchanged throughout virtual router migration in our framework, there is no network disruption or discontinuities when the physical platforms enter or leave hibernation. However, this migration places extra demands on the optical layer as additional connections are needed to preserve the logical IP-layer topology whilst forwarding traffic to the new virtual router location. Consequently, dynamic optical connection management is needed for the new framework. Two important issues are considered in the framework, i.e. when to trigger the virtual router migration and where to move virtual router instances to? For the first issue, a reactive mechanism is used to trigger the virtual router migration by monitoring the network state. Then, a new evolutionary-based algorithm called VRM_MOEA is proposed for solving the destination physical platform selection problem, which chooses the appropriate location of virtual router instances as traffic demand varies. A novel hybrid simulation platform is developed to measure the performance of new framework, which is able to capture the functionality of the optical layer, the IP layer data-path and the IP/optical control plane. Simulation results show that the performance of network energy saving depends on many factors, such as network topology, quiet and busy thresholds, and traffic load; however, savings of around 30% are possible with typical medium-sized network topologies

Virtualisation and resource allocation in MECEnabled metro optical networks

The appearance of new network services and the ever-increasing network traffic and number of connected devices will push the evolution of current communication networks towards the Future Internet. In the area of optical networks, wavelength routed optical networks (WRONs) are evolving to elastic optical networks (EONs) in which, thanks to the use of OFDM or Nyquist WDM, it is possible to create super-channels with custom-size bandwidth. The basic element in these networks is the lightpath, i.e., all-optical circuits between two network nodes. The establishment of lightpaths requires the selection of the route that they will follow and the portion of the spectrum to be used in order to carry the requested traffic from the source to the destination node. That problem is known as the routing and spectrum assignment (RSA) problem, and new algorithms must be proposed to address this design problem. Some early studies on elastic optical networks studied gridless scenarios, in which a slice of spectrum of variable size is assigned to a request. However, the most common approach to the spectrum allocation is to divide the spectrum into slots of fixed width and allocate multiple, consecutive spectrum slots to each lightpath, depending on the requested bandwidth. Moreover, EONs also allow the proposal of more flexible routing and spectrum assignment techniques, like the split-spectrum approach in which the request is divided into multiple "sub-lightpaths". In this thesis, four RSA algorithms are proposed combining two different levels of flexibility with the well-known k-shortest paths and first fit heuristics. After comparing the performance of those methods, a novel spectrum assignment technique, Best Gap, is proposed to overcome the inefficiencies emerged when combining the first fit heuristic with highly flexible networks. A simulation study is presented to demonstrate that, thanks to the use of Best Gap, EONs can exploit the network flexibility and reduce the blocking ratio. On the other hand, operators must face profound architectural changes to increase the adaptability and flexibility of networks and ease their management. Thanks to the use of network function virtualisation (NFV), the necessary network functions that must be applied to offer a service can be deployed as virtual appliances hosted by commodity servers, which can be located in data centres, network nodes or even end-user premises. The appearance of new computation and networking paradigms, like multi-access edge computing (MEC), may facilitate the adaptation of communication networks to the new demands. Furthermore, the use of MEC technology will enable the possibility of installing those virtual network functions (VNFs) not only at data centres (DCs) and central offices (COs), traditional hosts of VFNs, but also at the edge nodes of the network. Since data processing is performed closer to the enduser, the latency associated to each service connection request can be reduced. MEC nodes will be usually connected between them and with the DCs and COs by optical networks. In such a scenario, deploying a network service requires completing two phases: the VNF-placement, i.e., deciding the number and location of VNFs, and the VNF-chaining, i.e., connecting the VNFs that the traffic associated to a service must transverse in order to establish the connection. In the chaining process, not only the existence of VNFs with available processing capacity, but the availability of network resources must be taken into account to avoid the rejection of the connection request. Taking into consideration that the backhaul of this scenario will be usually based on WRONs or EONs, it is necessary to design the virtual topology (i.e., the set of lightpaths established in the networks) in order to transport the tra c from one node to another. The process of designing the virtual topology includes deciding the number of connections or lightpaths, allocating them a route and spectral resources, and finally grooming the traffic into the created lightpaths. Lastly, a failure in the equipment of a node in an NFV environment can cause the disruption of the SCs traversing the node. This can cause the loss of huge amounts of data and affect thousands of end-users. In consequence, it is key to provide the network with faultmanagement techniques able to guarantee the resilience of the established connections when a node fails. For the mentioned reasons, it is necessary to design orchestration algorithms which solve the VNF-placement, chaining and network resource allocation problems in 5G networks with optical backhaul. Moreover, some versions of those algorithms must also implements protection techniques to guarantee the resilience system in case of failure. This thesis makes contribution in that line. Firstly, a genetic algorithm is proposed to solve the VNF-placement and VNF-chaining problems in a 5G network with optical backhaul based on star topology: GASM (genetic algorithm for effective service mapping). Then, we propose a modification of that algorithm in order to be applied to dynamic scenarios in which the reconfiguration of the planning is allowed. Furthermore, we enhanced the modified algorithm to include a learning step, with the objective of improving the performance of the algorithm. In this thesis, we also propose an algorithm to solve not only the VNF-placement and VNF-chaining problems but also the design of the virtual topology, considering that a WRON is deployed as the backhaul network connecting MEC nodes and CO. Moreover, a version including individual VNF protection against node failure has been also proposed and the effect of using shared/dedicated and end-to-end SC/individual VNF protection schemes are also analysed. Finally, a new algorithm that solves the VNF-placement and chaining problems and the virtual topology design implementing a new chaining technique is also proposed. Its corresponding versions implementing individual VNF protection are also presented. Furthermore, since the method works with any type of WDM mesh topologies, a technoeconomic study is presented to compare the effect of using different network topologies in both the network performance and cost.Departamento de Teoría de la Señal y Comunicaciones e Ingeniería TelemáticaDoctorado en Tecnologías de la Información y las Telecomunicacione

Accurate and Resource-Efficient Monitoring for Future Networks

Monitoring functionality is a key component of any network management system. It is essential for profiling network resource usage, detecting attacks, and capturing the performance of a multitude of services using the network. Traditional monitoring solutions operate on long timescales producing periodic reports, which are mostly used for manual and infrequent network management tasks. However, these practices have been recently questioned by the advent of Software Defined Networking (SDN). By empowering management applications with the right tools to perform automatic, frequent, and fine-grained network reconfigurations, SDN has made these applications more dependent than before on the accuracy and timeliness of monitoring reports. As a result, monitoring systems are required to collect considerable amounts of heterogeneous measurement data, process them in real-time, and expose the resulting knowledge in short timescales to network decision-making processes. Satisfying these requirements is extremely challenging given today’s larger network scales, massive and dynamic traffic volumes, and the stringent constraints on time availability and hardware resources. This PhD thesis tackles this important challenge by investigating how an accurate and resource-efficient monitoring function can be realised in the context of future, software-defined networks. Novel monitoring methodologies, designs, and frameworks are provided in this thesis, which scale with increasing network sizes and automatically adjust to changes in the operating conditions. These achieve the goal of efficient measurement collection and reporting, lightweight measurement- data processing, and timely monitoring knowledge delivery