Traffic matrix estimation in IP networks

An Origin-Destination (OD) traffic matrix provides a major input to the design, planning and management of a telecommunications network. Since the Internet is being proposed as the principal delivery mechanism for telecommunications traffic at the present time, and this network is not owned or managed by a single entity, there are significant challenges for network planners and managers needing to determine equipment and topology configurations for the various sections of the Internet that are currently the responsibility of ISPs and traditional telcos. Planning of these sub-networks typically requires a traffic matrix of demands that is then used to infer the flows on the administrator's network. Unfortunately, computation of the traffic matrix from measurements of individual flows is extremely difficult due to the fact that the problem formulation generally leads to the need to solve an under-determined system of equations. Thus, there has been a major effort from among researchers to obtain the traffic matrix using various inference techniques. The major contribution of this thesis is the development of inference techniques for traffic matrix estimation problem according to three different approaches, viz: (1) deterministic, (2) statistical, and (3) dynamic approaches. Firstly, for the deterministic approach, the traffic matrix estimation problem is formulated as a nonlinear optimization problem based on the generalized Kruithof approach which uses the Kullback distance to measure the probabilistic distance between two traffic matrices. In addition, an algorithm using the Affine scaling method is developed to solve the constrained optimization problem. Secondly, for the statistical approach, a series of traffic matrices are obtained by applying a standard deterministic approach. The components of these matrices represent estimates of the volumes of flows being exchanged between all pairs of nodes at the respective measurement points and they form a stochastic counting process. Then, a Markovian Arrival Process of order two (MAP-2) is applied to model the counting processes formed from this series of estimated traffic matrices. Thirdly, for the dynamic approach, the dual problem of the multi-commodity flow problem is formulated to obtain a set of link weights. The new weight set enables flows to be rerouted along new paths, which create new constraints to overcome the under-determined nature of traffic matrix estimation. Since a weight change disturbs a network, the impact of weight changes on the network is investigated by using simulation based on the well-known ns2 simulator package. Finally, we introduce two network applications that make use of the deterministic and the statistical approaches to obtain a traffic matrix respectively and also describe a scenario for the use of the dynamic approach

Aspects of proactive traffic engineering in IP networks

To deliver a reliable communication service over the Internet it is essential for the network operator to manage the traffic situation in the network. The traffic situation is controlled by the routing function which determines what path traffic follows from source to destination. Current practices for setting routing parameters in IP networks are designed to be simple to manage. This can lead to congestion in parts of the network while other parts of the network are far from fully utilized. In this thesis we explore issues related to optimization of the routing function to balance load in the network and efficiently deliver a reliable communication service to the users. The optimization takes into account not only the traffic situation under normal operational conditions, but also traffic situations that appear under a wide variety of circumstances deviating from the nominal case. In order to balance load in the network knowledge of the traffic situations is needed. Consequently, in this thesis we investigate methods for efficient derivation of the traffic situation. The derivation is based on estimation of traffic demands from link load measurements. The advantage of using link load measurements is that they are easily obtained and consist of a limited amount of data that need to be processed. We evaluate and demonstrate how estimation based on link counts gives the operator a fast and accurate description of the traffic demands. For the evaluation we have access to a unique data set of complete traffic demands from an operational IP backbone. However, to honor service level agreements at all times the variability of the traffic needs to be accounted for in the load balancing. In addition, optimization techniques are often sensitive to errors and variations in input data. Hence, when an optimized routing setting is subjected to real traffic demands in the network, performance often deviate from what can be anticipated from the optimization. Thus, we identify and model different traffic uncertainties and describe how the routing setting can be optimized, not only for a nominal case, but for a wide range of different traffic situations that might appear in the network. Our results can be applied in MPLS enabled networks as well as in networks using link state routing protocols such as the widely used OSPF and IS-IS protocols. Only minor changes may be needed in current networks to implement our algorithms. The contributions of this thesis is that we: demonstrate that it is possible to estimate the traffic matrix with acceptable precision, and we develop methods and models for common traffic uncertainties to account for these uncertainties in the optimization of the routing configuration. In addition, we identify important properties in the structure of the traffic to successfully balance uncertain and varying traffic demands

Accurate and Resource-Efficient Monitoring for Future Networks

Monitoring functionality is a key component of any network management system. It is essential for profiling network resource usage, detecting attacks, and capturing the performance of a multitude of services using the network. Traditional monitoring solutions operate on long timescales producing periodic reports, which are mostly used for manual and infrequent network management tasks. However, these practices have been recently questioned by the advent of Software Defined Networking (SDN). By empowering management applications with the right tools to perform automatic, frequent, and fine-grained network reconfigurations, SDN has made these applications more dependent than before on the accuracy and timeliness of monitoring reports. As a result, monitoring systems are required to collect considerable amounts of heterogeneous measurement data, process them in real-time, and expose the resulting knowledge in short timescales to network decision-making processes. Satisfying these requirements is extremely challenging given today’s larger network scales, massive and dynamic traffic volumes, and the stringent constraints on time availability and hardware resources. This PhD thesis tackles this important challenge by investigating how an accurate and resource-efficient monitoring function can be realised in the context of future, software-defined networks. Novel monitoring methodologies, designs, and frameworks are provided in this thesis, which scale with increasing network sizes and automatically adjust to changes in the operating conditions. These achieve the goal of efficient measurement collection and reporting, lightweight measurement- data processing, and timely monitoring knowledge delivery

Design of Wireless Communication Networks for Cyber-Physical Systems with Application to Smart Grid

Cyber-Physical Systems (CPS) are the next generation of engineered systems in which computing, communication, and control technologies are tightly integrated. On one hand, CPS are generally large with components spatially distributed in physical world that has lots of dynamics; on the other hand, CPS are connected, and must be robust and responsive. Smart electric grid, smart transportation system are examples of emerging CPS that have significant and far-reaching impact on our daily life. In this dissertation, we design wireless communication system for CPS. To make CPS robust and responsive, it is critical to have a communication subsystem that is reliable, adaptive, and scalable. Our design uses a layered structure, which includes physical layer, multiple access layer, network layer, and application layer. Emphases are placed on multiple access and network layer. At multiple access layer, we have designed three approaches, namely compressed multiple access, sample-contention multiple access, and prioritized multiple access, for reliable and selective multiple access. At network layer, we focus on the problem of creating reliable route, with service interruption anticipated. We propose two methods: the first method is a centralized one that creates backup path around zones posing high interruption risk; the other method is a distributed one that utilizes Ant Colony Optimization (ACO) and positive feedback, and is able to update multipath dynamically. Applications are treated as subscribers to the data service provided by the communication system. Their data quality requirements and Quality of Service (QoS) feedback are incorporated into cross-layer optimization in our design. We have evaluated our design through both simulation and testbed. Our design demonstrates desired reliability, scalability and timeliness in data transmission. Performance gain is observed over conventional approaches as such random access

Performance modelling with adaptive hidden Markov models and discriminatory processor sharing queues

In modern computer systems, workload varies at different times and locations. It is important to model the performance of such systems via workload models that are both representative and efficient. For example, model-generated workloads represent realistic system behaviour, especially during peak times, when it is crucial to predict and address performance bottlenecks. In this thesis, we model performance, namely throughput and delay, using adaptive models and discrete queues. Hidden Markov models (HMMs) parsimoniously capture the correlation and burstiness of workloads with spatiotemporal characteristics. By adapting the batch training of standard HMMs to incremental learning, online HMMs act as benchmarks on workloads obtained from live systems (i.e. storage systems and financial markets) and reduce time complexity of the Baum-Welch algorithm. Similarly, by extending HMM capabilities to train on multiple traces simultaneously it follows that workloads of different types are modelled in parallel by a multi-input HMM. Typically, the HMM-generated traces verify the throughput and burstiness of the real data. Applications of adaptive HMMs include predicting user behaviour in social networks and performance-energy measurements in smartphone applications. Equally important is measuring system delay through response times. For example, workloads such as Internet traffic arriving at routers are affected by queueing delays. To meet quality of service needs, queueing delays must be minimised and, hence, it is important to model and predict such queueing delays in an efficient and cost-effective manner. Therefore, we propose a class of discrete, processor-sharing queues for approximating queueing delay as response time distributions, which represent service level agreements at specific spatiotemporal levels. We adapt discrete queues to model job arrivals with distributions given by a Markov-modulated Poisson process (MMPP) and served under discriminatory processor-sharing scheduling. Further, we propose a dynamic strategy of service allocation to minimise delays in UDP traffic flows whilst maximising a utility function.Open Acces

Investigating the Effects of Network Dynamics on Quality of Delivery Prediction and Monitoring for Video Delivery Networks

Video streaming over the Internet requires an optimized delivery system given the advances in network architecture, for example, Software Defined Networks. Machine Learning (ML) models have been deployed in an attempt to predict the quality of the video streams. Some of these efforts have considered the prediction of Quality of Delivery (QoD) metrics of the video stream in an effort to measure the quality of the video stream from the network perspective. In most cases, these models have either treated the ML algorithms as black-boxes or failed to capture the network dynamics of the associated video streams. This PhD investigates the effects of network dynamics in QoD prediction using ML techniques. The hypothesis that this thesis investigates is that ML techniques that model the underlying network dynamics achieve accurate QoD and video quality predictions and measurements. The thesis results demonstrate that the proposed techniques offer performance gains over approaches that fail to consider network dynamics. This thesis results highlight that adopting the correct model by modelling the dynamics of the network infrastructure is crucial to the accuracy of the ML predictions. These results are significant as they demonstrate that improved performance is achieved at no additional computational or storage cost. These techniques can help the network manager, data center operatives and video service providers take proactive and corrective actions for improved network efficiency and effectiveness