Structural Analysis of Network Traffic Matrix via Relaxed Principal Component Pursuit

The network traffic matrix is widely used in network operation and management. It is therefore of crucial importance to analyze the components and the structure of the network traffic matrix, for which several mathematical approaches such as Principal Component Analysis (PCA) were proposed. In this paper, we first argue that PCA performs poorly for analyzing traffic matrix that is polluted by large volume anomalies, and then propose a new decomposition model for the network traffic matrix. According to this model, we carry out the structural analysis by decomposing the network traffic matrix into three sub-matrices, namely, the deterministic traffic, the anomaly traffic and the noise traffic matrix, which is similar to the Robust Principal Component Analysis (RPCA) problem previously studied in [13]. Based on the Relaxed Principal Component Pursuit (Relaxed PCP) method and the Accelerated Proximal Gradient (APG) algorithm, we present an iterative approach for decomposing a traffic matrix, and demonstrate its efficiency and flexibility by experimental results. Finally, we further discuss several features of the deterministic and noise traffic. Our study develops a novel method for the problem of structural analysis of the traffic matrix, which is robust against pollution of large volume anomalies.Comment: Accepted to Elsevier Computer Network

Transport on complex networks: Flow, jamming and optimization

Many transport processes on networks depend crucially on the underlying network geometry, although the exact relationship between the structure of the network and the properties of transport processes remain elusive. In this paper we address this question by using numerical models in which both structure and dynamics are controlled systematically. We consider the traffic of information packets that include driving, searching and queuing. We present the results of extensive simulations on two classes of networks; a correlated cyclic scale-free network and an uncorrelated homogeneous weakly clustered network. By measuring different dynamical variables in the free flow regime we show how the global statistical properties of the transport are related to the temporal fluctuations at individual nodes (the traffic noise) and the links (the traffic flow). We then demonstrate that these two network classes appear as representative topologies for optimal traffic flow in the regimes of low density and high density traffic, respectively. We also determine statistical indicators of the pre-jamming regime on different network geometries and discuss the role of queuing and dynamical betweenness for the traffic congestion. The transition to the jammed traffic regime at a critical posting rate on different network topologies is studied as a phase transition with an appropriate order parameter. We also address several open theoretical problems related to the network dynamics

Statistical Traffic State Analysis in Large-scale Transportation Networks Using Locality-Preserving Non-negative Matrix Factorization

Statistical traffic data analysis is a hot topic in traffic management and control. In this field, current research progresses focus on analyzing traffic flows of individual links or local regions in a transportation network. Less attention are paid to the global view of traffic states over the entire network, which is important for modeling large-scale traffic scenes. Our aim is precisely to propose a new methodology for extracting spatio-temporal traffic patterns, ultimately for modeling large-scale traffic dynamics, and long-term traffic forecasting. We attack this issue by utilizing Locality-Preserving Non-negative Matrix Factorization (LPNMF) to derive low-dimensional representation of network-level traffic states. Clustering is performed on the compact LPNMF projections to unveil typical spatial patterns and temporal dynamics of network-level traffic states. We have tested the proposed method on simulated traffic data generated for a large-scale road network, and reported experimental results validate the ability of our approach for extracting meaningful large-scale space-time traffic patterns. Furthermore, the derived clustering results provide an intuitive understanding of spatial-temporal characteristics of traffic flows in the large-scale network, and a basis for potential long-term forecasting.Comment: IET Intelligent Transport Systems (2013

Sonification of Network Traffic Flow for Monitoring and Situational Awareness

Maintaining situational awareness of what is happening within a network is challenging, not least because the behaviour happens within computers and communications networks, but also because data traffic speeds and volumes are beyond human ability to process. Visualisation is widely used to present information about the dynamics of network traffic dynamics. Although it provides operators with an overall view and specific information about particular traffic or attacks on the network, it often fails to represent the events in an understandable way. Visualisations require visual attention and so are not well suited to continuous monitoring scenarios in which network administrators must carry out other tasks. Situational awareness is critical and essential for decision-making in the domain of computer network monitoring where it is vital to be able to identify and recognize network environment behaviours.Here we present SoNSTAR (Sonification of Networks for SiTuational AwaReness), a real-time sonification system to be used in the monitoring of computer networks to support the situational awareness of network administrators. SoNSTAR provides an auditory representation of all the TCP/IP protocol traffic within a network based on the different traffic flows between between network hosts. SoNSTAR raises situational awareness levels for computer network defence by allowing operators to achieve better understanding and performance while imposing less workload compared to visual techniques. SoNSTAR identifies the features of network traffic flows by inspecting the status flags of TCP/IP packet headers and mapping traffic events to recorded sounds to generate a soundscape representing the real-time status of the network traffic environment. Listening to the soundscape allows the administrator to recognise anomalous behaviour quickly and without having to continuously watch a computer screen.Comment: 17 pages, 7 figures plus supplemental material in Github repositor

Network link dimensioning : a measurement & modeling based approach

Adequate network link dimensioning requires a thorough insight into the interrelationship between: (i) the traffic offered (in terms of the average load, but also its fluctuations), (ii) the desired level of performance, and (iii) the required bandwidth capacity. It is clear that more capacity is needed when the average traffic load becomes higher, the fluctuations become fiercer, or the performance criterion becomesmore stringent. Existing approaches to network link dimensioning are often based on rules of thumb, e.g., `take the average traffic rate at timeswhen the network is relatively busy, and add 30%to cater for fluctuations¿. Clearly, such an approach does not explicitly incorporate the fierceness of the traffic rate¿s fluctuations, or the desired level of performance.\ud A common approach to estimate the average traffic rate is as follows. A networkmanager regularly polls the so-called Interfaces Group MIB via the Simple NetworkManagement Protocol (SNMP), for instance through a tool such as the Multi-Router Traffic Grapher (MRTG). This yields the average rate of the offered traffic since the last poll. The polling interval generally is in the order of 5minutes. Evidently, the fierceness of fluctuation of the traffic rate within these 5 minute intervals is unknown to the network manager. These fluctuations may, however, be considerably large, and noticeable to users of the network. If, at timescales of say 5 seconds, more traffic is offered to a network link than it can transfer during that interval, traffic may be lost. Such loss is generally known as possibly leading to performance degradation and this may well be noticeable to a network user; for instance, entirewords may be lost in a (voice) conversation. Hence, it is in the interest of network users, and for obvious business reasons also to network operators, to have sufficient bandwidth capacity available to meet the demand at timescales considerably smaller than 5 minutes. \ud In this thesis, we develop an alternative approach to network link dimensioning, which explicitly incorporates the offered traffic in terms of both the average rate as well as its fluctuations at small timescales, and the desired level of performance. This is expressed throughmathematical formulas that give the required bandwidth capacity, given the characteristics of the offered traffic, and the performance criterion