Large Scale Cross-Correlations in Internet Traffic

The Internet is a complex network of interconnected routers and the existence of collective behavior such as congestion suggests that the correlations between different connections play a crucial role. It is thus critical to measure and quantify these correlations. We use methods of random matrix theory (RMT) to analyze the cross-correlation matrix C of information flow changes of 650 connections between 26 routers of the French scientific network `Renater'. We find that C has the universal properties of the Gaussian orthogonal ensemble of random matrices: The distribution of eigenvalues--up to a rescaling which exhibits a typical correlation time of the order 10 minutes--and the spacing distribution follow the predictions of RMT. There are some deviations for large eigenvalues which contain network-specific information and which identify genuine correlations between connections. The study of the most correlated connections reveals the existence of `active centers' which are exchanging information with a large number of routers thereby inducing correlations between the corresponding connections. These strong correlations could be a reason for the observed self-similarity in the WWW traffic.Comment: 7 pages, 6 figures, final versio

A critical look at power law modelling of the Internet

This paper takes a critical look at the usefulness of power law models of the Internet. The twin focuses of the paper are Internet traffic and topology generation. The aim of the paper is twofold. Firstly it summarises the state of the art in power law modelling particularly giving attention to existing open research questions. Secondly it provides insight into the failings of such models and where progress needs to be made for power law research to feed through to actual improvements in network performance.Comment: To appear Computer Communication

Traffic measurement and analysis

Measurement and analysis of real traffic is important to gain knowledge about the characteristics of the traffic. Without measurement, it is impossible to build realistic traffic models. It is recent that data traffic was found to have self-similar properties. In this thesis work traffic captured on the network at SICS and on the Supernet, is shown to have this fractal-like behaviour. The traffic is also examined with respect to which protocols and packet sizes are present and in what proportions. In the SICS trace most packets are small, TCP is shown to be the predominant transport protocol and NNTP the most common application. In contrast to this, large UDP packets sent between not well-known ports dominates the Supernet traffic. Finally, characteristics of the client side of the WWW traffic are examined more closely. In order to extract useful information from the packet trace, web browsers use of TCP and HTTP is investigated including new features in HTTP/1.1 such as persistent connections and pipelining. Empirical probability distributions are derived describing session lengths, time between user clicks and the amount of data transferred due to a single user click. These probability distributions make up a simple model of WWW-sessions

From Linked Data to Relevant Data -- Time is the Essence

The Semantic Web initiative puts emphasis not primarily on putting data on the Web, but rather on creating links in a way that both humans and machines can explore the Web of data. When such users access the Web, they leave a trail as Web servers maintain a history of requests. Web usage mining approaches have been studied since the beginning of the Web given the log's huge potential for purposes such as resource annotation, personalization, forecasting etc. However, the impact of any such efforts has not really gone beyond generating statistics detailing who, when, and how Web pages maintained by a Web server were visited.Comment: 1st International Workshop on Usage Analysis and the Web of Data (USEWOD2011) in the 20th International World Wide Web Conference (WWW2011), Hyderabad, India, March 28th, 201

Validation of hardware events for successful performance pattern identification in High Performance Computing

Hardware performance monitoring (HPM) is a crucial ingredient of performance analysis tools. While there are interfaces like LIKWID, PAPI or the kernel interface perf\_event which provide HPM access with some additional features, many higher level tools combine event counts with results retrieved from other sources like function call traces to derive (semi-)automatic performance advice. However, although HPM is available for x86 systems since the early 90s, only a small subset of the HPM features is used in practice. Performance patterns provide a more comprehensive approach, enabling the identification of various performance-limiting effects. Patterns address issues like bandwidth saturation, load imbalance, non-local data access in ccNUMA systems, or false sharing of cache lines. This work defines HPM event sets that are best suited to identify a selection of performance patterns on the Intel Haswell processor. We validate the chosen event sets for accuracy in order to arrive at a reliable pattern detection mechanism and point out shortcomings that cannot be easily circumvented due to bugs or limitations in the hardware

Core-periphery organization of complex networks

Networks may, or may not, be wired to have a core that is both itself densely connected and central in terms of graph distance. In this study we propose a coefficient to measure if the network has such a clear-cut core-periphery dichotomy. We measure this coefficient for a number of real-world and model networks and find that different classes of networks have their characteristic values. For example do geographical networks have a strong core-periphery structure, while the core-periphery structure of social networks (despite their positive degree-degree correlations) is rather weak. We proceed to study radial statistics of the core, i.e. properties of the n-neighborhoods of the core vertices for increasing n. We find that almost all networks have unexpectedly many edges within n-neighborhoods at a certain distance from the core suggesting an effective radius for non-trivial network processes

Beyond Counting: New Perspectives on the Active IPv4 Address Space

In this study, we report on techniques and analyses that enable us to capture Internet-wide activity at individual IP address-level granularity by relying on server logs of a large commercial content delivery network (CDN) that serves close to 3 trillion HTTP requests on a daily basis. Across the whole of 2015, these logs recorded client activity involving 1.2 billion unique IPv4 addresses, the highest ever measured, in agreement with recent estimates. Monthly client IPv4 address counts showed constant growth for years prior, but since 2014, the IPv4 count has stagnated while IPv6 counts have grown. Thus, it seems we have entered an era marked by increased complexity, one in which the sole enumeration of active IPv4 addresses is of little use to characterize recent growth of the Internet as a whole. With this observation in mind, we consider new points of view in the study of global IPv4 address activity. Our analysis shows significant churn in active IPv4 addresses: the set of active IPv4 addresses varies by as much as 25% over the course of a year. Second, by looking across the active addresses in a prefix, we are able to identify and attribute activity patterns to network restructurings, user behaviors, and, in particular, various address assignment practices. Third, by combining spatio-temporal measures of address utilization with measures of traffic volume, and sampling-based estimates of relative host counts, we present novel perspectives on worldwide IPv4 address activity, including empirical observation of under-utilization in some areas, and complete utilization, or exhaustion, in others.Comment: in Proceedings of ACM IMC 201

The structure of Inter-Urban traffic: A weighted network analysis

We study the structure of the network representing the interurban commuting traffic of the Sardinia region, Italy, which amounts to 375 municipalities and 1,600,000 inhabitants. We use a weighted network representation where vertices correspond to towns and the edges to the actual commuting flows among those. We characterize quantitatively both the topological and weighted properties of the resulting network. Interestingly, the statistical properties of commuting traffic exhibit complex features and non-trivial relations with the underlying topology. We characterize quantitatively the traffic backbone among large cities and we give evidences for a very high heterogeneity of the commuter flows around large cities. We also discuss the interplay between the topological and dynamical properties of the network as well as their relation with socio-demographic variables such as population and monthly income. This analysis may be useful at various stages in environmental planning and provides analytical tools for a wide spectrum of applications ranging from impact evaluation to decision-making and planning support.Comment: 12 pages, 12 figures, 4 tables; 1 missing ref added and minor revision