Hypersparse Neural Network Analysis of Large-Scale Internet Traffic

The Internet is transforming our society, necessitating a quantitative understanding of Internet traffic. Our team collects and curates the largest publicly available Internet traffic data containing 50 billion packets. Utilizing a novel hypersparse neural network analysis of "video" streams of this traffic using 10,000 processors in the MIT SuperCloud reveals a new phenomena: the importance of otherwise unseen leaf nodes and isolated links in Internet traffic. Our neural network approach further shows that a two-parameter modified Zipf-Mandelbrot distribution accurately describes a wide variety of source/destination statistics on moving sample windows ranging from 100,000 to 100,000,000 packets over collections that span years and continents. The inferred model parameters distinguish different network streams and the model leaf parameter strongly correlates with the fraction of the traffic in different underlying network topologies. The hypersparse neural network pipeline is highly adaptable and different network statistics and training models can be incorporated with simple changes to the image filter functions.Comment: 11 pages, 10 figures, 3 tables, 60 citations; to appear in IEEE High Performance Extreme Computing (HPEC) 201

Implications of Selfish Neighbor Selection in Overlay Networks

In a typical overlay network for routing or content sharing, each node must select a fixed number of immediate overlay neighbors for routing traffic or content queries. A selfish node entering such a network would select neighbors so as to minimize the weighted sum of expected access costs to all its destinations. Previous work on selfish neighbor selection has built intuition with simple models where edges are undirected, access costs are modeled by hop-counts, and nodes have potentially unbounded degrees. However, in practice, important constraints not captured by these models lead to richer games with substantively and fundamentally different outcomes. Our work models neighbor selection as a game involving directed links, constraints on the number of allowed neighbors, and costs reflecting both network latency and node preference. We express a node's "best response" wiring strategy as a k-median problem on asymmetric distance, and use this formulation to obtain pure Nash equilibria. We experimentally examine the properties of such stable wirings on synthetic topologies, as well as on real topologies and maps constructed from PlanetLab and AS-level Internet measurements. Our results indicate that selfish nodes can reap substantial performance benefits when connecting to overlay networks composed of non-selfish nodes. On the other hand, in overlays that are dominated by selfish nodes, the resulting stable wirings are optimized to such great extent that even non-selfish newcomers can extract near-optimal performance through naive wiring strategies.Marie Curie Outgoing International Fellowship of the EU (MOIF-CT-2005-007230); National Science Foundation (CNS Cybertrust 0524477, CNS NeTS 0520166, CNS ITR 0205294, EIA RI 020206

An experimental study of client-side Spotify peering behaviour

Spotify is a popular music-streaming service which has seen widespread use across Europe. While Spotify’s server-side behaviour has previously been studied, little is known about the client-side behaviour. In this paper, we describe an experimental study where we collect packet headers for Spotify traffic over multiple 24-hour time frames at a client host. Two distinct types of behaviour are observed, when tracks are being downloaded, and when the client is only serving requests from other peers. We also note wide variation in connection lifetimes, as seen in other studies of peer-to-peer systems. These findings are relevant for improving Spotify itself, and for the designers of other hybrid peer-to-peer and server-based distribution architectures

P2P IPTV Measurement: A Comparison Study

With the success of P2P file sharing, new emerging P2P applications arise on the Internet for streaming content like voice (VoIP) or live video (IPTV). Nowadays, there are lots of works measuring P2P file sharing or P2P telephony systems, but there is still no comprehensive study about P2P IPTV, whereas it should be massively used in the future. During the last FIFA world cup, we measured network traffic generated by P2P IPTV applications like PPlive, PPstream, TVants and Sopcast. In this paper we analyze some of our results during the same games for the applications. We focus on traffic statistics and churn of peers within these P2P networks. Our objectives are threefold: we point out the traffic generated to understand the impact they will have on the network, we try to infer the mechanisms of such applications and highlight differences, and we give some insights about the users' behavior.Comment: 10 page

Cross-Layer Peer-to-Peer Track Identification and Optimization Based on Active Networking

P2P applications appear to emerge as ultimate killer applications due to their ability to construct highly dynamic overlay topologies with rapidly-varying and unpredictable traffic dynamics, which can constitute a serious challenge even for significantly over-provisioned IP networks. As a result, ISPs are facing new, severe network management problems that are not guaranteed to be addressed by statically deployed network engineering mechanisms. As a first step to a more complete solution to these problems, this paper proposes a P2P measurement, identification and optimisation architecture, designed to cope with the dynamicity and unpredictability of existing, well-known and future, unknown P2P systems. The purpose of this architecture is to provide to the ISPs an effective and scalable approach to control and optimise the traffic produced by P2P applications in their networks. This can be achieved through a combination of different application and network-level programmable techniques, leading to a crosslayer identification and optimisation process. These techniques can be applied using Active Networking platforms, which are able to quickly and easily deploy architectural components on demand. This flexibility of the optimisation architecture is essential to address the rapid development of new P2P protocols and the variation of known protocols

Evidences Behind Skype Outage

Skype is one of the most successful VoIP application in the current Internet spectrum. One of the most peculiar characteristics of Skype is that it relies on a P2P infrastructure for the exchange of signaling information amongst active peers. During August 2007, an unexpected outage hit the Skype overlay, yielding to a service blackout that lasted for more than two days: this paper aims at throwing light to this event. Leveraging on the use of an accurate Skype classification engine, we carry on an experimental study of Skype signaling during the outage. In particular, we focus on the signaling traffic before, during and after the outage, in the attempt to quantify interesting properties of the event. While it is very difficult to gather clear insights concerning the root causes of the breakdown itself, the collected measurement allow nevertheless to quantify several interesting aspects of the outage: for instance, measurements show that the outage caused, on average, a 3-fold increase of signaling traffic and a 10-fold increase of number of contacted peers, topping to more than 11 million connections for the most active node in our network - which immediately gives the feeling of the extent of the phenomeno

Dovetail: Stronger Anonymity in Next-Generation Internet Routing

Current low-latency anonymity systems use complex overlay networks to conceal a user's IP address, introducing significant latency and network efficiency penalties compared to normal Internet usage. Rather than obfuscating network identity through higher level protocols, we propose a more direct solution: a routing protocol that allows communication without exposing network identity, providing a strong foundation for Internet privacy, while allowing identity to be defined in those higher level protocols where it adds value. Given current research initiatives advocating "clean slate" Internet designs, an opportunity exists to design an internetwork layer routing protocol that decouples identity from network location and thereby simplifies the anonymity problem. Recently, Hsiao et al. proposed such a protocol (LAP), but it does not protect the user against a local eavesdropper or an untrusted ISP, which will not be acceptable for many users. Thus, we propose Dovetail, a next-generation Internet routing protocol that provides anonymity against an active attacker located at any single point within the network, including the user's ISP. A major design challenge is to provide this protection without including an application-layer proxy in data transmission. We address this challenge in path construction by using a matchmaker node (an end host) to overlap two path segments at a dovetail node (a router). The dovetail then trims away part of the path so that data transmission bypasses the matchmaker. Additional design features include the choice of many different paths through the network and the joining of path segments without requiring a trusted third party. We develop a systematic mechanism to measure the topological anonymity of our designs, and we demonstrate the privacy and efficiency of our proposal by simulation, using a model of the complete Internet at the AS-level