Pushing BitTorrent Locality to the Limit

Peer-to-peer (P2P) locality has recently raised a lot of interest in the community. Indeed, whereas P2P content distribution enables financial savings for the content providers, it dramatically increases the traffic on inter-ISP links. To solve this issue, the idea to keep a fraction of the P2P traffic local to each ISP was introduced a few years ago. Since then, P2P solutions exploiting locality have been introduced. However, several fundamental issues on locality still need to be explored. In particular, how far can we push locality, and what is, at the scale of the Internet, the reduction of traffic that can be achieved with locality? In this paper, we perform extensive experiments on a controlled environment with up to 10 000 BitTorrent clients to evaluate the impact of high locality on inter-ISP links traffic and peers download completion time. We introduce two simple mechanisms that make high locality possible in challenging scenarios and we show that we save up to several orders of magnitude inter-ISP traffic compared to traditional locality without adversely impacting peers download completion time. In addition, we crawled 214 443 torrents representing 6 113 224 unique peers spread among 9 605 ASes. We show that whereas the torrents we crawled generated 11.6 petabytes of inter-ISP traffic, our locality policy implemented for all torrents would have reduced the global inter-ISP traffic by 40%

Experimental analysis of the socio-economic phenomena in the BitTorrent ecosystem

BitTorrent is the most successful Peer-to-Peer (P2P) application and is responsible for a major portion of Internet traffic. It has been largely studied using simulations, models and real measurements. Although simulations and modelling are easier to perform, they typically simplify analysed problems and in case of BitTorrent they are likely to miss some of the effects which occur in real swarms. Thus, in this thesis we rely on real measurements. In the first part of the thesis we present the summary of measurement techniques used so far and we use it as a base to design our tools that allow us to perform different types of analysis at different resolution level. Using these tools we collect several large-scale datasets to study different aspects of BitTorrent with a special focus on socio-economic aspects. Using our datasets, we first investigate the topology of real BitTorrent swarms and how the traffic is actually exchanged among peers. Our analysis shows that the resilience of BitTorrent swarms is lower than corresponding random graphs. We also observe that ISP policies, locality-aware clients and network events (e.g., network congestion) lead to locality-biased composition of neighbourhood in the swarms. This means that the peer contains more neighbours from local provider than expected from purely random neighbours selection process. Those results are of interest to the companies which use BitTorrent for daily operations as well as for ISPs which carry BitTorrent traffic. In the next part of the thesis we look at the BitTorrent from the perspective of the content and content publishers in a major BitTorrent portals. We focus on the factors that seem to drive the popularity of the BitTorrent and, as a result, could affect its associated traffic in the Internet. We show that a small fraction of publishers (around 100 users) is responsible for more than two-thirds of the published content. Those publishers can be divided into two groups: (i) profit driven and (ii)fake publishers. The former group leverages the published copyrighted content (typically very popular) on BitTorrent portals to attract content consumers to their web sites for financial gain. Removing this group may have a significant impact on the popularity of BitTorrent portals and, as a result, may affect a big portion of the Internet traffic associated to BitTorrent. The latter group is responsible for fake content, which is mostly linked to malicious activity and creates a serious threat for the Bit- Torrent ecosystem and for the Internet in general. To mitigate this threat, in the last part of the thesis we present a new tool named TorrentGuard for the early detection of fake content that could help to significantly reduce the number of computer infections and scams suffered by BitTorrent users. This tool is available through web portal and as a plugin to Vuze, a popular BitTorrent client. Finally, we present MYPROBE, the web portal that allows to query our database and to gather different pieces of information regarding BitTorrent content publishers. ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------BitTorrent es la aplicación peer-to-peer para compartición de ficheros de mayor éxito y responsable de una fracción importante del tráfico de Internet. Trabajos previos han estudiado BitTorrent usando técnicas de simulación, modelos analíticos y medidas reales. Aunque las técnicas analíticas y de simulación son más sencillas de aplicar, típicamente presentan versiones simplificadas de los problemas analizados y en el caso concreto de BitTorrent pueden obviar aspectos o interacciones fundamentales que ocurren en los enjambres de BitTorrent. Por lo tanto, en esta tesis utilizaremos como pilar de nuestra investigación técnicas de medidas reales. En primer lugar presentaremos un resumen de las técnicas de medidas usadas hasta el momento en el ámbito de BitTorrent que suponen la base teórica para el diseño de nuestras propias herramientas de medida que nos permitirán analizar enjambres reales de BitTorrent. Usando los datos obtenidos con estas herramientas estudiaremos aspectos diferentes de BitTorrent con un enfoque especial de los aspectos socioeconómicos. En la primera parte de la tesis, realizaremos un estudio detallado de la topología de los enjambres reales de BitTorrent así como de detalles acerca de las interacciones entre peers. Nuestro análisis demuestra que la resistencia de la topología de los enjambres reales de BitTorrent es menor que la ofrecida por grafos aleatorios equivalentes. Además, los resultados revelan que las políticas de los Provedores de Internet junto con la incipiente utilización de clientes de BitTorrent modificados y otros efectos en la red (p.ej. congestión) hacen que los enjambres reales de BitTorrent presentan una composicin de localidad. Es decir, un nodo tiene un número de vecinos dentro de su mismo Proveedor de Internet mayor del que obtendría en una topología puramente aleatoria. Estos resultados son de interés para las empresas que utilizan BitTorrent en sus operaciones, así como para los Provedores de Internet responsables de transportar el tráfico de BitTorrent. En la segunda parte de la tesis, analizamos los aspectos de publicación de contenido en los mayores portales de BitTorrent. En concreto, los resultados presentados muestran que sólo un pequeño grupo de publicadores (alrededor de 100) es responsable de hacer disponible más de dos tercios del contenido publicado. Además estos publicadores se pueden dividir en dos grupos: (i) aquellos con incentivos económicos y (ii) publicadores de contenido falso. El primer grupo hace disponible contenido protegido por derechos de autor (que es típicamente muy popular) en los principales portales de BitTorrent con el objetivo de atraer a los consumidores de dicho contenido a sus propios sitios web y obtener un beneficio económico. La eliminación de este grupo puede tener un impacto importante en la popularidad de los principales portales de BitTorrent así como en el tráfico generado por BitTorrent en Internet. El segundo grupo es responsable de la publicación de contenidos falsos. La mayor parte de dichos contenidos están asociados a una actividad maliciosa (p.ej. la distribución de software malicioso) y por tanto suponen una seria amenaza para el ecosistema de BitTorrent, en particular, y para Internet en general. Para minimizar los efectos de la amenaza que presentan estos publicadores, en la última parte de la tesis presentaremos una nueva herramienta denominada TorrentGuard para la pronta detección de contenidos falsos. Esta herramienta puede accederse a través de un portal web y a través de un plugin del cliente de BitTorrent Vuze. Finalmente, presentamos MYPROBE, un portal web que permite consultar una base de datos con información actualizada sobre los publicadores de contenidos en BitTorrent

온라인 게임과 컨텐트 공유 네트워크 분석을 통한 온라인 군집 현상의 이해

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2015. 2. 권태경.Quantification of collective human behavior and understanding the group characteristics in the Internet is important in user behavior studies since people tend to gather together and form groups due to their inherent nature. On the Internet, people are also often forming a group for a specific purpose such as i) an online group in games (e.g., MMORPGs) to experience various social interactions with other players or accomplish a difficult quest with teammates or ii) a swarm in peer-to-peer network to share a content to utilize a higher download rate with an availability. To this end, we studied the two most well-known major applications in the Internet that people are actively using with different purposesi) MMORPGs and ii) BitTorrent. In this dissertation, we analyze the i) group activities of users in Aion, one of the largest MMORPGs, based on the records of the activities of 94,497 users and ii) crowd phenomena of BitTorrent. First, in a case study of Aion, we focus on (i) how social interactions within a group differ from the ones across groups, (ii) what makes a group rise, sustain, or fall, (iii) how group members join and leave a group, and (iv) what makes a group end. We first find that structural patterns of social interactions within a group are more likely to be close-knit and reciprocative than the ones across groups.We also observe that members in a rising group (i.e., the number of members increases) are more cohesive, and communicate with more evenly within the group than the ones in other groups. Our analysis further reveals that if a group is not cohesive, not actively communicating, or not evenly communicating among members, members of the group tend to leave. Second, we investigate what kinds of crowd phenomena of content exist and why different patterns of crowd phenomena appears and how we can exploit content crowd phenomena considering the content category, publisher, and population of content in BitTorrent. To this end,We conduct comprehensive measurements on content locality in one of the largest BitTorrent portals: The Pirate Bay. In particular, we focus on (i) how content is consumed from spatial and temporal perspectives, (ii) what makes content be consumed with disparity in spatial and temporal domains, and (iii) how we can exploit the content locality. We find that content consumption in real swarms is 4.56 times and 1.46 times skewed in spatial (country) and temporal (time) domains, respectively. We observe that a cultural factor (e.g., language) mainly affects spatial locality of content. Not only the time-sensitivity of content but also the publishing purpose affects temporal locality of content.We reveal that spatial locality of content iii rarely changes on a daily basis (microscopic level), but there is notably spatial spread of content consumption over the years (macroscopic level). Based on the observation, we conduct simulations to show that bundling and caching can exploit the content locality.Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii I. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Crowd Phenomena in Massively Multi-player Online Role-Playing Games (MMORPGs) . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Crowd Phenomena in BitTorrent . . . . . . . . . . . . . . . . . . . 3 II. RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.1 Crowd Phenomena in MMORPGs . . . . . . . . . . . . . . . . . . 6 2.1.1 Social Interactions in MMORPGs . . . . . . . . . . . . . . 6 2.1.2 Group Activities in MMORPGs . . . . . . . . . . . . . . . 7 2.1.3 Group Activities in Other Online Services . . . . . . . . . . 7 2.2 Crowd Phenomena (Locality) in BitTorrent . . . . . . . . . . . . . 8 2.2.1 Peer Localization . . . . . . . . . . . . . . . . . . . . . . . 8 2.2.2 Crowd Phenomena in BitTorrent . . . . . . . . . . . . . . . 9 2.2.3 Locality in Other Domains . . . . . . . . . . . . . . . . . . 10 III. Group Activities in Online Social Game . . . . . . . . . . . . . . . . 11 3.1 Aion overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.1.1 Game Features . . . . . . . . . . . . . . . . . . . . . . . . 11 3.1.2 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.2 Group Affiliation . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 v 3.2.1 How prevalent are group activities in Aion? . . . . . . . . . 14 3.2.2 Effect of Joining a Group . . . . . . . . . . . . . . . . . . . 16 3.2.3 Social Interactions Within a Group . . . . . . . . . . . . . . 16 3.3 Group Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.3.1 Group Cohesion . . . . . . . . . . . . . . . . . . . . . . . 20 3.3.2 Group Diversity . . . . . . . . . . . . . . . . . . . . . . . . 24 3.3.3 Group Locality . . . . . . . . . . . . . . . . . . . . . . . . 28 3.3.4 Survival Rate . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.3.5 Dichotomy in Stable Groups . . . . . . . . . . . . . . . . . 32 3.4 Group Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.4.1 Properties of the Group Network . . . . . . . . . . . . . . . 36 3.4.2 Structural Holes . . . . . . . . . . . . . . . . . . . . . . . 38 3.5 Implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.5.1 Why people leave groups? . . . . . . . . . . . . . . . . . . 40 3.5.2 Why a group ends? . . . . . . . . . . . . . . . . . . . . . . 42 IV. Crowd phenomena of BitTorrent in Spatial and Temporal Perspective 46 4.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 4.1.1 Discovering Swarm Topology . . . . . . . . . . . . . . . . 46 4.1.2 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 4.1.3 Representativeness . . . . . . . . . . . . . . . . . . . . . . 49 4.2 Spatial Locality . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 4.2.1 Locality Metrics . . . . . . . . . . . . . . . . . . . . . . . 51 4.2.2 Swarm, Community, and Neighbor . . . . . . . . . . . . . 53 vi 4.2.3 Content Categories, Publishers, and Popularity . . . . . . . 55 4.2.4 Spatial Locality Over Time . . . . . . . . . . . . . . . . . . 58 4.3 Temporal Locality . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 4.3.1 Existence of Temporal Locality . . . . . . . . . . . . . . . 61 4.3.2 Categories, Publishers, and Popularity . . . . . . . . . . . . 63 4.3.3 Temporal Usage Trends . . . . . . . . . . . . . . . . . . . 68 4.4 How to Exploit Locality . . . . . . . . . . . . . . . . . . . . . . . 70 V. Summary & Future Work . . . . . . . . . . . . . . . . . . . . . . . . 74 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76Docto

High performance network function virtualization for user-oriented services

The Network Function Virtualization (NFV) paradigm proposes to transform those network functions today running on dedicated and often closed appliances (e.g., firewall, wan accelerator) into pure software images, called Virtual Network Functions (VNFs), which can be consolidated and executed on high-volume standard servers. In this context, this dissertation focuses on the possibility of enabling each single end user (and not only network operators) to set up network services by means of NFV, allowing him to custoimize the set of services that are active on his Internet connection. This goal mainly requires to address flexibility and performance issues. Regarding to the former, it is important: (i) to support services including both network (e.g., firewall) and cloud (e.g., storage server) applications; (ii) to allow the user to define the service with an intuitive and high-level abstraction, hiding infrastructure-layer details. Instead, with respect to performance, multiple software-based services operating on the user's traffic should not introduce penalties in the user’s Internet experience. This dissertation solves the above issues by proposing a number of improvements in the context of Network Function Virtualization, both in terms of high level models and architectures to define and instantiate network services, and in terms of mechanisms to efficiently interconnect VNFs. Experimental results demonstrate that the goal of allowing end users to deploy services operating on their own traffic is feasible without impacting the Internet experience

Static Web content distribution and request routing in a P2P overlay

The significance of collaboration over the Internet has become a corner-stone of modern computing, as the essence of information processing and content management has shifted to networked and Webbased systems. As a result, the effective and reliable access to networked resources has become a critical commodity in any modern infrastructure. In order to cope with the limitations introduced by the traditional client-server networking model, most of the popular Web-based services have employed separate Content Delivery Networks (CDN) to distribute the server-side resource consumption. Since the Web applications are often latency-critical, the CDNs are additionally being adopted for optimizing the content delivery latencies perceived by the Web clients. Because of the prevalent connection model, the Web content delivery has grown to a notable industry. The rapid growth in the amount of mobile devices further contributes to the amount of resources required from the originating server, as the content is also accessible on the go. While the Web has become one of the most utilized sources of information and digital content, the openness of the Internet is simultaneously being reduced by organizations and governments preventing access to any undesired resources. The access to information may be regulated or altered to suit any political interests or organizational benefits, thus conflicting with the initial design principle of an unrestricted and independent information network. This thesis contributes to the development of more efficient and open Internet by combining a feasibility study and a preliminary design of a peer-to-peer based Web content distribution and request routing mechanism. The suggested design addresses both the challenges related to effectiveness of current client-server networking model and the openness of information distributed over the Internet. Based on the properties of existing peer-to-peer implementations, the suggested overlay design is intended to provide low-latency access to any Web content without sacrificing the end-user privacy. The overlay is additionally designed to increase the cost of censorship by forcing a successful blockade to isolate the censored network from the rest of the Internet