Estimating Self-Sustainability in Peer-to-Peer Swarming Systems

Peer-to-peer swarming is one of the \emph{de facto} solutions for distributed content dissemination in today's Internet. By leveraging resources provided by clients, swarming systems reduce the load on and costs to publishers. However, there is a limit to how much cost savings can be gained from swarming; for example, for unpopular content peers will always depend on the publisher in order to complete their downloads. In this paper, we investigate this dependence. For this purpose, we propose a new metric, namely \emph{swarm self-sustainability}. A swarm is referred to as self-sustaining if all its blocks are collectively held by peers; the self-sustainability of a swarm is the fraction of time in which the swarm is self-sustaining. We pose the following question: how does the self-sustainability of a swarm vary as a function of content popularity, the service capacity of the users, and the size of the file? We present a model to answer the posed question. We then propose efficient solution methods to compute self-sustainability. The accuracy of our estimates is validated against simulation. Finally, we also provide closed-form expressions for the fraction of time that a given number of blocks is collectively held by peers.Comment: 27 pages, 5 figure

On the Optimization of BitTorrent-Like Protocols for Interactive On-Demand Streaming Systems

This paper proposes two novel optimized BitTorrent-like protocols for interactive multimedia streaming: the Simple Interactive Streaming Protocol (SISP) and the Exclusive Interactive Streaming Protocol (EISP). The former chiefly seeks a trade-off between playback continuity and data diversity, while the latter is mostly focused on playback continuity. To assure a thorough and up-to-date approach, related work is carefully examined and important open issues, concerning the design of BitTorrent-like algorithms, are analyzed as well. Through simulations, in a variety of near-real file replication scenarios, the novel protocols are evaluated using distinct performance metrics. Among the major findings, the final results show that the two novel proposals are efficient and, besides, focusing on playback continuity ends up being the best design concept to achieve high quality of service. Lastly, avenues for further research are included at the end of this paper as well.Comment: 20 page

BitTorrent 시스템에서 컨텐트 번들링 및 배포

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2013. 2. 최양희.BitTorrent는 컨텐트 공유에 사용되는 가장 인기있는 인터넷 소프트웨어이다. BitTorrent가 널리 사용됨에 따라, 연구자들은 BitTorrent의 처리량, 공정성, 인센티브와 같은 이슈에 대해 연구해 왔고, 이러한 연구들은 BitTorrent 성능과 관련된 가치있는 결과들을 보여주었다. 하지만 대부분의 연구에서는, BitTorrent에서의 컨텐트 번들링 및 배포 전략과 관련해서 (1) BitTorrent 배포자가 파일을 어떤 목적으로 어떻게 번들 하는지와 (2) BitTorrent의 배포자들이 그들의 목적을 성취하기 위해 어떠한 전략들을 사용하는지 등에 대해 다루고 있지 않다. 본 학위 논문에서는, 앞서 언급한 문제들을 측정된 데이터를 바탕으로 조사하기 위해서, BitTorrent 포탈중 가장 큰 규모인 The Pirate Bay (TPB)에 대한 종합적인 측정 연구를 수행하였다. 측정된 데이터셋은 12만개의 토런트와 1600만명의 사용자로 구성되었고, 컨텐트 배포자를 (i) 가짜 배포자, (ii) 이윤추구 배포자, (iii) 이타적 배포자 세가지 종류로 분류하여 연구를 진행하였다. 또한 영화, TV, 성인물, 음악, 응용프로그램, 게임, 전자책과 같은 컨텐트 카테고리에 따라 번들링과 컨테트 배포 현황이 어떻게 되는지 조사하였다. 첫번째로, 토런트의 구조적 패턴과 스왐 참여자의 행동 패턴을 파악하기 위해 컨텐트 번들링과 관련된 현황을 조사하였다. 특별히, (1) 얼마나 컨텐트 번들링이 널리 사용되는가, (2) 어떤 파일들이 어떻게 토런트로 번들되는가, (3) 왜 배포자들이 파일을 번들해서 사용하는가, (4) 사용자들이 번들된 파일들을 어떻게 다운로드 받는가에 초점을 맞추어 연구를 수행하였다. 측정결과 72% 이상의 토런트들이 여러개의 파일로 구성되어 있는 것을 알 수 있었고, 이것은 번들이 BitTorrent의 파일 공유를 위해 널리 사용되고 있음을 보여준다. 그리고 경제적인 이득을 위해 웹사이트를 광고하는 이윤추구 배포자들이 번들을 선호하여 사용하는 경향이 있음을 알 수 있었다. 또한 번들된 토런트의 대부분의 파일(94%)이 사용자들에 의해 선택되고, 번들된 토런트가 번들이 아닌 토런트보다 평균적으로 더 인기가 좋음을 알 수 있었다. 전체적으로, 토런트의 구조적 패턴과 스왐 참여자의 특징은 컨텐트의 카테고리 종류에 따라서, 그리고 번들된 토런트인지 번들되지 않은 토런트인지에 따라서 주목할만한 차이점이 있음을 발견할 수 있었다. 다음으로, 사회경제적 관점에서 BitTorrent의 컨텐트 배포 패턴을 (1) 배포자에 의해서 파일이 어떻게 배포되는가, (2) 각 배포자들은 어떤 전략들을 사용하는가, (3) 배포 전략들이 얼마나 효과가 있는가의 측면에서 조사하였다. 측정결과 상당한 양의 트래픽(61%)이 가짜 토런트를 다운받을 때 발생하고 있는 것을 알 수 있었고, 이는 많은 양의 인터넷 트래픽이 불필요하게 낭비되고 있음을 보여 주는 것이다. 따라서 본 측정 결과로부터 알 수 있는 가짜 배포자들의 배포 패턴을 고려해서 TPB의 가짜 배포자를 걸러낼 수 있는 방법을 제안하였고, 제안된 방법이 전체 다운로드 트래픽의 45% 가량을 줄일 수 있음을 보여 주었다. 또한 이윤추구 배포자들은 그들의 수익모델(예를 들어, 개인 트래커 사이트에 새로운 사용자를 영입하는 것이나 사람들이 사진과 연결된 URL 링크를 클릭하도록 하는 것)에 따라 다른 배포 전략을 이용하고 있음을 알 수 있었다.BitTorrent is one of the most popular applications for sharing contents over the Internet. The huge success of BitTorrent has attracted the research community to investigate BitTorrent's behavior in terms of throughput, fairness, and incentive issues, revealing valuable insights into the performance aspects of BitTorrent. However, most of these studies paid little attention to understand content bundling and publishing strategies in BitTorrent from the following perspectives: (1) how, and for what purposes, are constituent files bundled by BitTorrent publishers? and (2) what strategies are adopted by BitTorrent publishers to achieve their goals? To answer these questions with data from a large-scale BitTorrent system, we conduct comprehensive measurements on one of the largest BitTorrent portals: the Pirate Bay (TPB). From the datasets of the 120 K torrents and 16 M peers, we classify BitTorrent publishers into three types: (i) fake publishers, (ii) profit-driven publishers, and (iii) altruistic publishers. Throughout this dissertation, we investigate the current practice of bundling and publishing across different content categories: Movie, TV, Porn, Music, Application, Game, and E-book. We first investigate the current practice of content bundling to understand the structural patterns of torrents and the participant behaviors of swarms. In particular, we focus on: (1) how prevalent content bundling is, (2) how and what files are bundled into torrents, (3) what motivates publishers to bundle files, and (4) how peers access the bundled files. We find that over 72% of BitTorrent torrents contain multiple files, which indicates that bundling is widely used for file sharing. We reveal that profit-driven BitTorrent publishers who promote their own web sites for financial gains like advertising tend to prefer to use the bundling. We also observe that most files (94%) in a bundle torrent are selected by users and the bundle torrents are more popular than the single (or non-bundle) ones on average. Overall, there are notable differences in the structural patterns of torrents and swarm characteristics (i) across different content categories and (ii) between single and bundle torrents. We next investigate the current practice of content publishing in BitTorrent from a socio-economic point of view, by unraveling (1) how files are published by publishers, (2) what strategies are adopted by publishers, and (3) how effective those strategies are. We show that a significant amount of traffic (61%) of BitTorrent has been generated (i.e., unnecessarily wasted) to download fake torrents. Therefore, we suggest a method to filter out fake publishers on TPB by considering their distinct publishing patterns learned from our measurement study, and show that the proposed method can reduce around 45% of the total download traffic. We also reveal that profit-driven publishers adopt different publishing strategies according to their revenue models (e.g., advertising private tracker sites to attract potential new members, or exposing image URLs to make people click the URL links).Abstract i I. Introduction 1 II. Related Work 5 2.1 Multi-torrent Systems 5 2.2 Bundling in BitTorrent 6 2.3 Bundling in Economics 7 2.4 Content publishing in BitTorrent 7 III. Methodology 9 3.1 Measurement Methodology 9 3.2 Publisher Classification 11 IV. Bundling Practice in BitTorrent: What, How, and Why 14 4.1 Introduction 14 4.2 Datasets 16 4.2.1 Torrent Datasets 17 4.2.2 Swarm Datasets 17 4.3 Single vs. Bundle 18 4.3.1 Bundling is widespread 18 4.3.2 How files are bundled 20 4.4 Main File Analysis in Bundling 27 4.4.1 Identifying Main Files 28 4.4.2 Constituents of Bundle-k 29 4.5 Publisher Analysis 32 4.5.1 Contribution of Top-20 Publishers 33 4.5.2 Cross-category Publishing of Top-20 Publishers 39 4.6 User Access Pattern Analysis 40 4.6.1 Popularity Analysis 40 4.6.2 Availability Analysis 43 4.6.3 The Number of Files Requested by Users in a Bundle Torrent 44 4.6.4 Swarm Behaviors versus Bundle-k 47 4.7 Discussions 50 V. Content Publishing Practice in BitTorrent 52 5.1 Introduction 52 5.2 The Number of Published Torrents 54 5.3 Publishers Strategies 58 5.3.1 Lifetime of Publishers and their Publishing Rates 59 5.3.2 Content Categories 60 5.3.3 Advertising Strategies of Profit-driven Publishers 63 5.4 Downloaders Behavior 64 5.5 Implications on Publishers Strategies 69 5.5.1 Fake Publishers 69 5.5.2 Profit-driven Publishers 71 VI. Summary & Future Work 73 Bibliography 75 Korean Abstract 80Docto

Incentive-driven QoS in peer-to-peer overlays

A well known problem in peer-to-peer overlays is that no single entity has control over the software, hardware and configuration of peers. Thus, each peer can selfishly adapt its behaviour to maximise its benefit from the overlay. This thesis is concerned with the modelling and design of incentive mechanisms for QoS-overlays: resource allocation protocols that provide strategic peers with participation incentives, while at the same time optimising the performance of the peer-to-peer distribution overlay. The contributions of this thesis are as follows. First, we present PledgeRoute, a novel contribution accounting system that can be used, along with a set of reciprocity policies, as an incentive mechanism to encourage peers to contribute resources even when users are not actively consuming overlay services. This mechanism uses a decentralised credit network, is resilient to sybil attacks, and allows peers to achieve time and space deferred contribution reciprocity. Then, we present a novel, QoS-aware resource allocation model based on Vickrey auctions that uses PledgeRoute as a substrate. It acts as an incentive mechanism by providing efficient overlay construction, while at the same time allocating increasing service quality to those peers that contribute more to the network. The model is then applied to lagsensitive chunk swarming, and some of its properties are explored for different peer delay distributions. When considering QoS overlays deployed over the best-effort Internet, the quality received by a client cannot be adjudicated completely to either its serving peer or the intervening network between them. By drawing parallels between this situation and well-known hidden action situations in microeconomics, we propose a novel scheme to ensure adherence to advertised QoS levels. We then apply it to delay-sensitive chunk distribution overlays and present the optimal contract payments required, along with a method for QoS contract enforcement through reciprocative strategies. We also present a probabilistic model for application-layer delay as a function of the prevailing network conditions. Finally, we address the incentives of managed overlays, and the prediction of their behaviour. We propose two novel models of multihoming managed overlay incentives in which overlays can freely allocate their traffic flows between different ISPs. One is obtained by optimising an overlay utility function with desired properties, while the other is designed for data-driven least-squares fitting of the cross elasticity of demand. This last model is then used to solve for ISP profit maximisation

Experimental analysis of the socio-economic phenomena in the BitTorrent ecosystem

BitTorrent is the most successful Peer-to-Peer (P2P) application and is responsible for a major portion of Internet traffic. It has been largely studied using simulations, models and real measurements. Although simulations and modelling are easier to perform, they typically simplify analysed problems and in case of BitTorrent they are likely to miss some of the effects which occur in real swarms. Thus, in this thesis we rely on real measurements. In the first part of the thesis we present the summary of measurement techniques used so far and we use it as a base to design our tools that allow us to perform different types of analysis at different resolution level. Using these tools we collect several large-scale datasets to study different aspects of BitTorrent with a special focus on socio-economic aspects. Using our datasets, we first investigate the topology of real BitTorrent swarms and how the traffic is actually exchanged among peers. Our analysis shows that the resilience of BitTorrent swarms is lower than corresponding random graphs. We also observe that ISP policies, locality-aware clients and network events (e.g., network congestion) lead to locality-biased composition of neighbourhood in the swarms. This means that the peer contains more neighbours from local provider than expected from purely random neighbours selection process. Those results are of interest to the companies which use BitTorrent for daily operations as well as for ISPs which carry BitTorrent traffic. In the next part of the thesis we look at the BitTorrent from the perspective of the content and content publishers in a major BitTorrent portals. We focus on the factors that seem to drive the popularity of the BitTorrent and, as a result, could affect its associated traffic in the Internet. We show that a small fraction of publishers (around 100 users) is responsible for more than two-thirds of the published content. Those publishers can be divided into two groups: (i) profit driven and (ii)fake publishers. The former group leverages the published copyrighted content (typically very popular) on BitTorrent portals to attract content consumers to their web sites for financial gain. Removing this group may have a significant impact on the popularity of BitTorrent portals and, as a result, may affect a big portion of the Internet traffic associated to BitTorrent. The latter group is responsible for fake content, which is mostly linked to malicious activity and creates a serious threat for the Bit- Torrent ecosystem and for the Internet in general. To mitigate this threat, in the last part of the thesis we present a new tool named TorrentGuard for the early detection of fake content that could help to significantly reduce the number of computer infections and scams suffered by BitTorrent users. This tool is available through web portal and as a plugin to Vuze, a popular BitTorrent client. Finally, we present MYPROBE, the web portal that allows to query our database and to gather different pieces of information regarding BitTorrent content publishers. ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------BitTorrent es la aplicación peer-to-peer para compartición de ficheros de mayor éxito y responsable de una fracción importante del tráfico de Internet. Trabajos previos han estudiado BitTorrent usando técnicas de simulación, modelos analíticos y medidas reales. Aunque las técnicas analíticas y de simulación son más sencillas de aplicar, típicamente presentan versiones simplificadas de los problemas analizados y en el caso concreto de BitTorrent pueden obviar aspectos o interacciones fundamentales que ocurren en los enjambres de BitTorrent. Por lo tanto, en esta tesis utilizaremos como pilar de nuestra investigación técnicas de medidas reales. En primer lugar presentaremos un resumen de las técnicas de medidas usadas hasta el momento en el ámbito de BitTorrent que suponen la base teórica para el diseño de nuestras propias herramientas de medida que nos permitirán analizar enjambres reales de BitTorrent. Usando los datos obtenidos con estas herramientas estudiaremos aspectos diferentes de BitTorrent con un enfoque especial de los aspectos socioeconómicos. En la primera parte de la tesis, realizaremos un estudio detallado de la topología de los enjambres reales de BitTorrent así como de detalles acerca de las interacciones entre peers. Nuestro análisis demuestra que la resistencia de la topología de los enjambres reales de BitTorrent es menor que la ofrecida por grafos aleatorios equivalentes. Además, los resultados revelan que las políticas de los Provedores de Internet junto con la incipiente utilización de clientes de BitTorrent modificados y otros efectos en la red (p.ej. congestión) hacen que los enjambres reales de BitTorrent presentan una composicin de localidad. Es decir, un nodo tiene un número de vecinos dentro de su mismo Proveedor de Internet mayor del que obtendría en una topología puramente aleatoria. Estos resultados son de interés para las empresas que utilizan BitTorrent en sus operaciones, así como para los Provedores de Internet responsables de transportar el tráfico de BitTorrent. En la segunda parte de la tesis, analizamos los aspectos de publicación de contenido en los mayores portales de BitTorrent. En concreto, los resultados presentados muestran que sólo un pequeño grupo de publicadores (alrededor de 100) es responsable de hacer disponible más de dos tercios del contenido publicado. Además estos publicadores se pueden dividir en dos grupos: (i) aquellos con incentivos económicos y (ii) publicadores de contenido falso. El primer grupo hace disponible contenido protegido por derechos de autor (que es típicamente muy popular) en los principales portales de BitTorrent con el objetivo de atraer a los consumidores de dicho contenido a sus propios sitios web y obtener un beneficio económico. La eliminación de este grupo puede tener un impacto importante en la popularidad de los principales portales de BitTorrent así como en el tráfico generado por BitTorrent en Internet. El segundo grupo es responsable de la publicación de contenidos falsos. La mayor parte de dichos contenidos están asociados a una actividad maliciosa (p.ej. la distribución de software malicioso) y por tanto suponen una seria amenaza para el ecosistema de BitTorrent, en particular, y para Internet en general. Para minimizar los efectos de la amenaza que presentan estos publicadores, en la última parte de la tesis presentaremos una nueva herramienta denominada TorrentGuard para la pronta detección de contenidos falsos. Esta herramienta puede accederse a través de un portal web y a través de un plugin del cliente de BitTorrent Vuze. Finalmente, presentamos MYPROBE, un portal web que permite consultar una base de datos con información actualizada sobre los publicadores de contenidos en BitTorrent

Mathematical analysis of scheduling policies in peer-to-peer video streaming networks

Las redes de pares son comunidades virtuales autogestionadas, desarrolladas en la capa de aplicación sobre la infraestructura de Internet, donde los usuarios (denominados pares) comparten recursos (ancho de banda, memoria, procesamiento) para alcanzar un fin común. La distribución de video representa la aplicación más desafiante, dadas las limitaciones de ancho de banda. Existen básicamente tres servicios de video. El más simple es la descarga, donde un conjunto de servidores posee el contenido original, y los usuarios deben descargar completamente este contenido previo a su reproducción. Un segundo servicio se denomina video bajo demanda, donde los pares se unen a una red virtual siempre que inicien una solicitud de un contenido de video, e inician una descarga progresiva en línea. El último servicio es video en vivo, donde el contenido de video es generado, distribuido y visualizado simultáneamente. En esta tesis se estudian aspectos de diseño para la distribución de video en vivo y bajo demanda. Se presenta un análisis matemático de estabilidad y capacidad de arquitecturas de distribución bajo demanda híbridas, asistidas por pares. Los pares inician descargas concurrentes de múltiples contenidos, y se desconectan cuando lo desean. Se predice la evolución esperada del sistema asumiendo proceso Poisson de arribos y egresos exponenciales, mediante un modelo determinístico de fluidos. Un sub-modelo de descargas secuenciales (no simultáneas) es globalmente y estructuralmente estable, independientemente de los parámetros de la red. Mediante la Ley de Little se determina el tiempo medio de residencia de usuarios en un sistema bajo demanda secuencial estacionario. Se demuestra teóricamente que la filosofía híbrida de cooperación entre pares siempre desempeña mejor que la tecnología pura basada en cliente-servidor

On the optimization of bittorrent-like protocols for interactive on-demand streaming systems

This paper proposes two novel optimized BitTorrent-like protocols for interactive multimedia streaming: the Simple Interactive Streaming Protocol (SISP) and the Exclusive Interactive Streaming Protocol (EISP). The former chiefly seeks a trade-off between playback continuity and data diversity, while the latter is mostly focused on playback continuity. To assure a thorough and up-to-date approach, related work is carefully examined and important open issues, concerning the design of BitTorrent-like algorithms, are analyzed as well. Through simulations, in a variety of near-real file replication scenarios, the novel protocols are evaluated using distinct performance metrics. Among the major findings, the final results show that the two novel proposals are efficient and, besides, focusing on playback continuity ends up being the best design concept to achieve high quality of service. Lastly, avenues for further research are included at the end of this paper as well