National audiencePouvoir prédire la bande passante disponible est une problématique cruciale pour un grand nombre d'applications distribuées sur Internet. Plusieurs solutions ont été proposées, mais l'absence d'implémentations communes et de jeux de données reconnus rend difficile la comparaison et la reproductibilité des résultats. Dans cet article, nous présentons bedibe, la combinaison de mesures de bande passante effectuées sur Planet-Lab et d'un logiciel pour faciliter l'écriture et l'étude d'algorithmes pour la prédiction de bande passante. bedibe inclut les implémentations des meilleures solutions de la littérature, et a pour but de faciliter la comparaison des résultats obtenus par les différentes équipes qui travaillent sur ce thème

Eyraud-Dubois, Lionel

Uznanski, Przemyslaw

INRIA a CCSD electronic archive server

HAL Id: hal-00690861https://hal.inria.fr/hal-00690861Submitted on 24 Apr 2012HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.Bedibe: Datasets and Software Tools for DistributedBandwidth PredictionLionel Eyraud-Dubois, Przemyslaw UznanskiTo cite this version:Lionel Eyraud-Dubois, Przemyslaw Uznanski. Bedibe: Datasets and Software Tools for DistributedBandwidth Prediction. AlgoTel’2012 - Rencontres Francophones sur les Aspects Algorithmiques desTélécommunications, May 2012, La Grande Motte, France. ￿hal-00690861￿Bedibe: Datasets and Software Tools forDistributed Bandwidth PredictionLionel Eyraud-Dubois1 † and Przemysław Uznański11INRIA Bordeaux Sud-Ouest F-78000 and LaBRI - Univ of Bordeaux F-33400 and LaBRI - CNRS F-33400Pouvoir prédire la bande passante disponible est une problématique cruciale pour un grand nombre d’applicationsdistribuées sur Internet. Plusieurs solutions ont été proposées, mais l’absence d’implémentations communes et dejeux de données reconnus rend difficile la comparaison et la reproductibilité des résultats. Dans cet article, nousprésentons bedibe, la combinaison de mesures de bande passante effectuées sur Planet-Lab et d’un logiciel pourfaciliter l’écriture et l’étude d’algorithmes pour la prédiction de bande passante. bedibe inclut les implémentationsdes meilleures solutions de la littérature, et a pour but de faciliter la comparaison des résultats obtenus par les différenteséquipes qui travaillent sur ce thème.Keywords: available bandwidth prediction, network coordinates systems, distributed systems, planet lab, open science,network measurement1 IntroductionPredicting network performance (latency or available bandwidth) is important for many Internet applica-tions. For video on demand [SDK+07] and peer-assisted streaming [LZSJ+08] for example, estimations ofavailable bandwidth allow the construction of an efficient overlay topology.A number of measurement tools have been developed [GS10] which measure the available bandwidthon the path between two given Internet nodes. However, in a large scale system, performing measurementsbetween all pairs of nodes would incur too much overhead. Thus, there is a need for the possibility to predictthe unmeasured bandwidth values from a limited number of actual available measurements.For latency estimation, several solutions have been successfully proposed (like GNP [NZ02], Vivaldi [DCKM04],and others), under the global terminology of Network Coordinate Systems. Available bandwidth estimationhas been recently considered as well. A number of solutions have been proposed, and there is at the momentno clear consensus about which solutions are most efficient. Because bandwidth measurements are moreexpensive, few datasets are available to compare the solutions. This paper presents the bedibe project,which regroups a software for “BEnchmarking DIstributed Bandwidth Estimation”, and a set of availablebandwidth measurements performed on the Planet-Lab platform using SPLAY [LRF09]. The common goalsof both contributions is to help promote the diffusion of results in the community, providing a framework tohelp design and compare tools for available bandwidth estimation, thus allowing to easily reproduce resultsobtained by others.Related works Several bandwidth prediction algorithms have been proposed in the literature. Many havefocused on embedding the nodes into a metric space, more precisely a hyperbolic structure [Din08]. Anexample of such a system is the Sequoia algorithm [RMK+09], which embeds the nodes as the leaves of aweighted tree, and uses distances in the tree as approximations of actual distances. The Sequoia algorithmcomes with a theoretically proved performance guarantee, and can be applied to both latency and band-width estimation. However, the algorithm is quite sensitive to violations of triangular inequalities, whichare common in the Internet. Sequoia has later been improved with a decentralized version [SKBS11], inorder to lower the load of the root of the tree, and also to make use of a smaller number of measurements.†This work is partially supported by the ANR SONGS project.Lionel Eyraud-Dubois and Przemysław UznańskiTo cope with triangular inequality violations, several studies have thus considered non-metric embed-dings. IDES [MSS06] uses matrix factorization to approximate the large measurement matrix by the productof two smaller matrices. The IDES system is based on a set of landmark nodes, and recently a decentralizedversion has been proposed, called DMF [LGL10] for decentralized matrix factorization. DMF is an itera-tive procedure in which each node locally minimizes the prediction error by solving a least square problem.Phoenix [CWS+11] improves over DMF by adding features which are commonly found in NCS to copewith node churn and to add more credibility to nodes that have stayed in the system for a long time.Last-mile [BEDW11] is a prediction mechanism based on the assumption that available bandwidth on apath is limited only by the the first and last links of the path. In this model, each node is characterized byits incoming and outgoing bandwidth. Despite the simplicity of this model, a procedure to compute thesevalues has been proposed which achieves reasonable accuracy.Other works include PathGuru [XCY09] which embeds nodes in an ultra-metric space but provides ratherpoor estimates, and BRoute [HS05] which requires network management tools (such as traceroute and BGProuting information) to identify the bottleneck links near each source and destination.2 Bandwidth measurements on Planet-LabContext The S-cube project from HP [YSB+06] was aimed at monitoring the large scale distributedplatform Planet-Lab ‡, and in particular provided available bandwidth measurements between pairs of nodesof this platform, obtained with the Spruce tool. These datasets have been used by the community to validatebandwidth estimation tools. However, this project is now stopped, and only the last snapshot is available.Planet-Lab is a large-scale, worldwide distributed platform which provides an access to nodes on morethan 500 sites across the world. It has become a standard for conducting large scale Internet experiments,and is thus well suited for our purpose. Although it is mainly based on academic networks and thus notrepresentative of the global Internet, measurements on this platform are very valuable for designing soundexperiments on Planet-Lab. Furthermore, its accessibility makes it relatively easy to conduct the requiredmeasurements.SPLAY [LRF09] § is a middleware and a development framework which aims at simplifying the pro-totyping and development of large scale distributed applications and overlay networks. It is based on Luaand provides tools for deploying and controlling a distributed application on a large platform. SPLAY alsoprovides an RPC mechanism for communications between nodes.Applicative measurements A number of tools have been developed for available bandwidth measure-ments [GS10]. Generally speaking, they rely on sending a few packets along the path, and analyze theeffects of intermediate nodes and cross traffic on these probe packets. Although they do not require privi-lege access, these tools require a fine grain access to the network.To simplify the deployment and remain within the SPLAY framework, we decided to perform applicative-level measurements: we measure the performance of the network as it is effectively available to the SPLAYapplication. To this end, we measured the time required to perform an RPC call with a reasonably largepayload (256KB of data) between all pairs of nodes. This measurement allows to take into account allthe aspects that impact communication performance for the application, and we see it as an interestingcomplement to low-level measurement.To perform our measurements, we select about 100 of the most responsive nodes from Planet-Lab, andeach of them simultaneously contacts a random participant to request a measurement. A distributed syn-chronization mechanism ensures that no two measurements are performed on the same host at the sametime. However, two independent measures (concerning different sources and destinations) take place in par-allel. This allows to perform one round of measures (between 140 nodes) in about 1 hour, including thetimeouts when nodes respond slowly or fail during the procedure.Description of the datasets We have performed the measurements every day for a week in November andDecember 2011, and in February 2012. Each of these datasets involves a hundred nodes. The large overlap‡. http://www.planet-lab.org§. http://www.splay-project.orgBedibe: Datasets and Software Tools for Distributed Bandwidth Prediction 0 0.2 0.4 0.6 0.8 1 1  1.5  2  2.5  3  3.5  4  4.5  5Proportion of pairs having error < eError eLMLM, k=10DMFDMF, k=10Sequoia, 10 treesFigure 1: Comparison of heuristics on a singledataset 0 0.2 0.4 0.6 0.8 1 1  1.5  2  2.5  3  3.5  4  4.5  5Proportion of pairs having error < eError eLM, k=10DMF, k=10Median of 5 datasetsDataset BFigure 2: Prediction capability of previousdatasets.between participating nodes for all these measurements allow to see the evolution of network performanceover time.We are currently working on improving our measurement procedure, to be able to use low-levelmeasurement tools in addition to our application-level measures. This will ensure that the datasets producedare as comprehensive as possible.3 The bedibe softwareBedibe is a tool for benchmarking bandwidth estimations. Its purpose is the development, testing,benchmarking and visualization of bandwidth estimation algorithms. It is written in Python, a very popularscripting language. It can be downloaded at http://bedibe.gforge.inria.fr/.Data is read from CSV files (Comma Separated Values), with source and target hostnames, and measuredvalues (this is also the format of SPLAY measures). Because network measures have high variation, datasetsoften contain multiple values per pair of nodes. Because of this, we provide a set of tools for easy pickingrepresentative value or set of values (in case of fuzzy computation) from several of them. We provide imple-mentations of the state-of-the art bandwidth estimation algorithms: DMF, LastMile, Sequoia, DSequoia. Wealso provide a library of decorators (function transformation) that for example, allow to transform functionsthat operate on matrices into ones working in our enviroment of CSV style data. Other decorators alloweasy control over what type of preprocessing an algorithm supports.Estimations done by algorithms can be stored into CSV files, or used in several comparison tools. Ourmodule for comparing allows easy comparison of the error ratio between different measures or estima-tions, or calculation of various standard metrics (stress, 80th percentile error. . .). Visualization tools allowto create images out of the computed matrices (assigning different colors to high/low bandwidth, or ac-curate/inaccurate estimation). Additionally, a plotting module can be used to produce (using gnuplot)different types of plots, for example a CDF of the dataset, or a CDF of relative error of the estimates.Various parameters of the plots can also be customized from inside the code.In summary, a programmer simply needs to write an implementation of his algorithm, and a few extralines of code allows the algorithm to be tested on real life data, tweaked, analyzed or compared to other algo-rithms from the literature. Finally, the tool can be used to output plots useful for performance visualizationin a scientific paper.Some results Figures 1 and 2 exhibit an example of the comparisons that our tool allows with minimaleffort. On Figure 1, we plot the CDF of the error ratio (ie, max(mp,pm) = 1+ |m−p|minm,pwhere p is the predictionand m is the measured value) for 5 estimation heuristics: DMF, Last-mile, Sequoia, all three with access tothe complete set of measurements, and also DMF and Last-mile restricted to only k = 10 measurements pernode. On Figure 2, we analyze the prediction capability of a previous dataset B to predict the performanceof the next dataset A. We also plot how well DMF and Last-Mile can predict the performance of dataset Awhen given dataset B as input. These results were obtained with very little coding effort thanks to the largepanel of helper functions in the bedibe tool.Lionel Eyraud-Dubois and Przemysław Uznański4 Concluding remarksEstimating the available bandwidth between nodes in a large scale distributed platform is a crucial issuein many distributed applications. Several solutions have been proposed to perform this prediction whileusing as few measurements as possible. Validating and comparing these solutions requires that their imple-mentation are available, preferably in a common framework. It is also necessary to use common datasets.In this paper, we present bedibe, a Python framework to simplify the design, implementation andvalidation of bandwidth estimation algorithms. We also present datasets obtained from application-levelmeasurements on the Planet-Lab platform using the SPLAY middleware. We hope that these tools will helpthe community to propose efficient solutions for bandwidth estimation, and will promote the diffusion ofknowledge in this field.In the future work, we plan to continue the development of the tool to increase further its usability.We also plan to make the list of featured algorithms as complete as possible. Last but not least, we planto include several of the existing low-level measurement tools [GS10] in our measurement procedure, sothat bedibe will feature the main solutions for available bandwidth measurement and prediction from theliterature.References[BEDW11] O. Beaumont, L. Eyraud-Dubois, and Y. Won. Using the last-mile model as a distributed scheme for available bandwidthprediction. In Proceedings of the EuroPar 2011 conference, 2011.[CWS+11] Y. Chen, X. Wang, C. Shi, E. K. Lua, X. Fu, B. Deng, and X. Li. Phoenix: A weight-based network coordinate systemusing matrix factorization. IEEE Transactions on Network and Service Management, 8(4):334–347, December 2011.[DCKM04] F. Dabek, R. Cox, F. Kaashoek, and R. Morris. Vivaldi: a decentralized network coordinate system. In ACM SIGCOMM’04, pages 15–26, Portland, OR, USA, Sept. 2004.[Din08] M. Dinitz. Online, dynamic, and distributed embeddings of approximate ultrametrics. In Distributed Computing ’08,volume 5218 of LNCS, pages 152–166. 2008.[GS10] E. Goldoni and M. Schivi. End-to-end available bandwidth estimation tools, an experimental comparison. In TrafficMonitoring and Analysis, volume 6003 of LNCS, pages 171–182. 2010.[HS05] N. Hu and P. Steenkiste. Exploiting internet route sharing for large scale available bandwidth estimation. In IMC ’05,pages 16–16, Oct. 2005.[LGL10] Y. Liao, P. Geurts, and G. Leduc. Network distance prediction based on decentralized matrix factorization. In IFIPNETWORKING ’10, pages 15–26, May 2010.[LRF09] L. Leonini, É. Rivière, and P. Felber. Splay: distributed systems evaluation made simple. In Proceedings of the 6thUSENIX symposium on Networked systems design and implementation, NSDI’09, pages 185–198, Berkeley, CA, USA,2009. USENIX Association.[LZSJ+08] S. Liu, R. Zhang-Shen, W. Jiang, J. Rexford, and M. Chiang. Performance bounds for peer-assisted live streaming. ACMSIGMETRICS Perform. Eval. Rev., 36:313–324, June 2008.[MSS06] Y. Mao, L.K. Saul, and J.M. Smith. Ides: An internet distance estimation service for large networks. IEEE JSAC,24(12):2273 –2284, Dec. 2006.[NZ02] T.S.E. Ng and H. Zhang. Predicting internet network distance with coordinates-based approaches. In IEEE INFOCOM’02, pages 170–179, June 2002.[RMK+09] V. Ramasubramanian, D. Malkhi, F. Kuhn, M. Balakrishnan, A. Gupta, and A. Akella. On the treeness of internet latencyand bandwidth. In ACM SIGMETRICS ’09, pages 61–72, Seattle, WA, USA, June 2009.[SDK+07] K. Suh, C. Diot, J. Kurose, L. Massoulie, C. Neumann, D. Towsley, and M. Varvello. Push-to-peer video-on-demandsystem: Design and evaluation. IEEE JSAC, 25(9):1706 –1716, Dec. 2007.[SKBS11] S. Song, P. Keleher, B. Bhattacharjee, and A. Sussman. Decentralized, accurate, and low-cost network bandwidth predic-tion. In INFOCOM, 2011 Proceedings IEEE, pages 6 –10, april 2011.[XCY09] C. Xing, M. Chen, and L. Yan. Predicting available bandwidth of internet path with ultra metric space-based approaches.In IEEE GLOBECOM ’09, Dec. 2009.[YSB+06] P. Yalagandula, P. Sharma, S. Banerjee, S. Basu, and S.-J. Lee. S3: a scalable sensing service for monitoring largenetworked systems. In ACM SIGCOMM workshop on Internet network management, pages 71–76, Pisa, Italy, Sept.2006.

Bedibe: Datasets and Software Tools for Distributed Bandwidth Prediction

Abstract

Similar works

Full text

Available Versions

INRIA a CCSD electronic archive server

HAL-Rennes 1