The Quest for Bandwidth Estimation Techniques for large-scale Distributed Systems

In recent years the research community has developed many techniques to estimate the end-to-end available bandwidth of an Internet path. This important metric has been proposed for use in several distributed systems and, more recently, has even been considered to improve the congestion control mechanism of TCP. Thus, it has been suggested that some existing estimation techniques could be used for this purpose. However, existing tools were not designed for large-scale deployments and were mostly validated in controlled settings, considering only one measurement running at a time. In this paper, we argue that current tools, while offering good estimates when used alone, might not work in large-scale systems where several estimations severely interfere with each other. We analyze the properties of the measurement paradigms employed today and discuss their functioning, study their overhead and analyze their interference. Our testbed results show that current techniques are insufficient as they are. Finally, we will discuss and propose some principles that should be taken into account for including available bandwidth measurements in large-scale distributed systems. 1

Large scale probabilistic available bandwidth estimation

The common utilization-based definition of available bandwidth and many of the existing tools to estimate it suffer from several important weaknesses: i) most tools report a point estimate of average available bandwidth over a measurement interval and do not provide a confidence interval; ii) the commonly adopted models used to relate the available bandwidth metric to the measured data are invalid in almost all practical scenarios; iii) existing tools do not scale well and are not suited to the task of multi-path estimation in large-scale networks; iv) almost all tools use ad-hoc techniques to address measurement noise; and v) tools do not provide enough flexibility in terms of accuracy, overhead, latency and reliability to adapt to the requirements of various applications. In this paper we propose a new definition for available bandwidth and a novel framework that addresses these issues. We define probabilistic available bandwidth (PAB) as the largest input rate at which we can send a traffic flow along a path while achieving, with specified probability, an output rate that is almost as large as the input rate. PAB is expressed directly in terms of the measurable output rate and includes adjustable parameters that allow the user to adapt to different application requirements. Our probabilistic framework to estimate network-wide probabilistic available bandwidth is based on packet trains, Bayesian inference, factor graphs and active sampling. We deploy our tool on the PlanetLab network and our results show that we can obtain accurate estimates with a much smaller measurement overhead compared to existing approaches.Comment: Submitted to Computer Network

Hop-by-hop routing in wireless mesh networks with bandwidth guarantees

Wireless Mesh Network (WMN) has become an important edge network to provide Internet access to remote areas and wireless connections in a metropolitan scale. In this paper, we study the problem of identifying the maximum available bandwidth path, a fundamental issue in supporting quality-of-service in WMNs. Due to interference among links, bandwidth, a well-known bottleneck metric in wired networks, is neither concave nor additive in wireless networks. We propose a new path weight which captures the available path bandwidth information. We formally prove that our hop-by-hop routing protocol based on the new path weight satisfies the consistency and loop-freeness requirements. The consistency property guarantees that each node makes a proper packet forwarding decision, so that a data packet does traverse over the intended path. Our extensive simulation experiments also show that our proposed path weight outperforms existing path metrics in identifying high-throughput paths. © 2012 IEEE.published_or_final_versio

A passive available bandwidth estimation methodology

The Available Bandwidth (AB) of an end-to-end path is its remaining capacity and it is an important metric for several applications such as overlay routing and P2P networking. That is why many AB estimation tools have been published recently. Most of these tools use the Probe Rate Model, which requires sending packet trains at a rate matching the AB. Its main issue is that it congests the path under measurement. We present a different approach: a novel passive methodology to estimate the AB that does not introduce probe traffic. Our methodology, intended to be applied between two separate nodes, estimates the path’s AB by analyzing specific parameters of the traffic exchanged. The main challenge is that we cannot rely on any given rate of this traffic. Therefore we rely on a different model, the Utilization Model. In this paper we present our passive methodology and a tool (PKBest) based on it. We evaluate its applicability and accuracy using public NLANR data traces. Our results -more than 300Gb- show that our tool is more accurate than pathChirp, a state-of-the-art active PRM-based tool. At the best of the authors’ knowledge this is the first passive AB estimation methodology.Preprin

Performance Analysis of a DEKF for Available Bandwidth Measurement

The paper presents a characterisation analysis of a measurement algorithm based on a Discrete-time Extended Kalman Filter (DEKF), which has recently been proposed for the estimation and tracking of end-to-end available bandwidth. The analysis is carried out by means of simulations for different rates of variations of the available bandwidth and permits assessing the performance of the measurement algorithm for different values of the filter parameters, that is, the covariance matrixes of the measurement and process noise

End-to-End Available Bandwidth Estimation Tools, An Experimental Comparison

Abstract. The available bandwidth of a network path impacts the per-formance of many applications, such as VoIP calls, video streaming and P2P content distribution systems. Several tools for bandwidth estimation have been proposed in the last years but there is still uncertainty in their accuracy and efficiency under different network conditions. Although a number of experimental evaluations have been carried out in order to compare some of these methods, a comprehensive evaluation of all the existing active tools for available bandwidth estimation is still missing. This article introduces an empirical comparison of most of the active esti-mation tools actually implemented and freely available nowadays. Abing, ASSOLO, DietTopp, IGI, pathChirp, Pathload, PTR, Spruce and Yaz have been compared in a controlled environment and in presence of dif-ferent sources of cross-traffic. The performance of each tool has been investigated in terms of accuracy, time and traffic injected into the net-work to perform an estimation.

Rate Control State-of-the-art Survey

The majority of Internet traffic use Transmission Control Protocol (TCP) as the transport level protocol. It provides a reliable ordered byte stream for the applications. However, applications such as live video streaming place an emphasis on timeliness over reliability. Also a smooth sending rate can be desirable over sharp changes in the sending rate. For these applications TCP is not necessarily suitable. Rate control attempts to address the demands of these applications. An important design feature in all rate control mechanisms is TCP friendliness. We should not negatively impact TCP performance since it is still the dominant protocol. Rate Control mechanisms are classified into two different mechanisms: window-based mechanisms and rate-based mechanisms. Window-based mechanisms increase their sending rate after a successful transfer of a window of packets similar to TCP. They typically decrease their sending rate sharply after a packet loss. Rate-based solutions control their sending rate in some other way. A large subset of rate-based solutions are called equation-based solutions. Equation-based solutions have a control equation which provides an allowed sending rate. Typically these rate-based solutions react slower to both packet losses and increases in available bandwidth making their sending rate smoother than that of window-based solutions. This report contains a survey of rate control mechanisms and a discussion of their relative strengths and weaknesses. A section is dedicated to a discussion on the enhancements in wireless environments. Another topic in the report is bandwidth estimation. Bandwidth estimation is divided into capacity estimation and available bandwidth estimation. We describe techniques that enable the calculation of a fair sending rate that can be used to create novel rate control mechanisms.Peer reviewe

Performance Analysis of a DEKF for Available Bandwidth Measurement

The paper presents a characterisation analysis of a measurement algorithm based on a Discrete-time Extended Kalman Filter (DEKF), which has recently been proposed for the estimation and tracking of end-to-end available bandwidth. The analysis is carried out by means of simulations for different rates of variations of the available bandwidth and permits assessing the performance of the measurement algorithm for different values of the filter parameters, that is, the covariance matrixes of the measurement and process noise

Mobile Ad-Hoc Networks

Being infrastructure-less and without central administration control, wireless ad-hoc networking is playing a more and more important role in extending the coverage of traditional wireless infrastructure (cellular networks, wireless LAN, etc). This book includes state-of-the-art techniques and solutions for wireless ad-hoc networks. It focuses on the following topics in ad-hoc networks: quality-of-service and video communication, routing protocol and cross-layer design. A few interesting problems about security and delay-tolerant networks are also discussed. This book is targeted to provide network engineers and researchers with design guidelines for large scale wireless ad hoc networks