QuickCast: Fast and Efficient Inter-Datacenter Transfers using Forwarding Tree Cohorts

Large inter-datacenter transfers are crucial for cloud service efficiency and are increasingly used by organizations that have dedicated wide area networks between datacenters. A recent work uses multicast forwarding trees to reduce the bandwidth needs and improve completion times of point-to-multipoint transfers. Using a single forwarding tree per transfer, however, leads to poor performance because the slowest receiver dictates the completion time for all receivers. Using multiple forwarding trees per transfer alleviates this concern--the average receiver could finish early; however, if done naively, bandwidth usage would also increase and it is apriori unclear how best to partition receivers, how to construct the multiple trees and how to determine the rate and schedule of flows on these trees. This paper presents QuickCast, a first solution to these problems. Using simulations on real-world network topologies, we see that QuickCast can speed up the average receiver's completion time by as much as $10\times$ while only using $1.04\times$ more bandwidth; further, the completion time for all receivers also improves by as much as $1.6\times$ faster at high loads.Comment: [Extended Version] Accepted for presentation in IEEE INFOCOM 2018, Honolulu, H

Automated performance attack discovery in distributed system implementations

Security and performance are critical goals for distributed systems. The increased complexity in design, incomplete expertise of developers, and limited functionality of existing testing tools often result in implementations with vulnerabilities and make the debugging process difficult and costly. The deployed vulnerabilities are often exploited by adversaries preventing the system from achieving its design goals. We refer to attacks that slow down the performance of a system as performance attacks. In the past, finding performance attacks has been a painstaking manual process that involved an expert of the target implementation. Given the cost associated with each vulnerability that occurs in the production, there is a need for tools to automatically check that the implementation of a protocol achieves its performance goals with respect to malicious components in the system. In this dissertation, we find performance attacks automatically from implementations of distributed systems. We do not try to show that an implementation is free from all attacks. Our goal is to find attacks and report them to the user in a timely manner. We first investigate how to find attacks automatically from implementations under a simulated environment. A simulated approach, however, has a fundamental limitation in terms of applicable target systems, as certain assumptions are made about languages, operating systems or libraries used. Therefore, we next investigate challenges and requirements to automatically find attacks in implementations of distributed systems under an emulated environment where no limiting assumptions are made

Quality of service (QoS) support for multimedia applications in large-scale networks

This dissertation studied issues pertaining to QoS provision for multimedia applications at the application layer. We initially studied Internet routing pathology and Internet routing stability by repeating experimental and analytical methods conducted by Paxson in 1996. No similar study was done in recent years. Our findings show that routing behavior of the Internet in 2006 are different from those reported in 1996 in some important aspects. Second, we investigated different stochastic models (e.g. self-similar processes, Auto-Regressive Integrated Moving-Average (ARIMA)) in order to find a suitable model that describes available bandwidth over time of an end-to-end path between two Internet hosts. Our finding of the suitable model is beneficial to predicting of future values of available bandwidth along an end-to-end path. To the best of our knowledge, no similar study was conducted. Third, we designed and evaluated a new path monitoring algorithm inferring available bandwidth of an end-to-end path without monitoring all the paths to minimize monitoring overhead. Our algorithm does not rely on underlying network-layer topology information as required in topology-aware path monitoring techniques. Finally, to complement the above study, we introduced our multicast protocol named core-set routing for transmitting multimedia data from a set of senders to a set of receivers, taking QoS into account. The protocol is suitable for interactive multi-sender multimedia applications such as video conferencing and network gaming

Flexible Application-Layer Multicast in Heterogeneous Networks

This work develops a set of peer-to-peer-based protocols and extensions in order to provide Internet-wide group communication. The focus is put to the question how different access technologies can be integrated in order to face the growing traffic load problem. Thereby, protocols are developed that allow autonomous adaptation to the current network situation on the one hand and the integration of WiFi domains where applicable on the other hand

Scalability in adaptive multi-metric overlays

Increasing application requirements have placed heavy emphasis on building overlay networks to efficiently deliver data to multiple receivers. A key performance challenge is simultaneously achieving adaptivity to changing network conditions and scalability to large numbers of users. In addition, most current algorithms focus on a single performance metric, such as delay or bandwidth, particular to individual application requirements. We introduce a two-fold approach for creating robust, high-performance overlays called adaptive multimetric overlays (AMMO). First, AMMO uses an adaptive, highly-parallel, and metric-independent protocol, TreeMaint, to build and maintain overlay trees. Second, AMMO provides a mechanism for comparing overlay edges along specified application performance goals to guide TreeMaint transformations. We have used AMMO to implement and evaluate a single-metric (bandwidth-optimized) tree similar to Overcast and a two-metric (delay-constrained, cost-optimized) overla

Scalability in Adaptive Multi-Metric Overlays

Increasing application requirements have placed heavy emphasis on building overlay networks to efficiently deliver data to multiple receivers. A key performance challenge is simultaneously achieving adaptivity to changing network conditions and scalability to large numbers of users. In addition, most current algorithms focus on a single performance metric, such as delay or bandwidth, particular to individual application requirements. In this paper, we introduce a two-fold approach for creating robust, high-performance overlays called Adaptive Multi-Metric Overlays (AMMO). First, AMMO uses an adaptive, highlyparallel, and metric-independent protocol, TreeMaint, to build and maintain overlay trees. Second, AMMO provides a mechanism for comparing overlay edges along specified application performance goals to guide TreeMaint transformations. We have used AMMO to implement and evaluate a single-metric (bandwidth-optimized) tree similar to Overcast and a two-metric (delay-constrained, cost-optimized) overlay

Scalability in adaptive multi-metric overlays

Increasing application requirements have placed heavy emphasis on building overlay networks to efficiently deliver data to multiple receivers. A key performance challenge is simultaneously achieving adaptivity to changing network conditions and scalability to large numbers of users. In addition, most current algorithms focus on a single performance metric, such as delay or bandwidth, particular to individual application requirements. We introduce a two-fold approach for creating robust, high-performance overlays called adaptive multimetric overlays (AMMO). First, AMMO uses an adaptive, highly-parallel, and metric-independent protocol, TreeMaint, to build and maintain overlay trees. Second, AMMO provides a mechanism for comparing overlay edges along specified application performance goals to guide TreeMaint transformations. We have used AMMO to implement and evaluate a single-metric (bandwidth-optimized) tree similar to Overcast and a two-metric (delay-constrained, cost-optimized) overlayDept. of Comput. Sci., Duke Univ., Durham, NC, USAQC 20140704</p