Latency-bandwidth tradeoffs in Internet applications

Wide-area Internet links are slow, expensive, and unreliable. This affects applications in two distinct ways. Back-end data processing applications, which need to transfer large amounts of data between data centers across the world, are primarily constrained by the limited capacity of Internet links. Front-end user facing applications, on the other hand, are primarily latency-sensitive, and are bottlenecked by the high, unpredictably variable delays in the wide-area network. Our work exploits this asymmetry in applications' requirements by developing techniques that trade off one of bandwidth and latency to improve the other. We first consider the problem of supporting analytics over the large volumes of geographically dispersed data produced by global-scale organizations. Current solutions for analyzing this data as a whole operate by copying it to a single central data center, an approach that incurs substantial data transfer costs. We instead propose an alternative geo-distributed approach, orchestrating distributed execution across data centers. Our system, Geode, incorporates two key optimizations --- a low-level syntactic network redundancy elimination mechanism, and a high-level semantically aware workload optimization process --- both of which operate by trading off increased processing overhead (and computation latency) within data centers for a reduction in cross-data center bandwidth usage. In experiments we find that Geode achieves an up to 360x cost reduction compared to the current centralized baseline on a range of workloads, both real and synthetic. Next, we evaluate a simple, general purpose technique for trading off bandwidth for reduced latency: initiate redundant copies of latency sensitive operations and take the first copy to complete. While redundancy has been explored in some past systems, its use is typically avoided because of a fear of the overhead that it adds. We study the latency-bandwidth tradeoff due to redundancy and (i) show via empirical evaluation that its use is indeed a net positive in a number of important applications, and (ii) provide a theoretical characterization of its effect, identifying when it should and should not be used and how systems can tune their use of redundancy to maximum effect. Our results suggest that redundancy should be used much more widely than it currently is

Low Latency Geo-distributed Data Analytics

Low latency analytics on geographically distributed dat-asets (across datacenters, edge clusters) is an upcoming and increasingly important challenge. The dominant approach of aggregating all the data to a single data-center significantly inflates the timeliness of analytics. At the same time, running queries over geo-distributed inputs using the current intra-DC analytics frameworks also leads to high query response times because these frameworks cannot cope with the relatively low and variable capacity of WAN links. We present Iridium, a system for low latency geo-distri-buted analytics. Iridium achieves low query response times by optimizing placement of both data and tasks of the queries. The joint data and task placement op-timization, however, is intractable. Therefore, Iridium uses an online heuristic to redistribute datasets among the sites prior to queries ’ arrivals, and places the tasks to reduce network bottlenecks during the query’s ex-ecution. Finally, it also contains a knob to budget WAN usage. Evaluation across eight worldwide EC2 re-gions using production queries show that Iridium speeds up queries by 3 × − 19 × and lowers WAN usage by 15% − 64 % compared to existing baselines

Application of Jaccard Distance Measure for IVIF MCDM Problems

This paper proposes an approach to solve Multiple Criteria Decision-making (MCDM) problems when the data given by expert is Interval-Valued Intuitionistic Fuzzy (IVIF) information. A decision-making model is constructed by using the distance measure: Normalized Jaccard distance measure. The robustness of the model is illustrated and validated through numerical example. Further, the problem of choosing best e-learning tool in higher education is considered as a case study