ATP: a Datacenter Approximate Transmission Protocol

Many datacenter applications such as machine learning and streaming systems do not need the complete set of data to perform their computation. Current approximate applications in datacenters run on a reliable network layer like TCP. To improve performance, they either let sender select a subset of data and transmit them to the receiver or transmit all the data and let receiver drop some of them. These approaches are network oblivious and unnecessarily transmit more data, affecting both application runtime and network bandwidth usage. On the other hand, running approximate application on a lossy network with UDP cannot guarantee the accuracy of application computation. We propose to run approximate applications on a lossy network and to allow packet loss in a controlled manner. Specifically, we designed a new network protocol called Approximate Transmission Protocol, or ATP, for datacenter approximate applications. ATP opportunistically exploits available network bandwidth as much as possible, while performing a loss-based rate control algorithm to avoid bandwidth waste and re-transmission. It also ensures bandwidth fair sharing across flows and improves accurate applications' performance by leaving more switch buffer space to accurate flows. We evaluated ATP with both simulation and real implementation using two macro-benchmarks and two real applications, Apache Kafka and Flink. Our evaluation results show that ATP reduces application runtime by 13.9% to 74.6% compared to a TCP-based solution that drops packets at sender, and it improves accuracy by up to 94.0% compared to UDP

Mathematical Models and Algorithms for Network Flow Problems Arising in Wireless Sensor Network Applications

We examine multiple variations on two classical network flow problems, the maximum flow and minimum-cost flow problems. These two problems are well-studied within the optimization community, and many models and algorithms have been presented for their solution. Due to the unique characteristics of the problems we consider, existing approaches cannot be directly applied. The problem variations we examine commonly arise in wireless sensor network (WSN) applications. A WSN consists of a set of sensors and collection sinks that gather and analyze environmental conditions. In addition to providing a taxonomy of relevant literature, we present mathematical programming models and algorithms for solving such problems. First, we consider a variation of the maximum flow problem having node-capacity restrictions. As an alternative to solving a single linear programming (LP) model, we present two alternative solution techniques. The first iteratively solves two smaller auxiliary LP models, and the second is a heuristic approach that avoids solving any LP. We also examine a variation of the maximum flow problem having semicontinuous restrictions that requires the flow, if positive, on any path to be greater than or equal to a minimum threshold. To avoid solving a mixed-integer programming (MIP) model, we present a branch-and-price algorithm that significantly improves the computational time required to solve the problem. Finally, we study two dynamic network flow problems that arise in wireless sensor networks under non-simultaneous flow assumptions. We first consider a dynamic maximum flow problem that requires an arc to transmit a minimum amount of flow each time it begins transmission. We present an MIP for solving this problem along with a heuristic algorithm for its solution. Additionally, we study a dynamic minimum-cost flow problem, in which an additional cost is incurred each time an arc begins transmission. In addition to an MIP, we present an exact algorithm that iteratively solves a relaxed version of the MIP until an optimal solution is found

Datacenter Traffic Control: Understanding Techniques and Trade-offs

Datacenters provide cost-effective and flexible access to scalable compute and storage resources necessary for today's cloud computing needs. A typical datacenter is made up of thousands of servers connected with a large network and usually managed by one operator. To provide quality access to the variety of applications and services hosted on datacenters and maximize performance, it deems necessary to use datacenter networks effectively and efficiently. Datacenter traffic is often a mix of several classes with different priorities and requirements. This includes user-generated interactive traffic, traffic with deadlines, and long-running traffic. To this end, custom transport protocols and traffic management techniques have been developed to improve datacenter network performance. In this tutorial paper, we review the general architecture of datacenter networks, various topologies proposed for them, their traffic properties, general traffic control challenges in datacenters and general traffic control objectives. The purpose of this paper is to bring out the important characteristics of traffic control in datacenters and not to survey all existing solutions (as it is virtually impossible due to massive body of existing research). We hope to provide readers with a wide range of options and factors while considering a variety of traffic control mechanisms. We discuss various characteristics of datacenter traffic control including management schemes, transmission control, traffic shaping, prioritization, load balancing, multipathing, and traffic scheduling. Next, we point to several open challenges as well as new and interesting networking paradigms. At the end of this paper, we briefly review inter-datacenter networks that connect geographically dispersed datacenters which have been receiving increasing attention recently and pose interesting and novel research problems.Comment: Accepted for Publication in IEEE Communications Surveys and Tutorial

Review of Optimization Problems in Wireless Sensor Networks

International audienc

Foundations of Infrastructure CPS

Infrastructures have been around as long as urban centers, supporting a society’s needs for its planning, operation, and safety. As we move deeper into the 21st century, these infrastructures are becoming smart – they monitor themselves, communicate, and most importantly self-govern, which we denote as Infrastructure CPS. Cyber-physical systems are now becoming increasingly prevalent and possibly even mainstream. With the basics of CPS in place, such as stability, robustness, and reliability properties at a systems level, and hybrid, switched, and eventtriggered properties at a network level, we believe that the time is right to go to the next step, Infrastructure CPS, which forms the focus of the proposed tutorial. We discuss three different foundations, (i) Human Empowerment, (ii) Transactive Control, and (iii) Resilience. This will be followed by two examples, one on the nexus between power and communication infrastructure, and the other between natural gas and electricity, both of which have been investigated extensively of late, and are emerging to be apt illustrations of Infrastructure CPS