ATP: a Datacenter Approximate Transmission Protocol

Many datacenter applications such as machine learning and streaming systems do not need the complete set of data to perform their computation. Current approximate applications in datacenters run on a reliable network layer like TCP. To improve performance, they either let sender select a subset of data and transmit them to the receiver or transmit all the data and let receiver drop some of them. These approaches are network oblivious and unnecessarily transmit more data, affecting both application runtime and network bandwidth usage. On the other hand, running approximate application on a lossy network with UDP cannot guarantee the accuracy of application computation. We propose to run approximate applications on a lossy network and to allow packet loss in a controlled manner. Specifically, we designed a new network protocol called Approximate Transmission Protocol, or ATP, for datacenter approximate applications. ATP opportunistically exploits available network bandwidth as much as possible, while performing a loss-based rate control algorithm to avoid bandwidth waste and re-transmission. It also ensures bandwidth fair sharing across flows and improves accurate applications' performance by leaving more switch buffer space to accurate flows. We evaluated ATP with both simulation and real implementation using two macro-benchmarks and two real applications, Apache Kafka and Flink. Our evaluation results show that ATP reduces application runtime by 13.9% to 74.6% compared to a TCP-based solution that drops packets at sender, and it improves accuracy by up to 94.0% compared to UDP

Using Operations Data for Planning the the Delaware Valley: First Steps

Real-time traffic operations data has been gathered for several years on an increasing number of roads throughout the Delaware Valley. The archives of this data are a tremendous potential resource for transportation planning. Use of the data, however, has posed significant technical challenges. This report summarizes how the data can be used, the state of operations data for planning in the Delaware Valley, and the results of two case studies. The first case study used data from the Pennsylvania Department of Transportation's Dynac system about speed and travel time on a section of I-76. The second case study used data provided by the I-95 Corridor Coalition Vehicle Probe Project (VPP) from INRIX, a private-sector traffic data company. The second case study analyzed duration of congestion on weekdays in 2009 for freeways in the Delaware Valley. This analysis was used in the region's 2011 Congestion Management Process