Maiter: An Asynchronous Graph Processing Framework for Delta-based Accumulative Iterative Computation

Myriad of graph-based algorithms in machine learning and data mining require parsing relational data iteratively. These algorithms are implemented in a large-scale distributed environment in order to scale to massive data sets. To accelerate these large-scale graph-based iterative computations, we propose delta-based accumulative iterative computation (DAIC). Different from traditional iterative computations, which iteratively update the result based on the result from the previous iteration, DAIC updates the result by accumulating the "changes" between iterations. By DAIC, we can process only the "changes" to avoid the negligible updates. Furthermore, we can perform DAIC asynchronously to bypass the high-cost synchronous barriers in heterogeneous distributed environments. Based on the DAIC model, we design and implement an asynchronous graph processing framework, Maiter. We evaluate Maiter on local cluster as well as on Amazon EC2 Cloud. The results show that Maiter achieves as much as 60x speedup over Hadoop and outperforms other state-of-the-art frameworks.Comment: ScienceCloud 2012, TKDE 201

Quantifying AS Path Inflation by Routing Policies

A route in the Internet may take a longer AS path than the shortest AS path due to routing policies. In this paper, we systematically analyze AS paths and quantify the extent to which routing policies inflate AS paths. The results show that AS path inflation in the Internet is more prevalent than expected. We first present the extent of AS path inflation observed from the RouteView and RIPE routing tables. We then employ three common routing policies to show the extent of AS path inflation. We find that No-Valley routing policy causes the least AS path inflation among the three routing policies. PreferCustomer-and-Peer-over-Provider policy causes the most AS path inflation. In addition, we find that single-homed stub ASes experience more path inflations than transit ASes and multi-homed ASes. The AS pairs with shortest AS path of 3 AS hops experience more path inflations than other AS pairs. Finally, we investigate the AS path inflation on the end-to-end path from end users to two popular content providers, Google and Comcast. Although the majority of the shortest AS paths from end users to the two providers consists of no more than three AS hops, the actual end-to-end paths that the traffic will take are longer than the shortest AS paths in many cases. Quantifying AS path inflation in the Internet has important implications on the extent of routing policies, traffic engineering performed on the Internet, and BGP convergence speed