DMFSGD: A Decentralized Matrix Factorization Algorithm for Network Distance Prediction

The knowledge of end-to-end network distances is essential to many Internet applications. As active probing of all pairwise distances is infeasible in large-scale networks, a natural idea is to measure a few pairs and to predict the other ones without actually measuring them. This paper formulates the distance prediction problem as matrix completion where unknown entries of an incomplete matrix of pairwise distances are to be predicted. The problem is solvable because strong correlations among network distances exist and cause the constructed distance matrix to be low rank. The new formulation circumvents the well-known drawbacks of existing approaches based on Euclidean embedding. A new algorithm, so-called Decentralized Matrix Factorization by Stochastic Gradient Descent (DMFSGD), is proposed to solve the network distance prediction problem. By letting network nodes exchange messages with each other, the algorithm is fully decentralized and only requires each node to collect and to process local measurements, with neither explicit matrix constructions nor special nodes such as landmarks and central servers. In addition, we compared comprehensively matrix factorization and Euclidean embedding to demonstrate the suitability of the former on network distance prediction. We further studied the incorporation of a robust loss function and of non-negativity constraints. Extensive experiments on various publicly-available datasets of network delays show not only the scalability and the accuracy of our approach but also its usability in real Internet applications.Comment: submitted to IEEE/ACM Transactions on Networking on Nov. 201

How to Create an Innovation Accelerator

Too many policy failures are fundamentally failures of knowledge. This has become particularly apparent during the recent financial and economic crisis, which is questioning the validity of mainstream scholarly paradigms. We propose to pursue a multi-disciplinary approach and to establish new institutional settings which remove or reduce obstacles impeding efficient knowledge creation. We provided suggestions on (i) how to modernize and improve the academic publication system, and (ii) how to support scientific coordination, communication, and co-creation in large-scale multi-disciplinary projects. Both constitute important elements of what we envision to be a novel ICT infrastructure called "Innovation Accelerator" or "Knowledge Accelerator".Comment: 32 pages, Visioneer White Paper, see http://www.visioneer.ethz.c

Ant-inspired Interaction Networks For Decentralized Vehicular Traffic Congestion Control

Mimicking the autonomous behaviors of animals and their adaptability to changing or foreign environments lead to the development of swarm intelligence techniques such as ant colony optimization (ACO) and particle swarm optimization (PSO) now widely used to tackle a variety of optimization problems. The aim of this dissertation is to develop an alternative swarm intelligence model geared toward decentralized congestion avoidance and to determine qualities of the model suitable for use in a transportation network. A microscopic multi-agent interaction network inspired by insect foraging behaviors, especially ants, was developed and consequently adapted to prioritize the avoidance of congestion, evaluated as perceived density of other agents in the immediate environment extrapolated from the occurrence of direct interactions between agents, while foraging for food outside the base/nest. The agents eschew pheromone trails or other forms of stigmergic communication in favor of these direct interactions whose rate is the primary motivator for the agents\u27 decision making process. The decision making process at the core of the multi-agent interaction network is consequently transferred to transportation networks utilizing vehicular ad-hoc networks (VANETs) for communication between vehicles. Direct interactions are replaced by dedicated short range communications for wireless access in vehicular environments (DSRC/WAVE) messages used for a variety of applications like left turn assist, intersection collision avoidance, or cooperative adaptive cruise control. Each vehicle correlates the traffic on the wireless network with congestion in the transportation network and consequently decides whether to reroute and, if so, what alternate route to take in a decentralized, non-deterministic manner. The algorithm has been shown to increase throughput and decrease mean travel times significantly while not requiring access to centralized infrastructure or up-to-date traffic information