An Internet Heartbeat

Obtaining sound inferences over remote networks via active or passive measurements is difficult. Active measurement campaigns face challenges of load, coverage, and visibility. Passive measurements require a privileged vantage point. Even networks under our own control too often remain poorly understood and hard to diagnose. As a step toward the democratization of Internet measurement, we consider the inferential power possible were the network to include a constant and predictable stream of dedicated lightweight measurement traffic. We posit an Internet "heartbeat," which nodes periodically send to random destinations, and show how aggregating heartbeats facilitates introspection into parts of the network that are today generally obtuse. We explore the design space of an Internet heartbeat, potential use cases, incentives, and paths to deployment

Active Learning of Multiple Source Multiple Destination Topologies

We consider the problem of inferring the topology of a network with $M$ sources and $N$ receivers (hereafter referred to as an $M$-by-$N$ network), by sending probes between the sources and receivers. Prior work has shown that this problem can be decomposed into two parts: first, infer smaller subnetwork components (i.e., $1$-by-$N$'s or $2$-by-$2$'s) and then merge these components to identify the $M$-by-$N$ topology. In this paper, we focus on the second part, which had previously received less attention in the literature. In particular, we assume that a $1$-by-$N$ topology is given and that all $2$-by-$2$ components can be queried and learned using end-to-end probes. The problem is which $2$-by-$2$'s to query and how to merge them with the given $1$-by-$N$, so as to exactly identify the $2$-by-$N$ topology, and optimize a number of performance metrics, including the number of queries (which directly translates into measurement bandwidth), time complexity, and memory usage. We provide a lower bound, $\lceil \frac{N}{2} \rceil$, on the number of $2$-by-$2$'s required by any active learning algorithm and propose two greedy algorithms. The first algorithm follows the framework of multiple hypothesis testing, in particular Generalized Binary Search (GBS), since our problem is one of active learning, from $2$-by-$2$ queries. The second algorithm is called the Receiver Elimination Algorithm (REA) and follows a bottom-up approach: at every step, it selects two receivers, queries the corresponding $2$-by-$2$, and merges it with the given $1$-by-$N$; it requires exactly $N-1$ steps, which is much less than all $\binom{N}{2}$ possible $2$-by-$2$'s. Simulation results over synthetic and realistic topologies demonstrate that both algorithms correctly identify the $2$-by-$N$ topology and are near-optimal, but REA is more efficient in practice

Task Specific Uncertainty in Coordinate Measurement

Task specific uncertainty is the measurement uncertainty associated with the measurement of a specific feature using a specific measurement plan. This paper surveys techniques developed to model and estimate task specific uncertainty for coordinate measuring systems, primarily coordinate measuring machines using contacting probes. Sources of uncertainty are also reviewed

Intrusion Detection Systems Using Adaptive Regression Splines

Past few years have witnessed a growing recognition of intelligent techniques for the construction of efficient and reliable intrusion detection systems. Due to increasing incidents of cyber attacks, building effective intrusion detection systems (IDS) are essential for protecting information systems security, and yet it remains an elusive goal and a great challenge. In this paper, we report a performance analysis between Multivariate Adaptive Regression Splines (MARS), neural networks and support vector machines. The MARS procedure builds flexible regression models by fitting separate splines to distinct intervals of the predictor variables. A brief comparison of different neural network learning algorithms is also given

DMFSGD: A Decentralized Matrix Factorization Algorithm for Network Distance Prediction

The knowledge of end-to-end network distances is essential to many Internet applications. As active probing of all pairwise distances is infeasible in large-scale networks, a natural idea is to measure a few pairs and to predict the other ones without actually measuring them. This paper formulates the distance prediction problem as matrix completion where unknown entries of an incomplete matrix of pairwise distances are to be predicted. The problem is solvable because strong correlations among network distances exist and cause the constructed distance matrix to be low rank. The new formulation circumvents the well-known drawbacks of existing approaches based on Euclidean embedding. A new algorithm, so-called Decentralized Matrix Factorization by Stochastic Gradient Descent (DMFSGD), is proposed to solve the network distance prediction problem. By letting network nodes exchange messages with each other, the algorithm is fully decentralized and only requires each node to collect and to process local measurements, with neither explicit matrix constructions nor special nodes such as landmarks and central servers. In addition, we compared comprehensively matrix factorization and Euclidean embedding to demonstrate the suitability of the former on network distance prediction. We further studied the incorporation of a robust loss function and of non-negativity constraints. Extensive experiments on various publicly-available datasets of network delays show not only the scalability and the accuracy of our approach but also its usability in real Internet applications.Comment: submitted to IEEE/ACM Transactions on Networking on Nov. 201