Robust and Scalable Sampling Algorithms for Network Measurement

Recent growth of the Internet in both scale and complexity has imposed a number of difficult challenges on existing measurement techniques and approaches, which are essential for both network management and many ongoing research projects. For any measurement algorithm, achieving both accuracy and scalability is very challenging given hard resource constraints (e.g., bandwidth, delay, physical memory, and CPU speed). My dissertation research tackles this problem by first proposing a novel mechanism called residual sampling, which intentionally introduces a predetermined amount of bias into the measurement process. We show that such biased sampling can be extremely scalable; moreover, we develop residual estimation algorithms that can unbiasedly recover the original information from the sampled data. Utilizing these results, we further develop two versions of the residual sampling mechanism: a continuous version for characterizing the user lifetime distribution in large-scale peer-to-peer networks and a discrete version for monitoring flow statistics (including per-flow counts and the flow size distribution) in high-speed Internet routers. For the former application in P2P networks, this work presents two methods: ResIDual-based Estimator (RIDE), which takes single-point snapshots of the system and assumes systems with stationary arrivals, and Uniform RIDE (U-RIDE), which takes multiple snapshots and adapts to systems with arbitrary (including non-stationary) arrival processes. For the latter application in traffic monitoring, we introduce Discrete RIDE (D-RIDE), which allows one to sample each flow with a geometric random variable. Our numerous simulations and experiments with P2P networks and real Internet traces confirm that these algorithms are able to make accurate estimation about the monitored metrics and simultaneously meet the requirements of hard resource constraints. These results show that residual sampling indeed provides an ideal solution to balancing between accuracy and scalability

Monitoring Peer to Peer Systems

National audienceIn this paper, we are concerned with the distributed monitoring of P2P systems. We introduce the P2P Monitor system and a new declarative language, namely P2PML, for specifying monitoring tasks. A subscription is compiled into a distributed algebraic plan which is described using an algebra over XML streams. We introduce a filter for streams of XML documents that scales by processing first simple conditions and then, if still needed, evaluating complex queries. We also show how particular tasks can be supported by identifying subtasks already provided by existing streams

Monitoring Peer to Peer Systems

National audienceIn this paper, we are concerned with the distributed monitoring of P2P systems. We introduce the P2P Monitor system and a new declarative language, namely P2PML, for specifying monitoring tasks. A subscription is compiled into a distributed algebraic plan which is described using an algebra over XML streams. We introduce a filter for streams of XML documents that scales by processing first simple conditions and then, if still needed, evaluating complex queries. We also show how particular tasks can be supported by identifying subtasks already provided by existing streams

An adaptive clustering approach for the management of dynamic systems

Adaptive clustering is one of the fundamental problems behind autonomic systems and, more generally, an open research issue in the area of networking and distributed systems. The problem of giving structure to large-scale, dynamic systems through clustering and of electing centrally located nodes (cluster heads) is nontrivial. This is in fact an NP-complete problem when striving for optimality. We propose an innovative strategy based on code mobility that dynamically computes near-optimal clusters in linear time. Our approach is autonomic, does not require any user intervention, is self-configuring, self-optimal, and self-healing. We demonstrate these features through an extensive set of simulations, discussing the viability of the algorithm based on state-of-the art technologies, and elaborating on its applicability to distributed monitoring, peer-to-peer systems, application-level multicast, and content adaptation networks

An Adaptive Clustering Approach for the Management of Dynamic Systems

Abstract—Adaptive clustering is one of the fundamental problems behind autonomic systems and, more generally, an open research issue in the area of networking and distributed systems. The problem of giving structure to large-scale, dynamic systems through clustering and of electing centrally located nodes (cluster heads) is nontrivial. This is in fact an NP-complete problem when striving for optimality. We propose an innovative strategy based on code mobility that dynamically computes near-optimal clusters in linear time. Our approach is autonomic, does not require any user intervention, is self-configuring, self-optimal, and self-healing. We demonstrate these features through an extensive set of simulations, discussing the viability of the algorithm based on state-of-the art technologies, and elaborating on its applicability to distributed monitoring, peer-to-peer systems, application-level multicast, and content adaptation networks. Index Terms—Autonomic communication systems, clustering methods, code mobility, network partitioning, self-healing, self-management. I