11 research outputs found
Topology Discovery of Sparse Random Graphs With Few Participants
We consider the task of topology discovery of sparse random graphs using
end-to-end random measurements (e.g., delay) between a subset of nodes,
referred to as the participants. The rest of the nodes are hidden, and do not
provide any information for topology discovery. We consider topology discovery
under two routing models: (a) the participants exchange messages along the
shortest paths and obtain end-to-end measurements, and (b) additionally, the
participants exchange messages along the second shortest path. For scenario
(a), our proposed algorithm results in a sub-linear edit-distance guarantee
using a sub-linear number of uniformly selected participants. For scenario (b),
we obtain a much stronger result, and show that we can achieve consistent
reconstruction when a sub-linear number of uniformly selected nodes
participate. This implies that accurate discovery of sparse random graphs is
tractable using an extremely small number of participants. We finally obtain a
lower bound on the number of participants required by any algorithm to
reconstruct the original random graph up to a given edit distance. We also
demonstrate that while consistent discovery is tractable for sparse random
graphs using a small number of participants, in general, there are graphs which
cannot be discovered by any algorithm even with a significant number of
participants, and with the availability of end-to-end information along all the
paths between the participants.Comment: A shorter version appears in ACM SIGMETRICS 2011. This version is
scheduled to appear in J. on Random Structures and Algorithm
Active Topology Inference using Network Coding
Our goal is to infer the topology of a network when (i) we can send probes
between sources and receivers at the edge of the network and (ii) intermediate
nodes can perform simple network coding operations, i.e., additions. Our key
intuition is that network coding introduces topology-dependent correlation in
the observations at the receivers, which can be exploited to infer the
topology. For undirected tree topologies, we design hierarchical clustering
algorithms, building on our prior work. For directed acyclic graphs (DAGs),
first we decompose the topology into a number of two-source, two-receiver
(2-by-2) subnetwork components and then we merge these components to
reconstruct the topology. Our approach for DAGs builds on prior work on
tomography, and improves upon it by employing network coding to accurately
distinguish among all different 2-by-2 components. We evaluate our algorithms
through simulation of a number of realistic topologies and compare them to
active tomographic techniques without network coding. We also make connections
between our approach and alternatives, including passive inference, traceroute,
and packet marking
Multicast-based Weight Inference in General Network Topologies
Network topology plays an important role in many
network operations. However, it is very difficult to obtain
the topology of public networks due to the lack of internal
cooperation. Network tomography provides a powerful solution
that can infer the network routing topology from end-to-end
measurements. Existing solutions all assume that routes from a
single source form a tree. However, with the rapid deployment
of Software Defined Networking (SDN) and Network Function
Virtualization (NFV), the routing paths in modern networks are
becoming more complex. To address this problem, we propose
a novel inference problem, called the weight inference problem,
which infers the finest-granularity information from end-to-end
measurements on general routing paths in general topologies.
Our measurements are based on emulated multicast probes with
a controllable “width”. We show that the problem has a unique
solution when the multicast width is unconstrained; otherwise,
we show that the problem can be treated as a sparse approximation problem, which allows us to apply variations of the
pursuit algorithms. Simulations based on real network topologies
show that our solution significantly outperforms a state-of-theart network tomography algorithm, and increasing the width of
multicast substantially improves the inference accuracy
Active topology inference using network coding
Our goal, in this paper, is to infer the topology of a network when (i) we can send probes between sources and receivers at the edge of the network and (ii) intermediate nodes can perform simple network coding operations, i.e., additions. Our key intuition is that network coding introduces topology-dependent correlation in the observations at the receivers, which can be exploited to infer the topology. For undirected tree topologies, we design hierarchical clustering algorithms, building on our prior work in [24]. For directed acyclic graphs (DAGs), first we decompose the topology into a number of two source, two receiver (2-by-2) subnetwork components and then we merge these components to reconstruct the topology. Our approach for DAGs builds on prior work on tomography [36], and improves upon it by employing network coding to accurately distinguish among all different 2-by-2 components. We evaluate our algorithms through simulation of a number of realistic topologies and compare them to active tomographic techniques without network coding. We also make connections between our approach and other alternatives, including passive inference, traceroute, and packet marking
Improving the Performance of Cloud-based Scientific Services
Cloud computing provides access to a large scale set of readily available computing resources at the click of a button. The cloud paradigm has commoditised computing capacity and is often touted as a low-cost model for executing and scaling applications. However, there are significant technical challenges associated with selecting, acquiring, configuring, and managing cloud resources which can restrict the efficient utilisation of cloud capabilities.
Scientific computing is increasingly hosted on cloud infrastructure—in which scientific capabilities are delivered to the broad scientific community via Internet-accessible services. This migration from on-premise to on-demand cloud infrastructure is motivated by the sporadic usage patterns of scientific workloads and the associated potential cost savings without the need to purchase, operate, and manage compute infrastructure—a task that few scientific users are trained to perform. However, cloud platforms are not an automatic solution. Their flexibility is derived from an enormous number of services and configuration options, which in turn result in significant complexity for the user. In fact, naïve cloud usage can result in poor performance and excessive costs, which are then directly passed on to researchers.
This thesis presents methods for developing efficient cloud-based scientific services. Three real-world scientific services are analysed and a set of common requirements are derived. To address these requirements, this thesis explores automated and scalable methods for inferring network performance, considers various trade-offs (e.g., cost and performance) when provisioning instances, and profiles application performance, all in heterogeneous and dynamic cloud environments. Specifically, network tomography provides the mechanisms to infer network performance in dynamic and opaque cloud networks; cost-aware automated provisioning approaches enable services to consider, in real-time, various trade-offs such as cost, performance, and reliability; and automated application profiling allows a huge search space of applications, instance types, and configurations to be analysed to determine resource requirements and application performance. Finally, these contributions are integrated into an extensible and modular cloud provisioning and resource management service called SCRIMP. Cloud-based scientific applications and services can subscribe to SCRIMP to outsource their provisioning, usage, and management of cloud infrastructures. Collectively, the approaches presented in this thesis are shown to provide order of magnitude cost savings and significant performance improvement when employed by production scientific services