Bias reduction in traceroute sampling: towards a more accurate map of the Internet

Traceroute sampling is an important technique in exploring the internet router graph and the autonomous system graph. Although it is one of the primary techniques used in calculating statistics about the internet, it can introduce bias that corrupts these estimates. This paper reports on a theoretical and experimental investigation of a new technique to reduce the bias of traceroute sampling when estimating the degree distribution. We develop a new estimator for the degree of a node in a traceroute-sampled graph; validate the estimator theoretically in Erdos-Renyi graphs and, through computer experiments, for a wider range of graphs; and apply it to produce a new picture of the degree distribution of the autonomous system graph.Comment: 12 pages, 3 figure

A Graph Theoretic Perspective on Internet Topology Mapping

Understanding the topological characteristics of the Internet is an important research issue as the Internet grows with no central authority. Internet topology mapping studies help better understand the structure and dynamics of the Internet backbone. Knowing the underlying topology, researchers can better develop new protocols and services or fine-tune existing ones. Subnet-level Internet topology measurement studies involve three stages: topology collection, topology construction, and topology analysis. Each of these stages contains challenging tasks, especially when large-scale backbone topologies of millions of nodes are studied. In this dissertation, I first discuss issues in subnet-level Internet topology mapping and review state-of-the-art approaches to handle them. I propose a novel graph data indexing approach to to efficiently process large scale topology data. I then conduct an experimental study to understand how the responsiveness of routers has changed over the last decade and how it differs based on the probing mechanism. I then propose an efficient unresponsive resolution approach by incorporating our structural graph indexing technique. Finally, I introduce Cheleby, an integrated Internet topology mapping system. Cheleby first dynamically probes observed subnetworks using a team of PlanetLab nodes around the world to obtain comprehensive backbone topologies. Then, it utilizes efficient algorithms to resolve subnets, IP aliases, and unresponsive routers in the collected data sets to construct comprehensive subnet-level topologies. Sample topologies are provided at http://cheleby.cse.unr.edu

Efficient Internet Topology Discovery Techniques

Current macroscopic Internet topology discovery projects use large numbers of vantage points to conduct traceroute surveys of Internet paths. These projects send billions of unsolicited packets to millions of routers within the Internet. Due to the structure of the Internet, many of these packets are sent without gaining any new topology information. In this thesis, we implement and extensively test a largescale doubletree system designed to increase the efficiency of topology mapping projects and reduce the load that they place on the Internet. Also, for all of the effort that current projects put into gathering data, the methods used do not discover, with confidence, the entire set of paths. We propose, implement and critique a novel algorithm, economical MDA traceroute, which is designed to discover a comprehensive topology in a manner which is more efficient than the current state of the art. We show that, compared to current methods, well over 90% link coverage can be obtained while reducing the number of probes used by over 60%. We also evaluate alternate methods for making large scale topology discovery projects more efficient and comprehensive; such as using BGP routing data to guide probing

Affecting IP traceback with recent Internet topology maps

Computer network attacks are on the increase and are more sophisticated in today\u27s network environment than ever before. One step in tackling the increasing spate of attacks is the availability of a system that can trace attack packets back to their original sources irrespective of invalid or manipulated source addresses. IP Traceback is one of such methods, and several schemes have already been proposed in this area. Notably though, no traceback scheme is in wide use today due to reasons including a lack of compatibility with existing network protocols and infrastructure, as well as the high costs of deployment. Recently, remarkable progress has been made in the area of Internet topology mappings and more detailed and useful maps and metrics of the Internet are being made available to the corporate and academic research communities. This thesis introduces a novel use of these maps to influence IP Traceback in general, and packet marking schemes in particular. We note that while other schemes have previously taken advantage of such maps, most of these have viewed the maps from the available router node level. We take a novel router-aggregation node view of the Internet and explore ways to use this to make improvements to packet marking schemes and solving the problem of the limited space available in the current IP header for marking purposes. We evaluate our proposed schemes using real network paths traversed by several traceroute packets from diverse sources and to various destinations, and compare our results to other packet marking schemes. Finally, we explore the possibility of partial deployment of one of our schemes and estimate the probability of success at different stages of deployment

Measuring The Evolving Internet Ecosystem With Exchange Points

The Internet ecosystem comprising of thousands of Autonomous Systems (ASes) now include Internet eXchange Points (IXPs) as another critical component in the infrastructure. Peering plays a significant part in driving the economic growth of ASes and is contributing to a variety of structural changes in the Internet. IXPs are a primary component of this peering ecosystem and are playing an increasing role not only in the topology evolution of the Internet but also inter-domain path routing. In this dissertation we study and analyze the overall affects of peering and IXP infrastructure on the Internet. We observe IXP peering is enabling a quicker flattening of the Internet topology and leading to over-utilization of popular inter-AS links. Indiscriminate peering at these locations is leading to higher endto-end path latencies for ASes peering at an exchange point, an effect magnified at the most popular worldwide IXPs. We first study the effects of recently discovered IXP links on the inter-AS routes using graph based approaches and find that it points towards the changing and flattening landscape in the evolution of the Internet’s topology. We then study more IXP effects by using measurements to investigate the networks benefits of peering. We propose and implement a measurement framework which identifies default paths through IXPs and compares them with alternate paths isolating the IXP hop. Our system is running and recording default and alternate path latencies and made publicly available. We model the probability of an alternate path performing better than a default path through an IXP iii by identifying the underlying factors influencing the end-to end path latency. Our firstof-its-kind modeling study, which uses a combination of statistical and machine learning approaches, shows that path latencies depend on the popularity of the particular IXP, the size of the provider ASes of the networks peering at common locations and the relative position of the IXP hop along the path. An in-depth comparison of end-to-end path latencies reveal a significant percentage of alternate paths outperforming the default route through an IXP. This characteristic of higher path latencies is magnified in the popular continental exchanges as measured by us in a case study looking at the largest regional IXPs. We continue by studying another effect of peering which has numerous applications in overlay routing, Triangle Inequality Violations (TIVs). These TIVs in the Internet delay space are created due to peering and we compare their essential characteristics with overlay paths such as detour routes. They are identified and analyzed from existing measurement datasets but on a scale not carried out earlier. This implementation exhibits the effectiveness of GPUs in analyzing big data sets while the TIVs studied show that the a set of common inter-AS links create these TIVs. This result provides a new insight about the development of TIVs by analyzing a very large data set using GPGPUs. Overall our work presents numerous insights into the inner workings of the Internet’s peering ecosystem. Our measurements show the effects of exchange points on the evolving Internet and exhibits their importance to Internet routing

Improving Pan-African research and education networks through traffic engineering: A LISP/SDN approach

The UbuntuNet Alliance, a consortium of National Research and Education Networks (NRENs) runs an exclusive data network for education and research in east and southern Africa. Despite a high degree of route redundancy in the Alliance's topology, a large portion of Internet traffic between the NRENs is circuitously routed through Europe. This thesis proposes a performance-based strategy for dynamic ranking of inter-NREN paths to reduce latencies. The thesis makes two contributions: firstly, mapping Africa's inter-NREN topology and quantifying the extent and impact of circuitous routing; and, secondly, a dynamic traffic engineering scheme based on Software Defined Networking (SDN), Locator/Identifier Separation Protocol (LISP) and Reinforcement Learning. To quantify the extent and impact of circuitous routing among Africa's NRENs, active topology discovery was conducted. Traceroute results showed that up to 75% of traffic from African sources to African NRENs went through inter-continental routes and experienced much higher latencies than that of traffic routed within Africa. An efficient mechanism for topology discovery was implemented by incorporating prior knowledge of overlapping paths to minimize redundancy during measurements. Evaluation of the network probing mechanism showed a 47% reduction in packets required to complete measurements. An interactive geospatial topology visualization tool was designed to evaluate how NREN stakeholders could identify routes between NRENs. Usability evaluation showed that users were able to identify routes with an accuracy level of 68%. NRENs are faced with at least three problems to optimize traffic engineering, namely: how to discover alternate end-to-end paths; how to measure and monitor performance of different paths; and how to reconfigure alternate end-to-end paths. This work designed and evaluated a traffic engineering mechanism for dynamic discovery and configuration of alternate inter-NREN paths using SDN, LISP and Reinforcement Learning. A LISP/SDN based traffic engineering mechanism was designed to enable NRENs to dynamically rank alternate gateways. Emulation-based evaluation of the mechanism showed that dynamic path ranking was able to achieve 20% lower latencies compared to the default static path selection. SDN and Reinforcement Learning were used to enable dynamic packet forwarding in a multipath environment, through hop-by-hop ranking of alternate links based on latency and available bandwidth. The solution achieved minimum latencies with significant increases in aggregate throughput compared to static single path packet forwarding. Overall, this thesis provides evidence that integration of LISP, SDN and Reinforcement Learning, as well as ranking and dynamic configuration of paths could help Africa's NRENs to minimise latencies and to achieve better throughputs

Network-provider-independent overlays for resilience and quality of service.

PhDOverlay networks are viewed as one of the solutions addressing the inefficiency and slow evolution of the Internet and have been the subject of significant research. Most existing overlays providing resilience and/or Quality of Service (QoS) need cooperation among different network providers, but an inter-trust issue arises and cannot be easily solved. In this thesis, we mainly focus on network-provider-independent overlays and investigate their performance in providing two different types of service. Specifically, this thesis addresses the following problems: Provider-independent overlay architecture: A provider-independent overlay framework named Resilient Overlay for Mission-Critical Applications (ROMCA) is proposed. We elaborate its structure including component composition and functions and also provide several operational examples. Overlay topology construction for providing resilience service: We investigate the topology design problem of provider-independent overlays aiming to provide resilience service. To be more specific, based on the ROMCA framework, we formulate this problem mathematically and prove its NP-hardness. Three heuristics are proposed and extensive simulations are carried out to verify their effectiveness. Application mapping with resilience and QoS guarantees: Assuming application mapping is the targeted service for ROMCA, we formulate this problem as an Integer Linear Program (ILP). Moreover, a simple but effective heuristic is proposed to address this issue in a time-efficient manner. Simulations with both synthetic and real networks prove the superiority of both solutions over existing ones. Substrate topology information availability and the impact of its accuracy on overlay performance: Based on our survey that summarizes the methodologies available for inferring the selective substrate topology formed among a group of nodes through active probing, we find that such information is usually inaccurate and additional mechanisms are needed to secure a better inferred topology. Therefore, we examine the impact of inferred substrate topology accuracy on overlay performance given only inferred substrate topology information