The Scope of the IBGP Routing Anomaly Problem

Correctness problems in the iBGP routing, the de-facto standard to spread global routing information in Autonomous Systems, are a well-known issue. Configurations may route cost-suboptimal, inconsistent, or even behave non-convergent and -deterministic. However, even if a lot of studies have shown many exemplary problematic configurations, the exact scope of the problem is largely unknown: Up to now, it is not clear which problems may appear under which iBGP architectures. The exact scope of the iBGP correctness problem is of high theoretical and practical interest. Knowledge on the resistance of specific architecture schemes against certain anomaly classes and the reasons may help to improve other iBGP schemes. Knowledge on the specific problems of the different schemes helps to identify the right scheme for an AS and develop workarounds

Scalability of iBGP Path Diversity Concepts

Abstract. Improving the path diversity seems to be the next fundamental step in the iBGP evolution. Focusing the advantages an improvement of the path diversity implies, network protocol designers have disregarded the most critical drawback so far: The effect on the scalability of the iBGP routing, a fundamental requirement for production usage. This aspect is examined by the analyses discussed in our paper. In this paper, we provide the theoretical groundwork for scalability analyses of four highly relevant path diversity schemes. Based on this groundwork, we exemplarily predict the information load the schemes induce in a system of a large ISP. Generalizing the system-specific results, we give an outlook on the load that can be expected in comparable ASs. We found that for two schemes currently in the standardization process, scalability problems in large ASs as they are operated by ISPs seem likely

Automated Formal Analysis of Internet Routing Configurations

Today\u27s Internet interdomain routing protocol, the Border Gateway Protocol (BGP), is increasingly complicated and fragile due to policy misconfigurations by individual autonomous systems (ASes). To create provably correct networks, the past twenty years have witnessed, among many other efforts, advances in formal network modeling, system verification and testing, and point solutions for network management by formal reasoning. On the conceptual side, the formal models usually abstract away low-level details, specifying what are the correct functionalities but not how to achieve them. On the practical side, system verification of existing networked systems is generally hard, and system testing or simulation provide limited formal guarantees. This is known as a long standing challenge in network practice --- formal reasoning is decoupled from actual implementation. This thesis seeks to bridge formal reasoning and actual network implementation in the setting of the Border Gateway Protocol (BGP), by developing the Formally Verifiable Routing (FVR) toolkit that combines formal methods and programming language techniques. Starting from the formal model, FVR automates verification of routing models and the synthesis of faithful implementations that carries the correctness property. Conversely, starting from large real-world BGP systems with arbitrary policy configurations, automates the analysis of Internet routing configurations, and also includes a novel network reduction technique that scales up existing techniques for automated analysis. By developing the above formal theories and tools, this thesis aims to help network operators to create and manage BGP systems with correctness guarantee

Virtualization and Distribution of the BGP Control Plane

L'Internet est organisé sous la forme d'une multitude de réseaux appelés Systèmes Autonomes (AS). Le Border Gateway Protocol (BGP) est le langage commun qui permet à ces domaines administratifs de s'interconnecter. Grâce à BGP, deux utilisateurs situés n'importe où dans le monde peuvent communiquer, car ce protocole est responsable de la propagation des messages de routage entre tous les réseaux voisins. Afin de répondre aux nouvelles exigences, BGP a dû s'améliorer et évoluer à travers des extensions fréquentes et de nouvelles architectures. Dans la version d'origine, il était indispensable que chaque routeur maintienne une session avec tous les autres routeurs du réseau. Cette contrainte a soulevé des problèmes de scalabilité, puisque le maillage complet des sessions BGP internes (iBGP) était devenu difficile à réaliser dans les grands réseaux. Pour couvrir ce besoin de connectivité, les opérateurs de réseaux font appel à la réflection de routes (RR) et aux confédérations. Mais si elles résolvent un problème de scalabilité, ces deux solutions ont soulevé des nouveaux défis car elles sont accompagnées de multiples défauts; la perte de diversité des routes candidates au processus de sélection BGP ou des anomalies comme par exemple des oscillations de routage, des déflections et des boucles en font partie. Les travaux menés dans cette thèse se concentrent sur oBGP, une nouvelle architecture pour redistribuer les routes externes à l'intérieur d'un AS. `A la place des classiques sessions iBGP, un réseau de type overlay est responsable (I) de l'´echange d'informations de routage avec les autres AS, (II) du stockage distribué des routes internes et externes, (III) de l'application de la politique de routage au niveau de l'AS et (IV) du calcul et de la redistribution des meilleures routes vers les destinations de l'Internet pour tous les routeurs clients présents dans l'AS. ABSTRACT : The Internet is organized as a collection of networks called Autonomous Systems (ASes). The Border Gateway Protocol (BGP) is the glue that connects these administrative domains. Communication is thus possible between users worldwide and each network is responsible of sharing reachability information to peers through BGP. Protocol extensions are periodically added because the intended use and design of BGP no longer fit the current demands. Scalability concerns make the required internal BGP (iBGP) full mesh difficult to achieve in today's large networks and therefore network operators resort to confederations or Route Reflectors (RRs) to achieve full connectivity. These two options come with a set of flaws of their own such as route diversity loss, persistent routing oscillations, deflections, forwarding loops etc. In this dissertation we present oBGP, a new architecture for the redistribution of external routes inside an AS. Instead of relying on the usual statically configured set of iBGP sessions, we propose to use an overlay of routing instances that are collectively responsible for (I) the exchange of routes with other ASes, (II) the storage of internal and external routes, (III) the storage of the entire routing policy configuration of the AS and (IV) the computation and redistribution of the best routes towards Internet destinations to each client router in the AS

Proactive techniques for correct and predictable Internet routing

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, February 2006.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (p. 185-193).The Internet is composed of thousands of autonomous, competing networks that exchange reachability information using an interdomain routing protocol. Network operators must continually reconfigure the routing protocols to realize various economic and performance goals. Unfortunately, there is no systematic way to predict how the configuration will affect the behavior of the routing protocol or to determine whether the routing protocol will operate correctly at all. This dissertation develops techniques to reason about the dynamic behavior of Internet routing, based on static analysis of the router configurations, before the protocol ever runs on a live network. Interdomain routing offers each independent network tremendous flexibility in configuring the routing protocols to accomplish various economic and performance tasks. Routing configurations are complex, and writing them is similar to writing a distributed program; the (unavoidable) consequence of configuration complexity is the potential for incorrect and unpredictable behavior. These mistakes and unintended interactions lead to routing faults, which disrupt end-to-end connectivity. Network operators writing configurations make mistakes; they may also specify policies that interact in unexpected ways with policies in other networks.(cont.) To avoid disrupting network connectivity and degrading performance, operators would benefit from being able to determine the effects of configuration changes before deploying them on a live network; unfortunately, the status quo provides them no opportunity to do so. This dissertation develops the techniques to achieve this goal of proactively ensuring correct and predictable Internet routing. The first challenge in guaranteeing correct and predictable behavior from a routing protocol is defining a specification for correct behavior. We identify three important aspects of correctness-path visibility, route validity, and safety-and develop proactive techniques for guaranteeing that these properties hold. Path visibility states that the protocol disseminates information about paths in the topology; route validity says that this information actually corresponds to those paths; safety says that the protocol ultimately converges to a stable outcome, implying that routing updates actually correspond to topological changes. Armed with this correctness specification, we tackle the second challenge: analyzing routing protocol configurations that may be distributed across hundreds of routers.(cont.) We develop techniques to check whether a routing protocol satisfies the correctness specification within a single independently operated network. We find that much of the specification can be checked with static configuration analysis alone. We present examples of real-world routing faults and propose a systematic framework to classify, detect, correct, and prevent them. We describe the design and implementation of rcc ("router configuration checker"), a tool that uses static configuration analysis to enable network operators to debug configurations before deploying them in an operational network. We have used rcc to detect faults in 17 different networks, including several nationwide Internet service providers (ISPs). To date, rcc has been downloaded by over seventy network operators. A critical aspect of guaranteeing correct and predictable Internet routing is ensuring that the interactions of the configurations across multiple networks do not violate the correctness specification. Guaranteeing safety is challenging because each network sets its policies independently, and these policies may conflict. Using a formal model of today's Internet routing protocol, we derive conditions to guarantee that unintended policy interactions will never cause the routing protocol to oscillate.(cont.) This dissertation also takes steps to make Internet routing more predictable. We present algorithms that help network operators predict how a set of distributed router configurations within a single network will affect the flow of traffic through that network. We describe a tool based on these algorithms that exploits the unique characteristics of routing data to reduce computational overhead. Using data from a large ISP, we show that this tool correctly computes BGP routing decisions and has a running time that is acceptable for many tasks, such as traffic engineering and capacity planning.by Nicholas Greer Feamster.Ph.D

Virtualization and Distribution of the BGP Control Plane

The Internet is organized as a collection of networks called Autonomous Systems (ASes). The Border Gateway Protocol (BGP) is the glue that connects these administrative domains. Communication is thus possible between users worldwide and each network is responsible of sharing reachability information to peers through BGP. Protocol extensions are periodically added because the intended use and design of BGP no longer fit the current demands. Scalability concerns make the required internal BGP (iBGP) full mesh difficult to achieve in today's large networks and therefore network operators resort to confederations or Route Reflectors (RRs) to achieve full connectivity. These two options come with a set of flaws of their own such as route diversity loss, persistent routing oscillations, deflections, forwarding loops etc. In this dissertation we present oBGP, a new architecture for the redistribution of external routes inside an AS. Instead of relying on the usual statically configured set of iBGP sessions, we propose to use an overlay of routing instances that are collectively responsible for (I) the exchange of routes with other ASes, (II) the storage of internal and external routes, (III) the storage of the entire routing policy configuration of the AS and (IV) the computation and redistribution of the best routes towards Internet destinations to each client router in the AS

A Highly-Available Multiple Region Multi-access Edge Computing Platform with Traffic Failover

One of the main challenges in the Multi-access Edge Computing (MEC) is steering traffic from clients to the nearest MEC instances. If the nearest MEC fails, a failover mechanism should provide mitigation by steering the traffic to the next nearest MEC. There are two conventional approaches to solve this problem, i.e., GeoDNS and Internet Protocol (IP) anycast. GeoDNS is not failover friendly because of the Domain Name System (DNS) cache lifetime. Moreover, the use of a recursive resolver may inaccurately translate the IP address to its geolocation. Thus, this thesis studies and proposes a highly available MEC platform leveraging IP anycast. We built a proof-of-concept using Kubernetes, MetalLB, and a custom health-checker running on the GNS3 network emulator. We measured latency, failure percentage, and Mean Time To Repair (MTTR) to observe the system's behavior. The performance evaluation of the proposed solution shows an average recovery time better than one second. The number of failed requests and latency overhead grows linearly as the failover time and latency between two MECs increases. This thesis demonstrates the effectiveness of IP anycast for MEC applications to steer the traffic to the nearest MEC instance and to enhance resiliency with minor overhead

Strategies for internet route control: past, present and future

Uno de los problemas más complejos en redes de computadores es el de proporcionar garantías de calidad y confiabilidad a las comunicaciones de datos entre entidades que se encuentran en dominios distintos. Esto se debe a un amplio conjunto de razones -- las cuales serán analizadas en detalle en esta tesis -- pero de manera muy breve podemos destacar: i) la limitada flexibilidad que presenta el modelo actual de encaminamiento inter-dominio en materia de ingeniería de tráfico; ii) la naturaleza distribuida y potencialmente antagónica de las políticas de encaminamiento, las cuales son administradas individualmente y sin coordinación por cada dominio en Internet; y iii) las carencias del protocolo de encaminamiento inter-dominio utilizado en Internet, denominado BGP (Border Gateway Protocol).El objetivo de esta tesis, es precisamente el estudio y propuesta de soluciones que permitan mejorar drásticamente la calidad y confiabilidad de las comunicaciones de datos en redes conformadas por múltiples dominios.Una de las principales herramientas para lograr este fin, es tomar el control de las decisiones de encaminamiento y las posibles acciones de ingeniería de tráfico llevadas a cabo en cada dominio. Por este motivo, esta tesis explora distintas estrategias de como controlar en forma precisa y eficiente, tanto el encaminamiento como las decisiones de ingeniería de tráfico en Internet. En la actualidad este control reside principalmente en BGP, el cual como indicamos anteriormente, es uno de los principales responsables de las limitantes existentes. El paso natural sería reemplazar a BGP, pero su despliegue actual y su reconocida operatividad en muchos otros aspectos, resultan claros indicadores de que su sustitución (ó su posible evolución) será probablemente gradual. En este escenario, esta tesis propone analizar y contribuir con nuevas estrategias en materia de control de encaminamiento e ingeniería de tráfico inter-dominio en tres marcos temporales distintos: i) en la actualidad en redes IP; ii) en un futuro cercano en redes IP/MPLS (MultiProtocol Label Switching); y iii) a largo plazo en redes ópticas, modelando así una evolución progresiva y realista, facilitando el reemplazo gradual de BGP.Más concretamente, este trabajo analiza y contribuye mediante: - La propuesta de estrategias incrementales basadas en el Control Inteligente de Rutas (Intelligent Route Control, IRC) para redes IP en la actualidad. Las estrategias propuestas en este caso son de carácter incremental en el sentido de que interaccionan con BGP, solucionando varias de las carencias que éste presenta sin llegar a proponer aún su reemplazo. - La propuesta de estrategias concurrentes basadas en extender el concepto del PCE (Path Computation Element) proveniente del IETF (Internet Engineering Task Force) para redes IP/MPLS en un futuro cercano. Las estrategias propuestas en este caso son de carácter concurrente en el sentido de que no interaccionan con BGP y pueden ser desplegadas en forma paralela. En este caso, BGP continúa controlando el encaminamiento y las acciones de ingeniería de tráfico inter-dominio del tráfico IP, pero el control del tráfico IP/MPLS se efectúa en forma independiente de BGP mediante los PCEs.- La propuesta de estrategias que reemplazan completamente a BGP basadas en la incorporación de un nuevo agente de control, al cual denominamos IDRA (Inter-Domain Routing Agent). Estos agentes proporcionan un plano de control dedicado, físicamente independiente del plano de datos, y con gran capacidad computacional para las futuras redes ópticas multi-dominio.Los resultados expuestos aquí validan la efectividad de las estrategias propuestas, las cuales mejoran significativamente tanto la concepción como la performance de las actuales soluciones en el área de Control Inteligente de Rutas, del esperado PCE en un futuro cercano, y de las propuestas existentes para extender BGP al área de redes ópticas.One of the most complex problems in computer networks is how to provide guaranteed performance and reliability to the communications carried out between nodes located in different domains. This is due to several reasons -- which will be analyzed in detail in this thesis -- but in brief, this is mostly due to: i) the limited capabilities of the current inter-domain routing model in terms of Traffic Engineering (TE); ii) the distributed and potentially conflicting nature of policy-based routing, where routing policies are managed independently and without coordination among domains; and iii) the clear limitations of the inter-domain routing protocol, namely, the Border Gateway Protocol (BGP). The goal of this thesis is precisely to study and propose solutions allowing to drastically improve the performance and reliability of inter-domain communications. One of the most important tools to achieve this goal, is to control the routing and TE decisions performed by routing domains. Therefore, this thesis explores different strategies on how to control such decisions in a highly efficient and accurate way. At present, this control mostly resides in BGP, but as mentioned above, BGP is in fact one of the main causes of the existing limitations. The natural next-step would be to replace BGP, but the large installed base at present together with its recognized effectiveness in other aspects, are clear indicators that its replacement (or its possible evolution) will probably be gradually put into practice.In this framework, this thesis proposes to to study and contribute with novel strategies to control the routing and TE decisions of domains in three different time frames: i) at present in IP multi-domain networks; ii) in the near-future in IP/MPLS (MultiProtocol Label Switching) multi- domain networks; and iii) in the future optical Internet, modeling in this way a realistic and progressive evolution, facilitating the gradual replacement of BGP.More specifically, the contributions in this thesis can be summarized as follows. - We start by proposing incremental strategies based on Intelligent Route Control (IRC) solutions for IP networks. The strategies proposed in this case are incremental in the sense that they interact with BGP, and tackle several of its well-known limitations. - Then, we propose a set of concurrent route control strategies for MPLS networks, based on broadening the concept of the Path Computation Element (PCE) coming from the IETF (Internet Engineering Task Force). Our strategies are concurrent in the sense that they do not interact directly with BGP, and they can be deployed in parallel. In this case, BGP still controlls the routing and TE actions concerning regular IP-based traffic, but not how IP/MPLS paths are routed and controlled. These are handled independently by the PCEs.- We end with the proposal of a set of route control strategies for multi-domain optical networks, where BGP has been completely replaced. These strategies are supported by the introduction of a new route control element, which we named Inter-Domain Routing Agent (IDRA). These IDRAs provide a dedicated control plane, i.e., physically independent from the data plane, and with high computational capacity for future optical networks.The results obtained validate the effectiveness of the strategies proposed here, and confirm that our proposals significantly improve both the conception and performance of the current IRC solutions, the expected PCE in the near-future, as well as the existing proposals about the optical extension of BGP.Postprint (published version

Technologies, routing policies and relationships between autonomous systems in inter-domain routing

A deep exploration of the issues related to routing decisions in inter-domain routing is the scope of this thesis, through the analysis of the interconnection structure and the network hierarchy, the examination of the inter-domain routing protocol used to exchange network reachability information with other systems, the examination of the routing decision process between the entities according to their attributes and policies, the study of the topology generators of the AS relationships, reviewing the most interesting proposals in this area, describing why these issues are difficult to solve, and proposing solutions allowing to better understand the routing process and optimally solve the trade-off of implementing a Peering Engagement between two Autonomous Systems, against the extra cost that this solution represent. More specifically this thesis introduces a new scheme for the routing decision in a BGP speaker through a formalization of the routing decision process, and proposes a formulation of a real and exhaustive mathematical model of a Peering Engagement between Autonomous Systems, to be solved as a problem of maximization with an ad-hoc built Decision Support System (XESS) able to find an optimal reduced set of solutions to the proposed problem. -------------------------------------------------------------------------- ABSTRACT [IT] Un’analisi approfondita delle tematiche inerenti le decisioni di routing nel routing interdominio è oggetto di questa tesi, attraverso l’esame della struttura di interconnessione e delle gerarchia del network, lo studio del protocollo utilizzato nel routing interdominio per scambiare le informazioni di reachability con gli altri sistemi, l’analisi del processo decisionale tra le entità coinvolte nello scambio di tali informazioni in accordo con le politiche e gli attributi, lo studio delle topologie sintetiche derivate dallo studio delle relazioni tra gli AS, attraverso i lavori di ricerca in quest’area, la descrizione dei problemi e delle difficoltà, e offrendo un contributo atto a fornire una maggiore comprensione del processo decisionale nel routing interdominio e una soluzione per l’implementazione di un processo di Peering tra Autonomous System. In particolare, questa tesi introduce un nuovo modello per il processo decisionale in uno speaker BGP attraverso la formalizzazione del routing decision process, e propone un modello matematico esaustivo delle meccaniche legate al processo di Peering Engagement tra Autonomous System, da analizzare come problema di massimizzazione e da risolvere con un Decision Support System (XESS) creato per trovare un sottoinsieme ottimo di soluzioni al problema matematico proposto

Effective Wide-Area Network Performance Monitoring and Diagnosis from End Systems.

The quality of all network application services running on today’s Internet heavily depends on the performance assurance offered by the Internet Service Providers (ISPs). Large network providers inside the core of the Internet are instrumental in determining the network properties of their transit services due to their wide-area coverage, especially in the presence of the increasingly deployed real-time sensitive network applications. The end-to-end performance of distributed applications and network services are susceptible to network disruptions in ISP networks. Given the scale and complexity of the Internet, failures and performance problems can occur in different ISP networks. It is important to efficiently identify and proactively respond to potential problems to prevent large damage. Existing work to monitor and diagnose network disruptions are ISP-centric, which relying on each ISP to set up monitors and diagnose within its network. This approach is limited as ISPs are unwilling to revealing such data to the public. My dissertation research developed a light-weight active monitoring system to monitor, diagnose and react to network disruptions by purely using end hosts, which can help customers assess the compliance of their service-level agreements (SLAs). This thesis studies research problems from three indispensable aspects: efficient monitoring, accurate diagnosis, and effective mitigation. This is an essential step towards accountability and fairness on the Internet. To fully understand the limitation of relying on ISP data, this thesis first studies and demonstrates the monitor selection’s great impact on the monitoring quality and the interpretation of the results. Motivated by the limitation of ISP-centric approach, this thesis demonstrates two techniques to diagnose two types of finegrained causes accurately and scalably by exploring information across routing and data planes, as well as sharing information among multiple locations collaboratively. Finally, we demonstrate usefulness of the monitoring and diagnosis results with two mitigation applications. The first application is short-term prevention of avoiding choosing the problematic route by exploring the predictability from history. The second application is to scalably compare multiple ISPs across four important performance metrics, namely reachability, loss rate, latency, and path diversity completely from end systems without any ISP cooperation.Ph.D.Computer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/64770/1/wingying_1.pd