Исследование структурных свойств сети Интернет на основе метаграфовых моделей

Studying the Internet its structure is usually divided into levels: Autonomous Systems Level (AS), Point of Presence Level (PoP), Router Level, etc. The global network can be represented on each of them as a graph based on the initial data obtained from open sources. Consideration of a network within the framework of a separate level facilitates analysis, but does not allow to systematically assess its structural properties when providing the connectivity between several segments of the network related, particularly, to the objects of critical information infrastructure. To overcome this contradiction, a mathematical model of the global network in the form of a metagraph was developed at the interface between AS-level and PoP-level that takes into account the characteristics of each level and allows to find bottlenecks both in the interdomain routing system and in the topology of internal networks of Internet providers. Based on the proposed model some structural phenomena of the global network are described: stub, multihomed and transit autonomous systems, content providers. Taking into account available data from open sources about Internet structure, a method for constructing a metagraph is proposed. A comparative analysis of tools that automate the process of analyzing a network model is carried out. The practice-oriented problems of finding a cutting subset in a metagraph were set. Certain areas of further research are software implementation of the models using module MGtoolkit in Python and the assessment of structural phenomena of Russian segments of the Internet.При исследовании сети Интернет ее структуру разделяют на уровни: уровень автономных систем, уровень точек присутствия операторов связи, уровень оборудования и так далее. На каждом из них глобальная сеть может быть описана в виде графа на основании исходных данных, получаемых из открытых источников. Рассмотрение сети в рамках отдельного уровня упрощает анализ, однако не позволяет системно оценить ее структурные свойства при решении задач обеспечения связности нескольких сегментов сети, относящихся, в частности, к объектам критической информационной инфраструктуры. Для преодоления этого противоречия разработана математическая модель глобальной сети на стыке уровня автономных систем и уровня точек присутствия операторов связи в виде метаграфа, которая учитывает особенности каждого из уровней и позволяет находить «узкие» места как в системе междоменной маршрутизации, так и в топологии внутренних сетей интернет-провайдеров. На основе предложенной модели описаны некоторые структурные феномены глобальной сети: тупиковые, многоинтерфейсные и транзитные автономные системы, контент-провайдеры. С учетом доступных в открытых источниках данных о структуре сети Интернет предложен способ построения метаграфа. Проведен сравнительный анализ инструментов, автоматизирующих процесс анализа модели сети. Сформулированы ориентированные на практику задачи поиска разрезающего подмножества в метаграфе. Определены направления дальнейших исследований – программная реализация инструментов анализа структуры глобальной сети с использованием общедоступного модуля MGtoolkit на языке Python и оценивание структурных феноменов российского сегмента сети Интернет

Исследование структурных свойств сети Интернет на основе метаграфовых моделей

При исследовании сети Интернет ее структуру разделяют на уровни: уровень автономных систем, уровень точек присутствия операторов связи, уровень оборудования и так далее. На каждом из них глобальная сеть может быть описана в виде графа на основании исходных данных, получаемых из открытых источников. Рассмотрение сети в рамках отдельного уровня упрощает анализ, однако не позволяет системно оценить ее структурные свойства при решении задач обеспечения связности нескольких сегментов сети, относящихся, в частности, к объектам критической информационной инфраструктуры. Для преодоления этого противоречия разработана математическая модель глобальной сети на стыке уровня автономных систем и уровня точек присутствия операторов связи в виде метаграфа, которая учитывает особенности каждого из уровней и позволяет находить «узкие» места как в системе междоменной маршрутизации, так и в топологии внутренних сетей интернет-провайдеров. На основе предложенной модели описаны некоторые структурные феномены глобальной сети: тупиковые, многоинтерфейсные и транзитные автономные системы, контент-провайдеры. С учетом доступных в открытых источниках данных о структуре сети Интернет предложен способ построения метаграфа. Проведен сравнительный анализ инструментов, автоматизирующих процесс анализа модели сети. Сформулированы ориентированные на практику задачи поиска разрезающего подмножества в метаграфе. Определены направления дальнейших исследований – программная реализация инструментов анализа структуры глобальной сети с использованием общедоступного модуля MGtoolkit на языке Python и оценивание структурных феноменов российского сегмента сети Интернет

Systems for characterizing Internet routing

2018 Spring.Includes bibliographical references.Today the Internet plays a critical role in our lives; we rely on it for communication, business, and more recently, smart home operations. Users expect high performance and availability of the Internet. To meet such high demands, all Internet components including routing must operate at peak efficiency. However, events that hamper the routing system over the Internet are very common, causing millions of dollars of financial loss, traffic exposed to attacks, or even loss of national connectivity. Moreover, there is sparse real-time detection and reporting of such events for the public. A key challenge in addressing such issues is lack of methodology to study, evaluate and characterize Internet connectivity. While many networks operating autonomously have made the Internet robust, the complexity in understanding how users interconnect, interact and retrieve content has also increased. Characterizing how data is routed, measuring dependency on external networks, and fast outage detection has become very necessary using public measurement infrastructures and data sources. From a regulatory standpoint, there is an immediate need for systems to detect and report routing events where a content provider's routing policies may run afoul of state policies. In this dissertation, we design, build and evaluate systems that leverage existing infrastructure and report routing events in near-real time. In particular, we focus on geographic routing anomalies i.e., detours, routing failure i.e., outages, and measuring structural changes in routing policies

BGP-Multipath Routing in the Internet

BGP-Multipath, or BGP-M, is a routing technique for balancing traffic load in the Internet. It enables a Border Gateway Protocol (BGP) border router to install multiple ‘equally-good’ paths to a destination prefix. While other multipath routing techniques are deployed at internal routers, BGP-M is deployed at border routers where traffic is shared on multiple border links between Autonomous Systems (ASes). Although there are a considerable number of research efforts on multipath routing, there is so far no dedicated measurement or study on BGP-M in the literature. This thesis presents the first systematic study on BGP-M. I proposed a novel approach to inferring the deployment of BGP-M by querying Looking Glass (LG) servers. I conducted a detailed investigation on the deployment of BGP-M in the Internet. I also analysed BGP-M’s routing properties based on traceroute measurements using RIPE Atlas probes. My research has revealed that BGP-M has already been used in the Internet. In particular, Hurricane Electric (AS6939), a Tier-1 network operator, has deployed BGP-M at border routers across its global network to hundreds of its neighbour ASes on both IPv4 and IPv6 Internet. My research has provided the state-of-the-art knowledge and insights in the deployment, configuration and operation of BGP-M. The data, methods and analysis introduced in this thesis can be immensely valuable to researchers, network operators and regulators who are interested in improving the performance and security of Internet routing. This work has raised awareness of BGP-M and may promote more deployment of BGP-M in future because BGP-M not only provides all benefits of multipath routing but also has distinct advantages in terms of flexibility, compatibility and transparency

A collaborative P2P Scheme for NAT Traversal Server discovery based on topological information

In the current Internet picture more than 70% of the hosts are located behind Network Address Translators (NATs). This is not a problem for the client/server paradigm. However, the Internet has evolved, and nowadays the largest portion of the traffic is due to peer-to-peer (p2p) applications. This scenario presents an important challenge: two hosts behind NATs (NATed hosts) cannot establish direct communications. The easiest way to solve this problem is by using a third entity, called Relay, that forwards the traffic between the NATed hosts. Although many efforts have been devoted to avoid the use of Relays, they are still needed in many situations. Hence, the selection of a suitable Relay becomes critical to many p2p applications. In this paper, we propose the Gradual Proximity Algorithm (GPA): a simple algorithm that guarantees the selection of a topologically close-by Relay. We present a measurement-based analysis, showing that the GPA minimizes both the delay of the relayed communication and the transit traffic generated by the Relay, being a QoS-aware and ISP-friendly solution. Furthermore, the paper presents the Peer-to-Peer NAT Traversal Architecture (P2P-NTA), which is a global, distributed and collaborative solution, based on the GPA. This architecture addresses the Relay discovery/selection problem. We have performed large-scale simulations based on real measurements, which validate our proposal. The results demonstrate that the P2P-NTA performs similarly to direct communications with reasonably large deployments of p2p applications. In fact, only 5% of the communications experience an extra delay that may degrade the QoS due to the use of Relays. Furthermore, the amount of extra transit traffic generated is only 6%. We also show that the P2P-NTA largely outperforms other proposals, where the QoS degradation affects up to more than 50% of the communications, and the extra traffic generated goes beyond 80%.This work has been partially funded by the Grants MEDIANET (S2009/TIC-1466) from the Regional Government of Madrid and CON-PARTE (TEC2007-67966-C03- 03) by the Ministry of Science and Innovation of Spain.Publicad

Hermes: Distributed social network monitoring system

[ANGLÈS] Nowadays, social network services play a very important role in the way people interact with each other and with the world. This generates big amounts of data that can be used to study social relationships and extract useful information about preferences and trends. When analysing this information, two main problems emerge: The need to aggregate dif- ferent data coming from multiple sources, and hardware limitations due to the incapability traditional systems have to deal with large amounts of data. In order to solve the problems mentioned before, this project aims to implement a distributed, scalable social media analysis tool, ready to connect and gather data from multiple sources and show the aggregated results in real-time.[CASTELLÀ] Hoy en día, las redes sociales juegan un papel muy importante en la manera como las personas interactúa entre ellos y con el mundo. Esto genera grandes volúmenes de inforación que pueden ser utilizados para estudiar las relaciones sociales y extraer información útil acerca de gustos y tendencias. Cuando se analiza esta información, surgen dos problemas principales: La necesidad de agregar diferentes datos provenientes de múltiples fuentes, y las limitaciones hardware por la incapacidad de los sistemas tradicionales de manejar grandes cantidades de datos. Para poder solventar estos problemas, este proyecto propone implementar una herramienta de análisis de redes sociales distribuida y escalable, preparada para conectarse y recolectar datos de múltiples fuentes y mostrar los resultados agregados en tiempo real.[CATALÀ] Avui en dia, les xarxes socials juguen un paper molt important en la manera com les persones interactua entre ells i amb el mon. Això genera grans quantitats de dades que poden ser utilitzats per estudiar les relacions socials i extreure informació útil sobre gustos i tendències. 2 Quan s’analitza aquesta informació, sorgeixen dos problemes principals: La necessitat de agregar diferents dades provinents de múltiples fonts, i les limitacions hardware per la incapaci- tat dels sistemes tradicionals de gestionar grans quantitats de dades. Per poder solucionar aque- sts problemes, aquest projecte proposa implementar una eina d’anàlisi de xarxes socials dis- tribuïda i escalable, preparada per connectar-se i recol·lectar dades de múltiples fonts i mostrar els resultats agregats en temps real

Improving the Accuracy of the Internet Cartography

As the global Internet expands to satisfy the demands of the ever-increasing connected population, profound changes are occurring in its interconnection structure. The pervasive growth of IXPs and CDNs, two initially independent but synergistic infrastructure sectors, have contributed to the gradual flattening of the Internet’s inter-domain hierarchy with primary routing paths shifting from backbone networks to peripheral peering links. At the same time the IPv6 deployment has taken off due to the depletion of unallocated IPv4 addresses. These fundamental changes in Internet dynamics has obvious implications for network engineering and operations, which can be benefited by accurate topology maps to understand the properties of this critical infrastructure. This thesis presents a set of new measurement techniques and inference algorithms to construct a new type of semantically rich Internet map, and improve the state of the art in Internet cartography. The author first develops a methodology to extract large-scale validation data from the Communities BGP attribute, which encodes rich routing meta-data on BGP messages. Based on this better-informed dataset the author proceeds to analyse popular assumptions about inter-domain routing policies and devise a more accurate model to describe inter-AS business relationships. Accordingly, the thesis proposes a new relationship inference algorithm to accurately capture both simple and complex AS relationships across two dimensions: prefix type, and geographic location. Validation against three sources of ground-truth data reveals that the proposed algorithm achieves a near-perfect accuracy. However, any inference approach is constrained by the inability of the existing topology data sources to provide a complete view of the inter-domain topology. To limit the topology incompleteness problem the author augments traditional BGP data with routing policy data obtained directly from IXPs to discover massive peering meshes which have thus far been largely invisible

Damage Detection and Mitigation in Open Collaboration Applications

Collaborative functionality is changing the way information is amassed, refined, and disseminated in online environments. A subclass of these systems characterized by open collaboration uniquely allow participants to *modify* content with low barriers-to-entry. A prominent example and our case study, English Wikipedia, exemplifies the vulnerabilities: 7%+ of its edits are blatantly unconstructive. Our measurement studies show this damage manifests in novel socio-technical forms, limiting the effectiveness of computational detection strategies from related domains. In turn this has made much mitigation the responsibility of a poorly organized and ill-routed human workforce. We aim to improve all facets of this incident response workflow. Complementing language based solutions we first develop content agnostic predictors of damage. We implicitly glean reputations for system entities and overcome sparse behavioral histories with a spatial reputation model combining evidence from multiple granularity. We also identify simple yet indicative metadata features that capture participatory dynamics and content maturation. When brought to bear over damage corpora our contributions: (1) advance benchmarks over a broad set of security issues ( vandalism ), (2) perform well in the first anti-spam specific approach, and (3) demonstrate their portability over diverse open collaboration use cases. Probabilities generated by our classifiers can also intelligently route human assets using prioritization schemes optimized for capture rate or impact minimization. Organizational primitives are introduced that improve workforce efficiency. The whole of these strategies are then implemented into a tool ( STiki ) that has been used to revert 350,000+ damaging instances from Wikipedia. These uses are analyzed to learn about human aspects of the edit review process, properties including scalability, motivation, and latency. Finally, we conclude by measuring practical impacts of work, discussing how to better integrate our solutions, and revealing outstanding vulnerabilities that speak to research challenges for open collaboration security