Migration of networks in multi-cloud environment

Tese de mestrado, Engenharia Informática (Arquitetura, Sistemas e Redes de Computadores) Universidade de Lisboa, Faculdade de Ciências, 2018A forma como os centros de dados e os recursos computacionais são geridos tem vindo a mudar. O uso exclusivo de servidores físicos e os complexos processos para provisionamento de software são já passado, sendo agora possível e simples usar recursos de uma terceira parte a pedido, na nuvem (cloud). A técnica central que permitiu esta evolução foi a virtualização, uma abstração dos recursos computacionais que torna o software mais independente do hardware em que é executado. Os avanços tecnológicos nesta área permitiram a migração de máquinas virtuais, agilizando ainda mais os processos de gestão e manutenção de recursos. A possibilidade de migrar máquinas virtuais libertou o software da infraestrutura física, facilitando uma série de tarefas como manutenção, balanceamento de carga, tratamento de faltas, entre outras. Hoje em dia a migração de máquinas virtuais é uma ferramenta essencial para gerir clouds, tanto públicas como privadas. Os sistemas informáticos de grande escala existentes na cloud são complexos, compostos por múltiplas partes que trabalham em conjunto para atingir os seus objectivos. O facto de os sistemas estarem intimamente ligados coloca pressão nos sistemas de comunicação e nas redes que os suportam. Esta dependência do sistema na infraestrutura de comunicação vem limitar a flexibilidade da migração de máquinas virtuais. Isto porque actualmente a gestão de uma rede é pouco flexível, limitando por exemplo a migração de VMs a uma subrede ou obrigando a um processo de reconfiguração de rede para a migração, um processo difícil, tipicamente manual e sujeito a falhas. Idealmente, a infraestrutura de que as máquinas virtuais necessitam para comunicar seria também virtual, permitindo migrar tanto as máquinas virtuais como a rede virtual. Abstrair os recursos de comunicação permitiria que todo o sistema tivesse a flexibilidade de ser transferido para outro local. Neste sentido foi recentemente proposta a migração de redes usando redes definidas por software (SDN), um novo paradigma que separa a infraestrutura de encaminhamento (plano de dados) do plano de controlo. Numa SDN a responsabilidade de tomar as decisões de controlo fica delegada num elemento logicamente centralizado, o controlador, que tem uma visão global da rede e do seu estado. Esta separação do plano de controlo do processo de encaminhamento veio facilitar a virtualização de redes. No entanto, as recentes propostas de virtualização de redes usando SDN apresentam limitações. Nomeadamente, estas soluções estão limitadas a um único centro de dados ou provedor de serviços. Esta dependência é um problema. Em primeiro lugar, confiar num único provedor ou cloud limita a disponibilidade, tornando efectivamente o provedor num ponto de falha único. Em segundo lugar, certos serviços ficam severamente limitados por recorrerem apenas a uma cloud, devido a requisitos especiais (de privacidade, por exemplo) ou mesmo legais (que podem obrigar a que, por exemplo, dados de utilizadores fiquem guardados no próprio país). Idealmente, seria possível ter a possibilidade de tirar partido de múltiplas clouds e poder, de forma transparente, aproveitar as vantagens de cada uma delas (por exemplo, umas por apresentarem custos mais reduzidos, outras pela sua localização). Tal possibilidade garantiria uma maior disponibilidade, visto que a falha de uma cloud não comprometeria todo o sistema. Além disso, poderia permitir baixar os custos porque seria possível aproveitar a variação dos preços existente entre clouds ao longo do tempo. Neste contexto multi-cloud um dos grandes desafios é conseguir migrar recursos entre clouds de forma a aproveitar os recursos existentes. Num ambiente SDN, em particular, a migração de redes é problemática porque é necessario que o controlador comunique com os elementos físicos da rede para implementar novas políticas e para que estes possam informar o controlador de novos eventos. Se a capacidade de comunicação entre o controlador e os elementos de rede for afectada (por exemplo, devido a latências elevadas de comunicação) o funcionamento da rede é também afectado. O trabalho que aqui propomos tem como objectivo desenvolver algoritmos de orquestração para migração de redes virtuais, com o objectivo de minimizar as latências na comunicação controlador-switches, em ambientes multi-cloud. Para esse efeito foi desenvolvida uma solução óptima, usando programação linear, e várias heurísticas. A solução de programação linear, sendo óptima, resulta na menor disrupção possível da ligação ao controlador. No entanto, a complexidade computacional desta solução limita a sua usabilidade, levando a tempos de execução elevados. Por esta razão são prospostas heurísticas que visam resolver o problema em tempo útil e de forma satisfatória. Os resultados das nossas experiências mostram que nas várias topologias testadas algumas heurísticas conseguem resultados próximos da solução óptima. O objectivo é atingido com tempos de execução consideravelmente inferiores.The way datacenters and computer resources are managed has been changing, from bare metal servers and complex deployment processes to on-demand cloud resources and applications. The main technology behind this evolution was virtualization. By abstracting the hardware, virtualization decoupled software from the hardware it runs on. Virtual machine (VM) migration further increased the flexibility of management and maintenance procedures. Tasks like maintenance, load balancing and fault handling were made easier. Today, the migration of virtual machines is a fundamental tool in public and private clouds. However as VMs rarely act alone, when the VMs migrate, the virtual networks should migrate too. Solutions to this problem using traditional networks have several limitations: they are integrated with the devices and are hard to manage. For these reasons the logical centralisation offered by Software-Defined Networking (SDN) architectures has been shown recently as an enabler for transparent migration of networks. In an SDN a controller remotely controls the network switches by installing flow rules that implement the policies defined by the network operator. Recent proposals are a good step forward but have problems. Namely, they are limited to a single data center or provider. The user’s dependency on a single cloud provider is a fundamental limitation. A large number of incidents involving accidental and malicious faults in cloud infrastructures show that relying on a single provider can lead to the creation of internet-scale single points of failures for cloud-based services. Furthermore, giving clients the power to choose how to use their cloud resources and the flexibility to easily change cloud providers is of great value, enabling clients to lower costs, tolerate cloud-wide outages and enhance security. The objective of this dissertation is therefore to design, implement and evaluate solutions for network migration in an environment of multiple clouds. The main goal is to schedule the migration of a network in such a way that the migration process has the least possible impact on the SDN controller’s ability to manage the network. This is achieved by creating a migration plan that aims to minimize the experienced control plane latency (i.e., the latency between the controller and the switches). We have developed an optimal solution based on a linear program, and several heuristics. Our results show that it is possible to achieve results close to the optimal solution, within reasonable time frames

A proposal for secured, efficient and scalable layer 2 network virtualisation mechanism

El contenidos de los capítulos 3 y 4 está sujeto a confidencialidad. 291 p.La Internet del Futuro ha emergido como un esfuerzo investigador para superar estas limitaciones identificadas en la actual Internet. Para ello es necesario investigar en arquitecturas y soluciones novedosas (evolutivas o rompedoras), y las plataformas de experimentación surgen para proporcionar un entorno realista para validar estas nuevas propuestas a gran escala.Debido a la necesidad de compartir la misma infraestructura y recursos para testear simultáneamente diversas propuestas de red, la virtualización de red es la clave del éxito. Se propone una nueva taxonomía para poder analizar y comparar las diferentes propuestas. Se identifican tres tipos: el Nodo Virtual (vNode), la Virtualización posibilitada por SDN (SDNeV) y el overlay.Además, se presentan las plataformas experimentales más relevantes, con un foco especial en la forma en la que cada una de ellas permite la investigación en propuestas de red, las cuales no cumplen todos estos requisitos impuestos: aislamiento, seguridad, flexibilidad, escalabilidad, estabilidad, transparencia, soporte para la investigación en propuestas de red. Por lo tanto, una nueva plataforma de experimentación ortogonal a la experimentación es necesaria.Las principales contribuciones de esta tesis, sustentadas sobre tecnología SDN y NFV, son también los elementos clave para construir la plataforma de experimentación: la Virtualización de Red basada en Prefijos de Nivel 2 (Layer 2 Prefix-based Network Virtualisation, L2PNV), un Protocolo para la Configuración de Direcciones MAC (MAC Address Configuration Protocol, MACP), y un sistema de Control de Acceso a Red basado en Flujos (Flow-based Network Access Control, FlowNAC).Como resultado, se ha desplegado en la Universidad del Pais Vasco (UPV/EHU) una nueva plataforma experimental, la Plataforma Activada por OpenFlow de EHU (EHU OpenFlow Enabled Facility, EHU-OEF), para experimentar y validar estas propuestas realizadas

An outright open source approach for simple and pragmatic internet eXchange

L'Internet, le réseaux des réseaux, est indispensable à notre vie moderne et mondialisée et en tant que ressource publique il repose sur l'inter opérabilité et la confiance. Les logiciels libres et open source jouent un rôle majeur pour son développement. Les points d'échange Internet (IXP) où tous les opérateurs de type et de taille différents peuvent s'échanger du trafic sont essentiels en tant que lieux d'échange neutres et indépendants. Le service fondamental offert par un IXP est une fabrique de commutation de niveau 2 partagée. Aujourd'hui les IXP sont obligés d'utiliser des technologies propriétaires pour leur fabrique de commutations. Bien qu'une fabrique de commutations de niveau 2 se doit d'être une fonctionnalité de base, les solutions actuelles ne répondent pas correctement aux exigences des IXPs. Cette situation est principalement dûe au fait que les plans de contrôle et de données sont intriqués sans possibilités de programmer finement le plan de commutation. Avant toute mise en œuvre, il est primordial de tester chaque équipement afin de vérifier qu'il répond aux attentes mais les solutions de tests permettant de valider les équipements réseaux sont toutes non open source, commerciales et ne répondent pas aux besoins techniques d'indépendance et de neutralité. Le "Software Defined Networking" (SDN), nouveau paradigme découplant les plans de contrôle et de données utilise le protocole OpenFlow qui permet de programmer le plan de commutation Ethernet haute performance. Contrairement à tous les projets de recherches qui centralisent la totalité du plan de contrôle au dessus d'OpenFlow, altérant la stabilité des échanges, nous proposons d'utiliser OpenFlow pour gérer le plan de contrôle spécifique à la fabrique de commutation. L'objectif principal de cette thèse est de proposer "Umbrella", fabrique de commutation simple et pragmatique répondant à toutes les exigences des IXPs et en premier lieu à la garantie d'indépendance et de neutralité des échanges. Dans la première partie, nous présentons l'architecture "Umbrella" en détail avec l'ensemble des tests et validations démontrant la claire séparation du plan de contrôle et du plan de données pour augmenter la robustesse, la flexibilité et la fiabilité des IXPs. Pour une exigence d'autonomie des tests nécessaires pour les IXPs permettant l'examen de la mise en œuvre d'Umbrella et sa validation, nous avons développé l'"Open Source Network Tester" (OSNT), un système entièrement open source "hardware" de génération et de capture de trafic. OSNT est le socle pour l"OpenFLow Operations Per Second Turbo" (OFLOPS Turbo), la plate-forme d'évaluation de commutation OpenFlow. Le dernier chapitre présente le déploiement de l'architecture "Umbrella" en production sur un point d'échange régional. Les outils de test que nous avons développés ont été utilisés pour vérifier les équipements déployés en production. Ce point d'échange, stable depuis maintenant un an, est entièrement géré et contrôlé par une seule application Web remplaçant tous les systèmes complexes et propriétaires de gestion utilisés précédemment.In almost everything we do, we use the Internet. The Internet is indispensable for our today's lifestyle and to our globalized financial economy. The global Internet traffic is growing exponentially. IXPs are the heart of Internet. They are highly valuable for the Internet as neutral exchange places where all type and size of autonomous systems can "peer" together. The IXPs traffic explode. The 2013 global Internet traffic is equivalent with the largest european IXP today. The fundamental service offer by IXP is a shared layer2 switching fabric. Although it seems a basic functionality, today solutions never address their basic requirements properly. Today networks solutions are inflexible as proprietary closed implementation of a distributed control plane tight together with the data plane. Actual network functions are unmanageable and have no flexibility. We can understand how IXPs operators are desperate reading the EURO-IX "whishlist" of the requirements who need to be implemented in core Ethernet switching equipments. The network vendor solutions for IXPs based on MPLS are imperfect readjustment. SDN is an emerging paradigm decoupling the control and data planes, on opening high performance forwarding plane with OpenFlow. The aims of this thesis is to propose an IXP pragmatic Openflow switching fabric, addressing the critical requirements and bringing more flexibility. Transparency is better for neutrality. IXPs needs a straightforward more transparent layer2 fabric where IXP participants can exchange independently their traffic. Few SDN solutions have been presented already but all of them are proposing fuzzy layer2 and 3 separation. For a better stability not all control planes functions can be decoupled from the data plane. As other goal statement, networking testing tools are essential for qualifying networking equipment. Most of them are software based and enable to perform at high speed with accuracy. Moreover network hardware monitoring and testing being critical for computer networks, current solutions are both extremely expensive and inflexible. The experience in deploying Openflow in production networks has highlight so far significant limitations in the support of the protocol by hardware switches. We presents Umbrella, a new SDN-enabled IXP fabric architecture, that aims at strengthening the separation of control and data plane to increase both robustness, flexibility and reliability of the exchange. Umbrella abolish broadcasting with a pseudo wire and segment routing approach. We demonstrated for an IXP fabric not all the control plane can be decoupled from the date plane. We demonstrate Umbrella can scale and recycle legacy non OpenFlow core switch to reduce migration cost. Into the testing tools lacuna we launch the Open Source Network Tester (OSNT), a fully open-source traffic generator and capture system. Additionally, our approach has demonstrated lower-cost than comparable commercial systems while achieving comparable levels of precision and accuracy; all within an open-source framework extensible with new features to support new applications, while permitting validation and review of the implementation. And we presents the integration of OpenFLow Operations Per Second (OFLOPS), an OpenFlow switch evaluation platform, with the OSNT platform, a hardware-accelerated traffic generation and capturing platform. What is better justification than a real deployment ? We demonstrated the real flexibility and benefit of the Umbrella architecture persuading ten Internet Operators to migrate the entire Toulouse IXP. The hardware testing tools we have developed have been used to qualify the hardware who have been deployed in production. The TouIX is running stable from a year. It is fully managed and monitored through a single web application removing all the legacy complex management systems

On the Edge of Secure Connectivity via Software-Defined Networking

Securing communication in computer networks has been an essential feature ever since the Internet, as we know it today, was started. One of the best known and most common methods for secure communication is to use a Virtual Private Network (VPN) solution, mainly operating with an IP security (IPsec) protocol suite originally published in 1995 (RFC1825). It is clear that the Internet, and networks in general, have changed dramatically since then. In particular, the onset of the Cloud and the Internet-of-Things (IoT) have placed new demands on secure networking. Even though the IPsec suite has been updated over the years, it is starting to reach the limits of its capabilities in its present form. Recent advances in networking have thrown up Software-Defined Networking (SDN), which decouples the control and data planes, and thus centralizes the network control. SDN provides arbitrary network topologies and elastic packet forwarding that have enabled useful innovations at the network level. This thesis studies SDN-powered VPN networking and explains the benefits of this combination. Even though the main context is the Cloud, the approaches described here are also valid for non-Cloud operation and are thus suitable for a variety of other use cases for both SMEs and large corporations. In addition to IPsec, open source TLS-based VPN (e.g. OpenVPN) solutions are often used to establish secure tunnels. Research shows that a full-mesh VPN network between multiple sites can be provided using OpenVPN and it can be utilized by SDN to create a seamless, resilient layer-2 overlay for multiple purposes, including the Cloud. However, such a VPN tunnel suffers from resiliency problems and cannot meet the increasing availability requirements. The network setup proposed here is similar to Software-Defined WAN (SD-WAN) solutions and is extremely useful for applications with strict requirements for resiliency and security, even if best-effort ISP is used. IPsec is still preferred over OpenVPN for some use cases, especially by smaller enterprises. Therefore, this research also examines the possibilities for high availability, load balancing, and faster operational speeds for IPsec. We present a novel approach involving the separation of the Internet Key Exchange (IKE) and the Encapsulation Security Payload (ESP) in SDN fashion to operate from separate devices. This allows central management for the IKE while several separate ESP devices can concentrate on the heavy processing. Initially, our research relied on software solutions for ESP processing. Despite the ingenuity of the architectural concept, and although it provided high availability and good load balancing, there was no anti-replay protection. Since anti-replay protection is vital for secure communication, another approach was required. It thus became clear that the ideal solution for such large IPsec tunneling would be to have a pool of fast ESP devices, but to confine the IKE operation to a single centralized device. This would obviate the need for load balancing but still allow high availability via the device pool. The focus of this research thus turned to the study of pure hardware solutions on an FPGA, and their feasibility and production readiness for application in the Cloud context. Our research shows that FPGA works fluently in an SDN network as a standalone IPsec accelerator for ESP packets. The proposed architecture has 10 Gbps throughput, yet the latency is less than 10 µs, meaning that this architecture is especially efficient for data center use and offers increased performance and latency requirements. The high demands of the network packet processing can be met using several different approaches, so this approach is not just limited to the topics presented in this thesis. Global network traffic is growing all the time, so the development of more efficient methods and devices is inevitable. The increasing number of IoT devices will result in a lot of network traffic utilising the Cloud infrastructures in the near future. Based on the latest research, once SDN and hardware acceleration have become fully integrated into the Cloud, the future for secure networking looks promising. SDN technology will open up a wide range of new possibilities for data forwarding, while hardware acceleration will satisfy the increased performance requirements. Although it still remains to be seen whether SDN can answer all the requirements for performance, high availability and resiliency, this thesis shows that it is a very competent technology, even though we have explored only a minor fraction of its capabilities

Performance Optimization and Dynamics Control for Large-scale Data Transfer in Wide-area Networks

Transport control plays an important role in the performance of large-scale scientific and media streaming applications involving transfer of large data sets, media streaming, online computational steering, interactive visualization, and remote instrument control. In general, these applications have two distinctive classes of transport requirements: large-scale scientific applications require high bandwidths to move bulk data across wide-area networks, while media streaming applications require stable bandwidths to ensure smooth media playback. Unfortunately, the widely deployed Transmission Control Protocol is inadequate for such tasks due to its performance limitations. The purpose of this dissertation is to conduct rigorous analytical study of the design and performance of transport solutions, and develop an integrated transport solution in a systematical way to overcome the limitations of current transport methods. One of the primary challenges is to explore and compose a set of feasible route options with multiple constraints. Another challenge essentially arises from the randomness inherent in wide-area networks, particularly the Internet. This randomness must be explicitly accounted for to achieve both goodput maximization and stabilization over the constructed routes by suitably adjusting the source rate in response to both network and host dynamics.The superior and robust performance of the proposed transport solution is extensively evaluated in a simulated environment and further verified through real-life implementations and deployments over both Internet and dedicated connections under disparate network conditions in comparison with existing transport methods

Proposta de arquitetura para provedores de serviços de redes fixas e móveis utilizando SDN OpenFlow

Tese (doutorado)—Universidade de Brasília, Faculdade de Tecnologia, Departamento de Engenharia Elétrica, 2019.As redes de grande porte, como Provedores de Serviço, são arquiteturas robustas, capazes de dar suporte a grandes volumes de tráfego com características muito diferentes. Seus equipamentos dão suporte a cargas elevadas de processamento e ao mesmo tempo, são responsáveis por construir a lógica de roteamento e por encaminhar o tráfego. Por terem o controle implementado de forma distribuída e por serem construídas com equipamentos de um limitado número de fabricantes, estas redes apresentam limitações de controle e engenharia de tráfego, dificultando assim, a diferenciação entre os serviços que os diversos provedores fornecem. Adicionalmente, a inteligência da rede está oculta nos equipamentos, tornando as inovações muito lentas e amarradas aos interesses dos fabricantes. Como alternativa a este cenário, este trabalho propõe uma arquitetura de rede SDN-OpenFlow para redes móveis e redes de transporte que tenta solucionar os problemas previamente mencionados, bem como os inconvenientes da característica centralizadora que o OpenFlow possui. É apresentada uma arquitetura de rede OpenFlow robusta, capaz de dar suporte a tempos de resposta elevados e a quedas do Controlador, sem adição de tempos de espera no estabelecimento de novos fluxos e com significativa redução na carga submetida ao Controlador. Como prova de conceito, é implementado um protótipo utilizando o OpenvSwitch como software para a virtualização dos clientes OpenFlow, o Mininet para a criação da topologia e o Ryu como Controlador, todos com suporte OpenFlow 1.3. ou superior.Large scale networks, such as Service Providers, are robust architectures, capable of supporting large volumes of traffic with very different characteristics. Their network equipment have significant processing load, being responsible for building both a routing logic and a routing traffic at the same time. By having the network control implemented in a distributed manner and being built with a limited number of vendors, these networks have limitations of control and traffic engineering, hindering the differentiation between Service Providers. Additionally, the network intelligence is hidden in the network equipment, making the innovations very slow and conditioned to the vendors interests. As an alternative option, this work proposes an SDN-OpenFlow network architecture to transport and mobile networks that tries to improve the previously mentioned problems, and at the same time solves the arising difficulties related to the SDN network centralizing feature. With the proposed architecture, a robust OpenFlow network is created to support high Controller response times and Controller shut down, without additional delays in the creation of flows and with significant reduction of Controller’s load. A prototype has been constructed using Open vSwitch as a virtualization software for OpenFlow clients, Mininet for the topology construction and Ryu as the Controller, all with OpenFlow 1.3 support or higher

Journal of Telecommunications and Information Technology, nr 3

kwartalni