A Logically Centralized Approach for Control and Management of Large Computer Networks

Management of large enterprise and Internet Service Provider networks is a complex, error-prone, and costly challenge. It is widely accepted that the key contributors to this complexity are the bundling of control and data forwarding in traditional routers and the use of fully distributed protocols for network control. To address these limitations, the networking research community has been pursuing the vision of simplifying the functional role of a router to its primary task of packet forwarding. This enables centralizing network control at a decision plane where network-wide state can be maintained, and network control can be centrally and consistently enforced. However, scalability and fault-tolerance concerns with physical centralization motivate the need for a more flexible and customizable approach. This dissertation is an attempt at bridging the gap between the extremes of distribution and centralization of network control. We present a logically centralized approach for the design of network decision plane that can be realized by using a set of physically distributed controllers in a network. This approach is aimed at giving network designers the ability to customize the level of control and management centralization according to the scalability, fault-tolerance, and responsiveness requirements of their networks. Our thesis is that logical centralization provides a robust, reliable, and efficient paradigm for management of large networks and we present several contributions to prove this thesis. For network planning, we describe techniques for optimizing the placement of network controllers and provide guidance on the physical design of logically centralized networks. For network operation, algorithms for maintaining dynamic associations between the decision plane and network devices are presented, along with a protocol that allows a set of network controllers to coordinate their decisions, and present a unified interface to the managed network devices. Furthermore, we study the trade-offs in decision plane application design and provide guidance on application state and logic distribution. Finally, we present results of extensive numerical and simulative analysis of the feasibility and performance of our approach. The results show that logical centralization can provide better scalability and fault-tolerance while maintaining performance similarity with traditional distributed approach

A consistent and fault-tolerant data store for software defined networks

Tese de mestrado em Segurança Informática, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2013O sucesso da Internet é indiscutível. No entanto, desde há muito tempo que são feitas sérias críticas à sua arquitectura. Investigadores acreditam que o principal problema dessa arquitectura reside no facto de os dispositivos de rede incorporarem funções distintas e complexas que vão além do objectivo de encaminhar pacotes, para o qual foram criados [1]. O melhor exemplo disso são os protocolos distribuídos (e complexos) de encaminhamento, que os routers executam de forma a conseguir garantir o encaminhamento de pacotes. Algumas das consequências disso são a complexidade das redes tradicionais tanto em termos de inovação como de manutenção. Como resultado, temos redes dispendiosas e pouco resilientes. De forma a resolver este problema uma arquitectura de rede diferente tem vindo a ser adoptada, tanto pela comunidade científica como pela indústria. Nestas novas redes, conhecidas como Software Defined Networks (SDN), há uma separação física entre o plano de controlo do plano de dados. Isto é, toda a lógica e estado de controlo da rede é retirada dos dispositivos de rede, para passar a ser executada num controlador logicamente centralizado que com uma visão global, lógica e coerente da rede, consegue controlar a mesma de forma dinâmica. Com esta delegação de funções para o controlador os dispositivos de rede podem dedicar-se exclusivamente à sua função essencial de encaminhar pacotes de dados. Assim sendo, os dipositivos de redes permanecem simples e mais baratos, e o controlador pode implementar funções de controlo simplificadas (e possivelmente mais eficazes) graças à visão global da rede. No entanto um modelo de programação logicamente centralizado não implica um sistema centralizado. De facto, a necessidade de garantir níveis adequados de performance, escalabilidade e resiliência, proíbem que o plano de controlo seja centralizado. Em vez disso, as redes de SDN que operam a nível de produção utilizam planos de controlo distribuídos e os arquitectos destes sistemas têm que enfrentar os trade-offs fundamentais associados a sistemas distribuídos. Nomeadamente o equilíbrio adequado entre coerência e disponibilidade do sistema. Neste trabalho nós propomos uma arquitectura de um controlador distribuído, tolerante a faltas e coerente. O elemento central desta arquitectura é uma base de dados replicada e tolerante a faltas que mantém o estado da rede coerente, de forma a garantir que as aplicações de controlo da rede, que residem no controlador, possam operar com base numa visão coerente da rede que garanta coordenação, e consequentemente simplifique o desenvolvimento das aplicações. A desvantagem desta abordagem reflecte-se no decréscimo de performance, que limita a capacidade de resposta do controlador, e também a escalabilidade do mesmo. Mesmo assumindo estas consequências, uma conclusão importante do nosso estudo é que é possível atingir os objectivos propostos (i.e., coerência forte e tolerância a faltas) e manter a performance a um nível aceitável para determinados tipo de redes. Relativamente à tolerância a faltas, numa arquitectura SDN estas podem ocorrer em três domínios diferentes: o plano de dados (falhas do equipamento de rede), o plano de controlo (falhas da ligação entre o controlador e o equipamento de rede) e, finalmente, o próprio controlador. Este último é de uma importância particular, sendo que a falha do mesmo pode perturbar a rede por inteiro (i.e., deixando de existir conectividade entre os hosts). É portanto essencial que as redes de SDN que operam a nível de produção possuam mecanismos que possam lidar com os vários tipos de faltas e garantir disponibilidade perto de 100%. O trabalho recente em SDN têm explorado a questão da coerência a níveis diferentes. Linguagens de programação como a Frenetic [2] oferecem coerência na composição de políticas de rede, conseguindo resolver incoerências nas regras de encaminhamento automaticamente. Outra linha de trabalho relacionado propõe abstracções que garantem a coerência da rede durante a alteração das tabelas de encaminhamento do equipamento. O objectivo destes dois trabalhos é garantir a coerência depois de decidida a política de encaminhamento. O Onix (um controlador de SDN muitas vezes referenciado [3]) garante um tipo de coerência diferente: uma que é importante antes da política de encaminhamento ser tomada. Este controlador oferece dois tipos de coerência na salvaguarda do estado da rede: coerência eventual, e coerência forte. O nosso trabalho utiliza apenas coerência forte, e consegue demonstrar que esta pode ser garantida com uma performance superior à garantida pelo Onix. Actualmente, os controladores de SDN distribuídos (Onix e HyperFlow [4]) utilizam modelos de distribuição não transparentes, com propriedades fracas como coerência eventual que exigem maior cuidado no desenvolvimento de aplicações de controlo de rede no controlador. Isto deve-se à ideia (do nosso ponto de vista infundada) de que propriedades como coerência forte limitam significativamente a escalabilidade do controlador. No entanto um controlador com coerência forte traduz-se num modelo de programação mais simples e transparente à distribuição do controlador. Neste trabalho nós argumentámos que é possível utilizar técnicas bem conhecidas de replicação baseadas na máquina de estados distribuída [5], para construir um controlador SDN, que não só garante tolerância a faltas e coerência forte, mas também o faz com uma performance aceitável. Neste sentido a principal contribuição desta dissertação é mostrar que uma base de dados construída com as técnicas mencionadas anteriormente (como as providenciadas pelo BFT-SMaRt [6]), e integrada com um controlador open-source existente (como o Floodlight1), consegue lidar com vários tipos de carga, provenientes de aplicações de controlo de rede, eficientemente. As contribuições principais do nosso trabalho, podem ser resumidas em: 1. A proposta de uma arquitectura de um controlador distribuído baseado nas propriedades de coerência forte e tolerância a faltas; 2. Como a arquitectura proposta é baseada numa base de dados replicada, nós realizamos um estudo da carga produzida por três aplicações na base dados. 3. Para avaliar a viabilidade da nossa arquitectura nós analisamos a capacidade do middleware de replicação para processar a carga mencionada no ponto anterior. Este estudo descobre as seguintes variáveis: (a) Quantos eventos por segundo consegue o middleware processar por segundo; (b) Qual o impacto de tempo (i.e., latência) necessário para processar tais eventos; para cada uma das aplicações mencionadas, e para cada um dos possíveis eventos de rede processados por essas aplicações. Estas duas variáveis são importantes para entender a escalabilidade e performance da arquitectura proposta. Do nosso trabalho, nomeadamente do nosso estudo da carga das aplicações (numa primeira versão da nossa integração com a base de dados) e da capacidade do middleware resultou uma publicação: Fábio Botelho, Fernando Ramos, Diego Kreutz and Alysson Bessani; On the feasibility of a consistent and fault-tolerant data store for SDNs, in Second European Workshop on Software Defined Networks, Berlin, October 2013. Entretanto, nós submetemos esta dissertação cerca de cinco meses depois desse artigo, e portanto, contém um estudo muito mais apurado e melhorado.Even if traditional data networks are very successful, they exhibit considerable complexity manifested in the configuration of network devices, and development of network protocols. Researchers argue that this complexity derives from the fact that network devices are responsible for both processing control functions such as distributed routing protocols and forwarding packets. This work is motivated by the emergent network architecture of Software Defined Networks where the control functionality is removed from the network devices and delegated to a server (usually called controller) that is responsible for dynamically configuring the network devices present in the infrastructure. The controller has the advantage of logically centralizing the network state in contrast to the previous model where state was distributed across the network devices. Despite of this logical centralization, the control plane (where the controller operates) must be distributed in order to avoid being a single point of failure. However, this distribution introduces several challenges due to the heterogeneous, asynchronous, and faulty environment where the controller operates. Current distributed controllers lack transparency due to the eventual consistency properties employed in the distribution of the controller. This results in a complex programming model for the development of network control applications. This work proposes a fault-tolerant distributed controller with strong consistency properties that allows a transparent distribution of the control plane. The drawback of this approach is the increase in overhead and delay, which limits responsiveness and scalability. However, despite being fault-tolerant and strongly consistent, we show that this controller is able to provide performance results (in some cases) superior to those available in the literature

Edge/Fog Computing Technologies for IoT Infrastructure

The prevalence of smart devices and cloud computing has led to an explosion in the amount of data generated by IoT devices. Moreover, emerging IoT applications, such as augmented and virtual reality (AR/VR), intelligent transportation systems, and smart factories require ultra-low latency for data communication and processing. Fog/edge computing is a new computing paradigm where fully distributed fog/edge nodes located nearby end devices provide computing resources. By analyzing, filtering, and processing at local fog/edge resources instead of transferring tremendous data to the centralized cloud servers, fog/edge computing can reduce the processing delay and network traffic significantly. With these advantages, fog/edge computing is expected to be one of the key enabling technologies for building the IoT infrastructure. Aiming to explore the recent research and development on fog/edge computing technologies for building an IoT infrastructure, this book collected 10 articles. The selected articles cover diverse topics such as resource management, service provisioning, task offloading and scheduling, container orchestration, and security on edge/fog computing infrastructure, which can help to grasp recent trends, as well as state-of-the-art algorithms of fog/edge computing technologies

Algorithmic governance : A modes of governance approach

This article examines how modes of governance are reconfigured as a result of using algorithms in the governance process. We argue that deploying algorithmic systems creates a shift toward a special form of design-based governance, with power exercised ex ante via choice architectures defined through protocols, requiring lower levels of commitment from governing actors. We use governance of three policy problems - speeding, disinformation, and social sharing - to illustrate what happens when algorithms are deployed to enable coordination in modes of hierarchical governance, self-governance, and co-governance. Our analysis shows that algorithms increase efficiency while decreasing the space for governing actors' discretion. Furthermore, we compare the effects of algorithms in each of these cases and explore sources of convergence and divergence between the governance modes. We suggest design-based governance modes that rely on algorithmic systems might be re-conceptualized as algorithmic governance to account for the prevalence of algorithms and the significance of their effects.Peer reviewe

High Availability and Scalability Schemes for Software- Defined Networks (SDN)

Title from PDF of title page, viewed on September 8, 2015Dissertation advisor: Baek-Young ChoiVitaIncludes bibliographic references (pages 127-136)Thesis (Ph.D.)--School of Computing and Engineering. University of Missouri--Kansas City, 2015A proliferation of network-enabled devices and network-intensive applications require the underlying networks not only to be agile despite of complex and heterogeneous environments, but also to be highly available and scalable in order to guarantee service integrity and continuity. The Software-Defined Network (SDN) has recently emerged to address the problem of the ossified Internet protocol architecture and to enable agile and flexible network evolvement. SDN, however, heavily relies on control messages between a controller and the forwarding devices for the network operation. Thus, it becomes even more critical to guarantee network high availability (HA) and scalability between a controller and its forwarding devices in the SDN architecture. In this dissertation, we address HA and scalability issues that are inherent in the current OpenFlow specification and SDN architecture; and solve the problems using practical techniques. With extensive experiments using real systems, we have identified that iii the significant issues of HA and scalability in operations of a SDN such as single point of failure of multiple logical connections, multiple redundant configuration, unrecoverable interconnection failure, interface flapping, new flow attack, and event storm. We have designed and implemented the management frameworks that deal with SDN HA and scalability issues that we have observed from a real system. The proposed frameworks include various SDN HA and scalability strategies. For SDN HA, we have developed several SDN control path HA algorithms such as ensuring logical control path redundancy, transparency of a controller cluster, and fast and accurate failure detection. We validate the functionalities of the proposed SDN HA schemes with real network experiments. The proposed SDN control path HA algorithms overcome the limitations of the current Open- Flow specification and enhance performance as well as simplify management of SDN control path HA. For SDN scalability, we have proposed and developed our management framework in two different platforms; an embedded approach in the OpenFlow switch and an agent-based approach with the SUMA platform that is located near the Open- Flow switch. These platforms include various algorithms that enhance scalability of SDN such as Detect and Mitigate Abnormality (DMA), Modify and Annotate Control (MAC), and Message Prioritization and Classification (MPC). We have shown that the proposed framework effectively detects and filters malicious and abnormal network behaviors such as new flow attack, interface flapping, and event storm.Introduction -- Related work -- Measurement and Analysis of an Access Network’s Availability -- SDN Control Path High Availability -- SDN Scalable Network Management -- Summary and Future Wor

The Architecture of an Autonomic, Resource-Aware, Workstation-Based Distributed Database System

Distributed software systems that are designed to run over workstation machines within organisations are termed workstation-based. Workstation-based systems are characterised by dynamically changing sets of machines that are used primarily for other, user-centric tasks. They must be able to adapt to and utilize spare capacity when and where it is available, and ensure that the non-availability of an individual machine does not affect the availability of the system. This thesis focuses on the requirements and design of a workstation-based database system, which is motivated by an analysis of existing database architectures that are typically run over static, specially provisioned sets of machines. A typical clustered database system -- one that is run over a number of specially provisioned machines -- executes queries interactively, returning a synchronous response to applications, with its data made durable and resilient to the failure of machines. There are no existing workstation-based databases. Furthermore, other workstation-based systems do not attempt to achieve the requirements of interactivity and durability, because they are typically used to execute asynchronous batch processing jobs that tolerate data loss -- results can be re-computed. These systems use external servers to store the final results of computations rather than workstation machines. This thesis describes the design and implementation of a workstation-based database system and investigates its viability by evaluating its performance against existing clustered database systems and testing its availability during machine failures.Comment: Ph.D. Thesi

A decentralized framework for cross administrative domain data sharing

Federation of messaging and storage platforms located in remote datacenters is an essential functionality to share data among geographically distributed platforms. When systems are administered by the same owner data replication reduces data access latency bringing data closer to applications and enables fault tolerance to face disaster recovery of an entire location. When storage platforms are administered by different owners data replication across different administrative domains is essential for enterprise application data integration. Contents and services managed by different software platforms need to be integrated to provide richer contents and services. Clients may need to share subsets of data in order to enable collaborative analysis and service integration. Platforms usually include proprietary federation functionalities and specific APIs to let external software and platforms access their internal data. These different techniques may not be applicable to all environments and networks due to security and technological restrictions. Moreover the federation of dispersed nodes under a decentralized administration scheme is still a research issue. This thesis is a contribution along this research direction as it introduces and describes a framework, called \u201cWideGroups\u201d, directed towards the creation and the management of an automatic federation and integration of widely dispersed platform nodes. It is based on groups to exchange messages among distributed applications located in different remote datacenters. Groups are created and managed using client side programmatic configuration without touching servers. WideGroups enables the extension of the software platform services to nodes belonging to different administrative domains in a wide area network environment. It lets different nodes form ad-hoc overlay networks on-the-fly depending on message destinations located in distinct administrative domains. It supports multiple dynamic overlay networks based on message groups, dynamic discovery of nodes and automatic setup of overlay networks among nodes with no server-side configuration. I designed and implemented platform connectors to integrate the framework as the federation module of Message Oriented Middleware and Key Value Store platforms, which are among the most widespread paradigms supporting data sharing in distributed systems