데이터센터 네트워크에서의 소프트웨어 정의 네트워킹에 관한 연구

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2015. 2. 권태경.Software-Defined Data Center (SDDC) is a new paradigm of managing and operating IT infrastructure, where the resources in data center such as compute, storage, and networking are softwarized and delivered as a service to users on demand via application programming interface. Moreover, these resources are managed and controlled by software automatically--this is unprecedented in traditional IT infrastructure in which the infrastructure is typically defined by and tightly coupled with hardware and software. To realize this ideal environment, SDDC encompasses several virtualization technologies in compute, storage, and networking. In this thesis, we are more focusing on networking because the first two technologies are technologically advanced last few years, but networking evolution is still slow due to the vendor lock-in in which hardware and software of network element, which are typically proprietary, are tightly coupled. Moreover, in the network operators' perspective, configuring box-by-box manner with low-level commands results in increasing management complexity and being error-prone. To meet the requirements of today's users, enterprises, and carriers, Software-Defined Networking (SDN) is emerged. The main idea of SDN is to decouple control planes from data planes of network elements such as switches or routers and to replace the distributed, per-switch control plane with a (logically) centralized one on which SDN applications can control an operational network with a global network-wide view by enforcing packet forward- ing rules to the distributed data planes. This paradigm shift benefits network operators by (i) reducing the complexity of operations through automation while keeping more responsive, and (ii) optimizing the resources of operational networks with the global network-wide view to meet the dynamic nature of on-demand services in a cloud era. While SDN promises the enormous benefits as mentioned just before, it introduces new challenges: (i) increased control loop--gathers traffic and other measurements from the network and uses the gathered information to compute and install forwarding behaviours in the switches--due to the decoupling, and (ii) limitation on a distributed architecture--a (logically) centralized control plane is horizontally distributed to multiple physical servers--for large-scale production networks due to consistency overhead. To address the first challenge, we propose, implement and evaluate OpenSample: a low-latency, sampling-based network measurement platform targeted at building faster control loops for software-defined networks. OpenSample leverages sFlow packet sampling to provide near--real-time measurements of both network load and individual flows. While OpenSample is useful in any context, it is particularly useful in an SDN environment where a network controller can quickly take action based on the data it provides. Using sampling for network monitoring allows OpenSample to have a 100 millisecond control loop rather than the 1--5 second control loop of prior polling-based approaches. We implement OpenSample in the Floodlight OpenFlow controller and evaluate it both in simulation and on a testbed comprised of commodity switches. When used to inform traffic engineering, OpenSample provides up to a 150\% throughput improvement over both static equal-cost multi-path routing and a polling-based solution with a one second control loop. To address the second challenge, we propose FRACTAL, a framework for recursive abstraction of SDN control-plane, to address this problem. In FRACTAL, a large network is divided into multiple small networks, each of which is abstracted as a single virtual switch. This ``divide-and-abstract'' process is recursively iterated until a divided network can be handled by a single controller. A virtual switch is controlled by the higher level controller over OpenFlow, so that FRACTAL can coexist with other SDN mechanisms. We first carry out simulation experiments to demonstrate the issues of naive net- work partitioning. We then implement and evaluate FRACTAL with microbenchmark. Testbed-based experiments reveal that FRACTAL (i) adds small delays for non-local messages that cross divided networks, but (ii) achieves superlinearly increasing (control plane) throughput as the number of abstraction levels in the controller hierarchy grows.Abstract..................................... i I. Introduction ............................... 1 1.1 DataCenterNetworks......................... 2 1.2 Software-DefinedNetworking .................... 3 1.3 Challenges in Managing Data Center Networks through SDN . . . . 7 1.4 ThesisStructure............................ 8 II. OpenSample: A Low-latency, Sampling-based Measurement Plat- formforCommoditySDN........................ 10 2.1 Introduction.............................. 10 2.2 Background.............................. 14 2.2.1 Data Center Network Workloads and Topologies . . . . . . 15 2.2.2 Non-OpenFlowNetworkMonitoring . . . . . . . . . . . . 16 2.3 OpenSampleDesign ......................... 20 2.3.1 Protocol-Aware Flow Statistics Detection . . . . . . . . . . 22 2.3.2 Probability of Flow Statistics Detection . . . . . . . . . . . 23 2.3.3 FlowDetectionDelay .................... 26 2.3.4 EstimatingSwitchPortUtilization . . . . . . . . . . . . . . 27 2.3.5 NetworkStateSnapshotDatabase .............. 28 2.3.6 TrafficEngineering...................... 28 2.4 Evaluation............................... 30 2.4.1 Methodology ......................... 32 2.4.2 Results ............................ 36 2.4.3 Scalability .......................... 37 2.5 RelatedWork ............................. 38 FRACTAL: A Framework for Recursive Abstraction of SDN Control- PlaneforLarge-ScaleProductionNetworks . . . . . . . . . . . . . . 41 3.1 Introduction.............................. 41 3.2 Background.............................. 44 3.2.1 ADistributedControlPlaneinSDN . . . . . . . . . . . . . 44 3.2.2 ImpactofADistributedControlPlane . . . . . . . . . . . . 46 3.3 FRACTALDesign .......................... 50 3.3.1 DomainManager....................... 53 3.3.2 Many-to-OneMapping................... 54 3.3.3 RuleConflictDetectionandResolution . . . . . . . . . . . 56 3.4 Evaluation............................... 58 3.4.1 Microbenchmark ....................... 58 3.4.2 Experimentsoncampusnetwork............... 60 3.5 RelatedWork ............................. 65 IV. Conclusion&FutureWork....................... 67 Bibliography.................................. 69Docto

Routing choices in intelligent transport systems

Road congestion is a phenomenon that can often be avoided; roads become popular, travel times increase, which could be mitigated with better coordination mechanisms. The choice of route, mode of transport, and departure time all play a crucial part in controlling congestion levels. Technology, such as navigation applications, have the ability to influence these decisions and play an essential role in congestion reduction. To predict vehicles' routing behaviours, we model the system as a game with rational players. Players choose a path between origin and destination nodes in a network. Each player seeks to minimise their own journey time, often leading to inefficient equilibria with poor social welfare. Traffic congestion motivates the results in this thesis. However, the results also hold true for many other applications where congestion occurs, e.g. power grid demand. Coordinating route selection to reduce congestion constitutes a social dilemma for vehicles. In sequential social dilemmas, players' strategies need to balance their vulnerability to exploitation from their opponents and to learn to cooperate to achieve maximal payouts. We address this trade-off between mathematical safety and cooperation of strategies in social dilemmas to motivate our proposed algorithm, a safe method of achieving cooperation in social dilemmas, including route choice games. Many vehicles use navigation applications to help plan their journeys, but these provide only partial information about the routes available to them. We find a class of networks for which route information distribution cannot harm the receiver's expected travel times. Additionally, we consider a game where players always follow the route chosen by an application or where vehicle route selection is controlled by a route planner, such as autonomous vehicles. We show that having multiple route planners controlling vehicle routing leads to inefficient equilibria. We calculate the Price of Anarchy (PoA) for polynomial function travel times and show that multiagent reinforcement learning algorithms suffer from the predicted Price of Anarchy when controlling vehicle routing. Finally, we equip congestion games with waiting times at junctions to model the properties of traffic lights at intersections. Here, we show that Braess' paradox can be avoided by implementing traffic light cycles and establish the PoA for realistic waiting times. By employing intelligent traffic lights that use myopic learning, such as multi-agent reinforcement learning, we prove a natural reward function guarantees convergence to equilibrium. Moreover, we highlight the impact of multi-agent reinforcement learning traffic lights on the fairness of journey times to vehicles

Aprendizagem de coordenação em sistemas multi-agente

The ability for an agent to coordinate with others within a system is a valuable property in multi-agent systems. Agents either cooperate as a team to accomplish a common goal, or adapt to opponents to complete different goals without being exploited. Research has shown that learning multi-agent coordination is significantly more complex than learning policies in singleagent environments, and requires a variety of techniques to deal with the properties of a system where agents learn concurrently. This thesis aims to determine how can machine learning be used to achieve coordination within a multi-agent system. It asks what techniques can be used to tackle the increased complexity of such systems and their credit assignment challenges, how to achieve coordination, and how to use communication to improve the behavior of a team. Many algorithms for competitive environments are tabular-based, preventing their use with high-dimension or continuous state-spaces, and may be biased against specific equilibrium strategies. This thesis proposes multiple deep learning extensions for competitive environments, allowing algorithms to reach equilibrium strategies in complex and partially-observable environments, relying only on local information. A tabular algorithm is also extended with a new update rule that eliminates its bias against deterministic strategies. Current state-of-the-art approaches for cooperative environments rely on deep learning to handle the environment’s complexity and benefit from a centralized learning phase. Solutions that incorporate communication between agents often prevent agents from being executed in a distributed manner. This thesis proposes a multi-agent algorithm where agents learn communication protocols to compensate for local partial-observability, and remain independently executed. A centralized learning phase can incorporate additional environment information to increase the robustness and speed with which a team converges to successful policies. The algorithm outperforms current state-of-the-art approaches in a wide variety of multi-agent environments. A permutation invariant network architecture is also proposed to increase the scalability of the algorithm to large team sizes. Further research is needed to identify how can the techniques proposed in this thesis, for cooperative and competitive environments, be used in unison for mixed environments, and whether they are adequate for general artificial intelligence.A capacidade de um agente se coordenar com outros num sistema é uma propriedade valiosa em sistemas multi-agente. Agentes cooperam como uma equipa para cumprir um objetivo comum, ou adaptam-se aos oponentes de forma a completar objetivos egoístas sem serem explorados. Investigação demonstra que aprender coordenação multi-agente é significativamente mais complexo que aprender estratégias em ambientes com um único agente, e requer uma variedade de técnicas para lidar com um ambiente onde agentes aprendem simultaneamente. Esta tese procura determinar como aprendizagem automática pode ser usada para encontrar coordenação em sistemas multi-agente. O documento questiona que técnicas podem ser usadas para enfrentar a superior complexidade destes sistemas e o seu desafio de atribuição de crédito, como aprender coordenação, e como usar comunicação para melhorar o comportamento duma equipa. Múltiplos algoritmos para ambientes competitivos são tabulares, o que impede o seu uso com espaços de estado de alta-dimensão ou contínuos, e podem ter tendências contra estratégias de equilíbrio específicas. Esta tese propõe múltiplas extensões de aprendizagem profunda para ambientes competitivos, permitindo a algoritmos atingir estratégias de equilíbrio em ambientes complexos e parcialmente-observáveis, com base em apenas informação local. Um algoritmo tabular é também extendido com um novo critério de atualização que elimina a sua tendência contra estratégias determinísticas. Atuais soluções de estado-da-arte para ambientes cooperativos têm base em aprendizagem profunda para lidar com a complexidade do ambiente, e beneficiam duma fase de aprendizagem centralizada. Soluções que incorporam comunicação entre agentes frequentemente impedem os próprios de ser executados de forma distribuída. Esta tese propõe um algoritmo multi-agente onde os agentes aprendem protocolos de comunicação para compensarem por observabilidade parcial local, e continuam a ser executados de forma distribuída. Uma fase de aprendizagem centralizada pode incorporar informação adicional sobre ambiente para aumentar a robustez e velocidade com que uma equipa converge para estratégias bem-sucedidas. O algoritmo ultrapassa abordagens estado-da-arte atuais numa grande variedade de ambientes multi-agente. Uma arquitetura de rede invariante a permutações é também proposta para aumentar a escalabilidade do algoritmo para grandes equipas. Mais pesquisa é necessária para identificar como as técnicas propostas nesta tese, para ambientes cooperativos e competitivos, podem ser usadas em conjunto para ambientes mistos, e averiguar se são adequadas a inteligência artificial geral.Apoio financeiro da FCT e do FSE no âmbito do III Quadro Comunitário de ApoioPrograma Doutoral em Informátic

A Self-Organizing Wireless Sensor Network and Distributed Computing Engine for Commodity and Future Palmtop Computers

The embedded class processors found in commodity palmtop computers continue to become increasingly capable while retaining an energy-efficient footprint. Palmtop computers themselves, including smartphones and tablets, provide a small form factor system integrating wireless communication and non-volatile storage with these energy-efficient processors. Also, various wireless connectivity functions on mobile devices provide new opportunities in designing more flexible, smarter wireless sensor networks (WSNs), and utilizing the computation power in a way we could never imagine before. In this dissertation, I present a WSN concept for current and future generation tablet devices. My contributions include developments at the system level, architecture level, and collaborative design between different layers of the system. At the system level, I developed Ocelot, a grid-like computing environment for palmtop computers in place of traditional workstation or server class machines to compute highly parallel light to medium-weight tasks in an energy efficient manner. Additionally, I developed Lynx, a self-organizing wireless sensor network, which is a further step taken in exploiting the potential of palmtop computers. At the architecture level, to increase the storage capacity of future palmtop computers, I explore the use of a new storage class magnetic memory, Racetrack Memory (RM), throughout the memory hierarchy. Thus, I developed FusedCache, a naturally inclusive, dual-level private cache design for RM that provides fast uniform access at one level, and non-uniform access at the next, which allows RM to be effective as close to the processor as an L1 cache. For higher levels of the memory hierarchy such as the last level cache, I propose a Multilane Racetrack Cache (MRC), an RM last level cache design utilizing lightweight compression combined with independent shifting. MRCs allow cache lines mapped to the same Racetrack structure to be accessed in parallel when compressed, mitigating potential shifting stalls in an RM cache. Finally, leveraging the lightweight compression from MRC and the need for efficient communication in Lynx, I present a cross-level design combining memory-level lightweight compression with network-level packet transfer, together with a technique called Source-Aware Layout Reorganization (SALR) to increase the compressibility of sensor data