Faithful reproduction of network experiments

The proliferation of cloud computing has compelled the research community to rethink fundamental aspects of network systems and architectures. However, the tools commonly used to evaluate new ideas have not kept abreast of the latest developments. Common simulation and emulation frameworks fail to provide scalability, fidelity, reproducibility and execute unmodified code, all at the same time. We present SELENA, a Xen-based network emulation framework that offers fully reproducible experiments via its automation interface and supports the use of unmodified guest operating systems. This allows out-of-the-box compatibility with common applications and OS components, such as network stacks and filesystems. In order to faithfully emulate faster and larger networks, SELENA adopts the technique of time-dilation and transparently slows down the passage of time for guest operating systems. This technique effectively virtualizes the availability of host’s hardware resources and allows the replication of scenarios with increased I/O and computational demands. Users can directly control the tradeoff between fidelity and running-times via intuitive tuning knobs. We evaluate the ability of SELENA to faithfully replicate the behaviour of real systems and compare it against existing popular experimentation platforms. Our results suggest that SELENA can accurately model networks with aggregate link speeds of 44 Gbps or more, while improving by four times the execution time in comparison to ns3 and exhibits near-linear scaling properties.This is the author accepted manuscript. The final version is available from ACM via http://dx.doi.org/10.1145/2658260.265827

Doctor of Philosophy

dissertationAs the base of the software stack, system-level software is expected to provide ecient and scalable storage, communication, security and resource management functionalities. However, there are many computationally expensive functionalities at the system level, such as encryption, packet inspection, and error correction. All of these require substantial computing power. What's more, today's application workloads have entered gigabyte and terabyte scales, which demand even more computing power. To solve the rapidly increased computing power demand at the system level, this dissertation proposes using parallel graphics pro- cessing units (GPUs) in system software. GPUs excel at parallel computing, and also have a much faster development trend in parallel performance than central processing units (CPUs). However, system-level software has been originally designed to be latency-oriented. GPUs are designed for long-running computation and large-scale data processing, which are throughput-oriented. Such mismatch makes it dicult to t the system-level software with the GPUs. This dissertation presents generic principles of system-level GPU computing developed during the process of creating our two general frameworks for integrating GPU computing in storage and network packet processing. The principles are generic design techniques and abstractions to deal with common system-level GPU computing challenges. Those principles have been evaluated in concrete cases including storage and network packet processing applications that have been augmented with GPU computing. The signicant performance improvement found in the evaluation shows the eectiveness and eciency of the proposed techniques and abstractions. This dissertation also presents a literature survey of the relatively young system-level GPU computing area, to introduce the state of the art in both applications and techniques, and also their future potentials

Real-Time Waveform Prototyping

Mobile Netzwerke der fünften Generation zeichen sich aus durch vielfältigen Anforderungen und Einsatzszenarien. Drei unterschiedliche Anwendungsfälle sind hierbei besonders relevant: 1) Industrie-Applikationen fordern Echtzeitfunkübertragungen mit besonders niedrigen Ausfallraten. 2) Internet-of-things-Anwendungen erfordern die Anbindung einer Vielzahl von verteilten Sensoren. 3) Die Datenraten für Anwendung wie z.B. der Übermittlung von Videoinhalten sind massiv gestiegen. Diese zum Teil gegensätzlichen Erwartungen veranlassen Forscher und Ingenieure dazu, neue Konzepte und Technologien für zukünftige drahtlose Kommunikationssysteme in Betracht zu ziehen. Ziel ist es, aus einer Vielzahl neuer Ideen vielversprechende Kandidatentechnologien zu identifizieren und zu entscheiden, welche für die Umsetzung in zukünftige Produkte geeignet sind. Die Herausforderungen, diese Anforderungen zu erreichen, liegen jedoch jenseits der Möglichkeiten, die eine einzelne Verarbeitungsschicht in einem drahtlosen Netzwerk bieten kann. Daher müssen mehrere Forschungsbereiche Forschungsideen gemeinsam nutzen. Diese Arbeit beschreibt daher eine Plattform als Basis für zukünftige experimentelle Erforschung von drahtlosen Netzwerken unter reellen Bedingungen. Es werden folgende drei Aspekte näher vorgestellt: Zunächst erfolgt ein Überblick über moderne Prototypen und Testbed-Lösungen, die auf großes Interesse, Nachfrage, aber auch Förderungsmöglichkeiten stoßen. Allerdings ist der Entwicklungsaufwand nicht unerheblich und richtet sich stark nach den gewählten Eigenschaften der Plattform. Der Auswahlprozess ist jedoch aufgrund der Menge der verfügbaren Optionen und ihrer jeweiligen (versteckten) Implikationen komplex. Daher wird ein Leitfaden anhand verschiedener Beispiele vorgestellt, mit dem Ziel Erwartungen im Vergleich zu den für den Prototyp erforderlichen Aufwänden zu bewerten. Zweitens wird ein flexibler, aber echtzeitfähiger Signalprozessor eingeführt, der auf einer software-programmierbaren Funkplattform läuft. Der Prozessor ermöglicht die Rekonfiguration wichtiger Parameter der physikalischen Schicht während der Laufzeit, um eine Vielzahl moderner Wellenformen zu erzeugen. Es werden vier Parametereinstellungen 'LLC', 'WiFi', 'eMBB' und 'IoT' vorgestellt, um die Anforderungen der verschiedenen drahtlosen Anwendungen widerzuspiegeln. Diese werden dann zur Evaluierung der die in dieser Arbeit vorgestellte Implementierung herangezogen. Drittens wird durch die Einführung einer generischen Testinfrastruktur die Einbeziehung externer Partner aus der Ferne ermöglicht. Das Testfeld kann hier für verschiedenste Experimente flexibel auf die Anforderungen drahtloser Technologien zugeschnitten werden. Mit Hilfe der Testinfrastruktur wird die Leistung des vorgestellten Transceivers hinsichtlich Latenz, erreichbarem Durchsatz und Paketfehlerraten bewertet. Die öffentliche Demonstration eines taktilen Internet-Prototypen, unter Verwendung von Roboterarmen in einer Mehrbenutzerumgebung, konnte erfolgreich durchgeführt und bei mehreren Gelegenheiten präsentiert werden.:List of figures List of tables Abbreviations Notations 1 Introduction 1.1 Wireless applications 1.2 Motivation 1.3 Software-Defined Radio 1.4 State of the art 1.5 Testbed 1.6 Summary 2 Background 2.1 System Model 2.2 PHY Layer Structure 2.3 Generalized Frequency Division Multiplexing 2.4 Wireless Standards 2.4.1 IEEE 802.15.4 2.4.2 802.11 WLAN 2.4.3 LTE 2.4.4 Low Latency Industrial Wireless Communications 2.4.5 Summary 3 Wireless Prototyping 3.1 Testbed Examples 3.1.1 PHY - focused Testbeds 3.1.2 MAC - focused Testbeds 3.1.3 Network - focused testbeds 3.1.4 Generic testbeds 3.2 Considerations 3.3 Use cases and Scenarios 3.4 Requirements 3.5 Methodology 3.6 Hardware Platform 3.6.1 Host 3.6.2 FPGA 3.6.3 Hybrid 3.6.4 ASIC 3.7 Software Platform 3.7.1 Testbed Management Frameworks 3.7.2 Development Frameworks 3.7.3 Software Implementations 3.8 Deployment 3.9 Discussion 3.10 Conclusion 4 Flexible Transceiver 4.1 Signal Processing Modules 4.1.1 MAC interface 4.1.2 Encoding and Mapping 4.1.3 Modem 4.1.4 Post modem processing 4.1.5 Synchronization 4.1.6 Channel Estimation and Equalization 4.1.7 Demapping 4.1.8 Flexible Configuration 4.2 Analysis 4.2.1 Numerical Precision 4.2.2 Spectral analysis 4.2.3 Latency 4.2.4 Resource Consumption 4.3 Discussion 4.3.1 Extension to MIMO 4.4 Summary 5 Testbed 5.1 Infrastructure 5.2 Automation 5.3 Software Defined Radio Platform 5.4 Radio Frequency Front-end 5.4.1 Sub 6 GHz front-end 5.4.2 26 GHz mmWave front-end 5.5 Performance evaluation 5.6 Summary 6 Experiments 6.1 Single Link 6.1.1 Infrastructure 6.1.2 Single Link Experiments 6.1.3 End-to-End 6.2 Multi-User 6.3 26 GHz mmWave experimentation 6.4 Summary 7 Key lessons 7.1 Limitations Experienced During Development 7.2 Prototyping Future 7.3 Open points 7.4 Workflow 7.5 Summary 8 Conclusions 8.1 Future Work 8.1.1 Prototyping Workflow 8.1.2 Flexible Transceiver Core 8.1.3 Experimental Data-sets 8.1.4 Evolved Access Point Prototype For Industrial Networks 8.1.5 Testbed Standardization A Additional Resources A.1 Fourier Transform Blocks A.2 Resource Consumption A.3 Channel Sounding using Chirp sequences A.3.1 SNR Estimation A.3.2 Channel Estimation A.4 Hardware part listThe demand to achieve higher data rates for the Enhanced Mobile Broadband scenario and novel fifth generation use cases like Ultra-Reliable Low-Latency and Massive Machine-type Communications drive researchers and engineers to consider new concepts and technologies for future wireless communication systems. The goal is to identify promising candidate technologies among a vast number of new ideas and to decide, which are suitable for implementation in future products. However, the challenges to achieve those demands are beyond the capabilities a single processing layer in a wireless network can offer. Therefore, several research domains have to collaboratively exploit research ideas. This thesis presents a platform to provide a base for future applied research on wireless networks. Firstly, by giving an overview of state-of-the-art prototypes and testbed solutions. Secondly by introducing a flexible, yet real-time physical layer signal processor running on a software defined radio platform. The processor enables reconfiguring important parameters of the physical layer during run-time in order to create a multitude of modern waveforms. Thirdly, by introducing a generic test infrastructure, which can be tailored to prototype diverse wireless technology and which is remotely accessible in order to invite new ideas by third parties. Using the test infrastructure, the performance of the flexible transceiver is evaluated regarding latency, achievable throughput and packet error rates.:List of figures List of tables Abbreviations Notations 1 Introduction 1.1 Wireless applications 1.2 Motivation 1.3 Software-Defined Radio 1.4 State of the art 1.5 Testbed 1.6 Summary 2 Background 2.1 System Model 2.2 PHY Layer Structure 2.3 Generalized Frequency Division Multiplexing 2.4 Wireless Standards 2.4.1 IEEE 802.15.4 2.4.2 802.11 WLAN 2.4.3 LTE 2.4.4 Low Latency Industrial Wireless Communications 2.4.5 Summary 3 Wireless Prototyping 3.1 Testbed Examples 3.1.1 PHY - focused Testbeds 3.1.2 MAC - focused Testbeds 3.1.3 Network - focused testbeds 3.1.4 Generic testbeds 3.2 Considerations 3.3 Use cases and Scenarios 3.4 Requirements 3.5 Methodology 3.6 Hardware Platform 3.6.1 Host 3.6.2 FPGA 3.6.3 Hybrid 3.6.4 ASIC 3.7 Software Platform 3.7.1 Testbed Management Frameworks 3.7.2 Development Frameworks 3.7.3 Software Implementations 3.8 Deployment 3.9 Discussion 3.10 Conclusion 4 Flexible Transceiver 4.1 Signal Processing Modules 4.1.1 MAC interface 4.1.2 Encoding and Mapping 4.1.3 Modem 4.1.4 Post modem processing 4.1.5 Synchronization 4.1.6 Channel Estimation and Equalization 4.1.7 Demapping 4.1.8 Flexible Configuration 4.2 Analysis 4.2.1 Numerical Precision 4.2.2 Spectral analysis 4.2.3 Latency 4.2.4 Resource Consumption 4.3 Discussion 4.3.1 Extension to MIMO 4.4 Summary 5 Testbed 5.1 Infrastructure 5.2 Automation 5.3 Software Defined Radio Platform 5.4 Radio Frequency Front-end 5.4.1 Sub 6 GHz front-end 5.4.2 26 GHz mmWave front-end 5.5 Performance evaluation 5.6 Summary 6 Experiments 6.1 Single Link 6.1.1 Infrastructure 6.1.2 Single Link Experiments 6.1.3 End-to-End 6.2 Multi-User 6.3 26 GHz mmWave experimentation 6.4 Summary 7 Key lessons 7.1 Limitations Experienced During Development 7.2 Prototyping Future 7.3 Open points 7.4 Workflow 7.5 Summary 8 Conclusions 8.1 Future Work 8.1.1 Prototyping Workflow 8.1.2 Flexible Transceiver Core 8.1.3 Experimental Data-sets 8.1.4 Evolved Access Point Prototype For Industrial Networks 8.1.5 Testbed Standardization A Additional Resources A.1 Fourier Transform Blocks A.2 Resource Consumption A.3 Channel Sounding using Chirp sequences A.3.1 SNR Estimation A.3.2 Channel Estimation A.4 Hardware part lis

Analyse und Optimierung von Hybriden Software-Defined Networks

Hybrid IP networks that use both control plane paradigms - distributed and centralized - promise the best of two worlds: programmability and flexible control of Software-Defined Networking (SDN), and at the same time the reliability and fault tolerance of distributed routing protocols like Open Shortest Path First (OSPF). Hybrid SDN/OSPF networks typically deploy OSPF to assure care-free operation of best effort traffic, while SDN can control prioritized traffic. This "ships-passing-in-the-night" approach, where both control planes are unaware of each other's configurations, only require hybrid SDN/OSPF routers that can participate in the domain-wide legacy routing protocol and additionally connect to a central SDN controller. This mode of operation is however known for a number of challenges in operational networks, including those related to network failures, size of forwarding tables, routing convergence time, and the increased complexity of network management. There are alternative modes of hybrid operation that provide a more holistic network control paradigm, either through an OSPF-enabled SDN controller, or a common network management system that allows the joint monitoring and configuration of both control planes, or via the partitioning of the legacy routing domain with SDN border nodes. The latter mode of operation offers to some extent to steer the working of the legacy routing protocol inside the sub-domains, which is new. The analysis, modeling, and evaluative comparison of this approach called SDN Partitioning with other modes of operation is the main contribution of this thesis. This thesis addresses important network planning tasks in hybrid SDN/OSPF networks and provides the according mathematical models to optimize network clustering, capacity planning, SDN node placement, and resource provisioning for a fault tolerant operation. It furthermore provides the mathematical models to optimize traffic engineering, failure recovery, reconfiguration scheduling, and traffic monitoring in hybrid SDN/OSPF networks, which are vital network operational tasks.Hybride IP-Netzwerke, die beide Control-Plane-Paradigmen einsetzen - verteilt und zentralisiert - versprechen das Beste aus beiden Welten: Programmierbarkeit und flexible Kontrolle des Software-Defined Networking (SDN) und gleichzeitig die Zuverlässigkeit und Fehlertoleranz von verteilten Routingprotokollen wie Open Shortest Path First (OSPF). Hybride SDN/OSPF-Netze nutzen typischerweise OSPF für die wartungsarme Bedienung des Best-Effort-Datenverkehrs, während SDN priorisierte Datenströme kontrolliert. Bei diesem Ansatz ist beiden Kontrollinstanzen die Konfiguration der jeweils anderen unbekannt, wodurch hierbei hybride SDN/OSPF Router benötigt werden, die am domänenweiten Routingprotokoll teilnehmen können und zusätzlich eine Verbindung zu einem SDN-Controller herstellen. Diese Arbeitsweise bereitet jedoch bekanntermaßen eine Reihe von Schwierigkeiten in operativen Netzen, wie zum Beispiel die Reaktion auf Störungen, die Größe der Forwarding-Tabellen, die benötigte Zeit zur Konvergenz des Routings, sowie die höhere Komplexität der Netzwerkadministration. Es existieren alternative Betriebsmodi für hybride Netze, die einen ganzheitlicheren Kontrollansatz bieten, entweder mittels OSPF-Erweiterungen im SDN-Controller, oder mittels eines übergreifenden Netzwerkmanagementsystems, dass das Monitoring und die Konfiguration aller Netzelemente erlaubt. Eine weitere Möglichkeit stellt das Clustering der ursprünglichen Routingdomäne in kleinere Subdomänen mittels SDN-Grenzknoten dar. Dieser neue Betriebsmodus erlaubt es zu einem gewissen Grad, die Operationen des Routingprotokolls in den Subdomänen zu steuern. Die Analyse, Modellierung und die vergleichende Evaluation dieses Ansatzes mit dem Namen SDN-Partitionierung und anderen hybriden Betriebsmodi ist der Hauptbeitrag dieser Dissertation. Diese Dissertation behandelt grundlegende Fragen der Netzplanung in hybriden SDN/OSPF-Netzen und beinhaltet entsprechende mathematische Modelle zur Optimierung des Clusterings, zur Kapazitätsplanung, zum Platzieren von SDN-Routern, sowie zur Bestimmung der notwendigen Ressourcen für einen fehlertoleranten Betrieb. Desweiteren enthält diese Dissertation Optimierungsmodelle für Traffic Engineering, zur Störungsbehebung, zur Ablaufplanung von Konfigurationsprozessen, sowie zum Monitoring des Datenverkehrs in hybriden SDN/OSPF-Netzen, was entscheidende Aufgaben der Netzadministration sind

Utilizing Advanced Network Context to Optimize Software-Defined Networks

Legacy network systems and protocols are mostly static and keep state information in silo-style storage, thus making state migration, transformation and re-use difficult. Software-Defined Network (SDN) approaches in unison with Network Function Virtualization (NFV) allow for more flexibility, yet they are currently restricted to a limited set of state migration options. Additionally, existing systems and protocols are mostly tailored to meet the requirements of specific application scenarios. As a result, the protocols cannot easily be adapted to novel application demands, organically growing networks, etc. Impeding the sharing of networking and system state, along with lacking support for dynamic transitions between systems and protocols, severely limits the ability to optimally manage resources and dynamically adapt to a desirable overall configuration. These limitations not only affect the network performance but also hinder the deployment of new and innovative protocols as a hard break is usually not feasible and thus full support for legacy systems is required. On the one hand, we propose a generalized way to collect, store, transform, and share context between systems and protocols in both the legacy Internet as well as NFV/SDN-driven networks. This allows us to share state information between multiple systems and protocols from NFs over BGP routers to protocols on all layers of the network stack. On the other hand, we introduce an architecture for designing modular protocols that are built with transition in mind. We argue that the modular design of systems and protocols can remove the key limitations of today’s monolithic protocols and allow for a more dynamic network management. First, we design and implement a Storage and Transformation Engine for Advanced Net- working context (STEAN) which constitutes a shared context storage, making network state information available to other systems and protocols. Its pivotal feature is the ability to allow for state transformation as well as for persisting state to enable future re-use. Second, we provide a Blueprint for Switching Between Mechanisms that serves as a framework and guideline for developers to standardize and ease the process of designing and implementing systems and protocols that support transitions as a first order principle. By means of experimentation, we show that our architecture covers a diverse set of challenging use cases in legacy systems—such as Wireless Multihop Networks (WMNs)—as well as in NFV/SDN-enabled systems. In particular, we demonstrate the feasibility of our approach by migrating state information between two instances of the PRADS NF in a virtualized Mininet environment, and show that our solution outperforms state of the art frameworks that are specifically built for NF migration. We further demonstrate that a dynamic switch between WMN routing protocols is possible at runtime and that the state information can be reutilized for bootstrapping novel protocol modules, thus minimizing the control overhead

Empowering Cloud Data Centers with Network Programmability

Cloud data centers are a critical infrastructure for modern Internet services such as web search, social networking and e-commerce. However, the gradual slow-down of Moore’s law has put a burden on the growth of data centers’ performance and energy efficiency. In addition, the increasing of millisecond-scale and microsecond-scale tasks also bring higher requirements to the throughput and latency for the cloud applications. Today’s server-based solutions are hard to meet the performance requirements in many scenarios like resource management, scheduling, high-speed traffic monitoring and testing. In this dissertation, we study these problems from a network perspective. We investigate a new architecture that leverages the programmability of new-generation network switches to improve the performance and reliability of clouds. As programmable switches only provide very limited memory and functionalities, we exploit compact data structures and deeply co-design software and hardware to best utilize the resource. More specifically, this dissertation presents four systems: (i) NetLock: A new centralized lock management architecture that co-designs programmable switches and servers to simultaneously achieve high performance and rich policy support. It provides orders-of-magnitude higher throughput than existing systems with microsecond-level latency, and supports many commonly-used policies such as performance isolation. (ii) HCSFQ: A scalable and practical solution to implement hierarchical fair queueing on commodity hardware at line rate. Instead of relying on a hierarchy of queues with complex queue management, HCSFQ does not keep per-flow states and uses only one queue to achieve hierarchical fair queueing. (iii) AIFO: A new approach for programmable packet scheduling that only uses a single FIFO queue. AIFO utilizes an admission control mechanism to approximate PIFO which is theoretically ideal but hard to implement with commodity devices. (iv) Lumina: A tool that enables fine-grained analysis of hardware network stack. By exploiting network programmability to emulate various network scenarios, Lumina is able to help users understand the micro-behaviors of hardware network stacks

High Performance Computing Facility Operational Assessment, FY 2011 Oak Ridge Leadership Computing Facility

Oak Ridge National Laboratory's Leadership Computing Facility (OLCF) continues to deliver the most powerful resources in the U.S. for open science. At 2.33 petaflops peak performance, the Cray XT Jaguar delivered more than 1.4 billion core hours in calendar year (CY) 2011 to researchers around the world for computational simulations relevant to national and energy security; advancing the frontiers of knowledge in physical sciences and areas of biological, medical, environmental, and computer sciences; and providing world-class research facilities for the nation's science enterprise. Users reported more than 670 publications this year arising from their use of OLCF resources. Of these we report the 300 in this review that are consistent with guidance provided. Scientific achievements by OLCF users cut across all range scales from atomic to molecular to large-scale structures. At the atomic scale, researchers discovered that the anomalously long half-life of Carbon-14 can be explained by calculating, for the first time, the very complex three-body interactions between all the neutrons and protons in the nucleus. At the molecular scale, researchers combined experimental results from LBL's light source and simulations on Jaguar to discover how DNA replication continues past a damaged site so a mutation can be repaired later. Other researchers combined experimental results from ORNL's Spallation Neutron Source and simulations on Jaguar to reveal the molecular structure of ligno-cellulosic material used in bioethanol production. This year, Jaguar has been used to do billion-cell CFD calculations to develop shock wave compression turbo machinery as a means to meet DOE goals for reducing carbon sequestration costs. General Electric used Jaguar to calculate the unsteady flow through turbo machinery to learn what efficiencies the traditional steady flow assumption is hiding from designers. Even a 1% improvement in turbine design can save the nation billions of gallons of fuel