High-Level Abstractions for Programming Network Policies

The emergence of network programmability enabled by innovations such as active network- ing, SDN and NFV offers tremendous flexibility to program network policies. However, it also poses a new demand to network operators on programming network policies. The motivation of this dissertation is to study the feasibility of using high-level abstractions to simplify the programming of network policies. First, we propose scenario-based programming, a framework that allows network operators to program stateful network policies by describing example behaviors in representative scenarios. Given these scenarios, our scenario-based programming tool NetEgg automatically infers the controller state that needs to be maintained along with the rules to process network events and update state. The NetEgg interpreter can execute the generated policy implementation on top of a centralized controller, but also automatically infers flow-table rules that can be pushed to switches to improve throughput. We study a range of policies considered in the literature and report our experience regarding specifying these policies using scenarios. We evaluate NetEgg based on the computational requirements of our synthesis algorithm as well as the overhead introduced by the generated policy implementation. Our results show that our synthesis algorithm can generate policy implementations in seconds, and the automatically generated policy implementations have performance comparable to their hand-crafted implementations. Our preliminary user study results show that NetEgg was able to reduce the programming time of the policies we studied. Second, we propose NetQRE, a high-level declarative language for programming quantitative network policies that require monitoring a stream of network packets. Based on a novel theoretical foundation of parameterized quantitative regular expressions, NetQRE integrates regular-expression-like pattern matching at flow-level as well as application-level payloads with aggregation operations such as sum and average counts. We describe a compiler for NetQRE that automatically generates an efficient implementation from the specification in NetQRE. Our evaluation results demonstrate that NetQRE is expressive to specify a wide range of quantitative network policies that cannot be naturally specified in other systems. The performance of the generated implementations is comparable with that of the manually-optimized low-level code. NetQRE can be deployed in different settings. Our proof-of-concept deployment shows that NetQRE can provide timely enforcement of quantitative network policies

Practical Network Programming Automation

Network configurations are notoriously hard to write and maintain correctly. It requiresexpertise about the domain to write, frequent and laborious updates, and sometimes formal proof to ensure the absence of certain mistakes. The problem becomes more challenging with the popularity of software-defined network(SDN) in recent years, which aims to give users more flexible control over the network’s dynamic behaviors. There has been research on automating the process of configuring the network. However, much of it requires users to learn a specific programming abstraction or interface. Since network operators are a group generally unfamiliar with programming, using these systems may go beyond their abilities. It is also hard to ensure these systems are scalable and accurate enough for real-world usecases. They mostly lack both design considerations to address scalability and accuracy, and also a systematic evaluation of the two metrics in practical scenarios. In this work, we propose a series of approaches to automate network programming. They are based on specifications that are easy and natural to obtain by network operators. We also apply novel program analysis techniques to speed up the process of finding a program that can accurately capture the intention of the specification. We have evaluated our systems on a broad range of benchmarks obtained from real-world data. They have shown ability to finish complex programming tasks within minutes and achieved very high accuracy

Tools and Algorithms for the Construction and Analysis of Systems

This open access two-volume set constitutes the proceedings of the 27th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2021, which was held during March 27 – April 1, 2021, as part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2021. The conference was planned to take place in Luxembourg and changed to an online format due to the COVID-19 pandemic. The total of 41 full papers presented in the proceedings was carefully reviewed and selected from 141 submissions. The volume also contains 7 tool papers; 6 Tool Demo papers, 9 SV-Comp Competition Papers. The papers are organized in topical sections as follows: Part I: Game Theory; SMT Verification; Probabilities; Timed Systems; Neural Networks; Analysis of Network Communication. Part II: Verification Techniques (not SMT); Case Studies; Proof Generation/Validation; Tool Papers; Tool Demo Papers; SV-Comp Tool Competition Papers

Real-time Decision Policies with Predictable Performance

As methods and tools for Cyber-Physical Systems grow in capabilities and use, one-size-fits-all solutions start to show their limitations. In particular, tools and languages for programming an algorithm or modeling a CPS that are specific to the application domain are typically more usable, and yield better performance, than general-purpose languages and tools. In the domain of cardiac arrhythmia monitoring, a small, implantable medical device continuously monitors the patient\u27s cardiac rhythm and delivers electrical therapy when needed. The algorithms executed by these devices are streaming algorithms, so they are best programmed in a streaming language that allows the programmer to reason about the incoming data stream as the basic object, rather than force her to think about lower-level details like state maintenance and minimization. Because these devices are resource-constrained, it is useful if the programming language allowed predictable performance in terms of processing runtime and energy consumption, or more general costs. StreamQRE is a declarative streaming programming language, with an efficient and portable implementation and strong theoretical guarantees. In particular, its evaluation algorithm guarantees constant cost (runtime, memory, energy) per data item, and also calculates upper bounds on the per-item cost. Such an estimate of the cost allows early exploration of the algorithmic possibilities, while maintaining a handle on worst-case performance, on the basis of which hardware can be designed and algorithms can be tuned

Accurate and Resource-Efficient Monitoring for Future Networks

Monitoring functionality is a key component of any network management system. It is essential for profiling network resource usage, detecting attacks, and capturing the performance of a multitude of services using the network. Traditional monitoring solutions operate on long timescales producing periodic reports, which are mostly used for manual and infrequent network management tasks. However, these practices have been recently questioned by the advent of Software Defined Networking (SDN). By empowering management applications with the right tools to perform automatic, frequent, and fine-grained network reconfigurations, SDN has made these applications more dependent than before on the accuracy and timeliness of monitoring reports. As a result, monitoring systems are required to collect considerable amounts of heterogeneous measurement data, process them in real-time, and expose the resulting knowledge in short timescales to network decision-making processes. Satisfying these requirements is extremely challenging given today’s larger network scales, massive and dynamic traffic volumes, and the stringent constraints on time availability and hardware resources. This PhD thesis tackles this important challenge by investigating how an accurate and resource-efficient monitoring function can be realised in the context of future, software-defined networks. Novel monitoring methodologies, designs, and frameworks are provided in this thesis, which scale with increasing network sizes and automatically adjust to changes in the operating conditions. These achieve the goal of efficient measurement collection and reporting, lightweight measurement- data processing, and timely monitoring knowledge delivery

Safe Programming Over Distributed Streams

The sheer scale of today\u27s data processing needs has led to a new paradigm of software systems centered around requirements for high-throughput, distributed, low-latency computation.Despite their widespread adoption, existing solutions have yet to provide a programming model with safe semantics -- and they disagree on basic design choices, in particular with their approach to parallelism. As a result, naive programmers are easily led to introduce correctness and performance bugs. This work proposes a reliable programming model for modern distributed stream processing, founded in a type system for partially ordered data streams. On top of the core type system, we propose language abstractions for working with streams -- mechanisms to build stream operators with (1) type-safe compositionality, (2) deterministic distribution, (3) run-time testing, and (4) static performance bounds. Our thesis is that viewing streams as partially ordered conveniently exposes parallelism without compromising safety or determinism. The ideas contained in this work are implemented in a series of open source software projects, including the Flumina, DiffStream, and Data Transducers libraries

Flexible and intelligent network programming for cloud networks

As modern online services are evolving promptly and involving larger amount of data and computation than ever, the demand for cloud networks keeps growing rapidly, which also brings new challenges to network programming. Network programming is a complicated and crucial task for building high-performance cloud networks. Current network programming mainly presents two shortcomings: (1) it is inflexible as adding new data-plane features usually takes several years; (2) it is unintelligent as it heavily depends on human-designed heuristic algorithms to solve production-scale problems. To overcome these shortcomings, this dissertation realizes flexible and intelligent network programming by leveraging the recent development of new technologies both in hardware and software. Specifically, it presents four systems with new performance features that cannot be achieved by conventional network programming: (i) Harmonia: A new replicated storage architecture that provides near-linear scalability without sacrificing consistency. By exploiting the programming flexibility of new-generation programmable switches, Harmonia checks read-write conflicts in network for guaranteeing consistency, and enables any replica to serve reads for objects with no pending writes for near-linear scalability. (ii) RackSched: A microsecond-scale scheduler for rack-scale computers. It proposes a two-layer scheduling framework that integrates the inter-server scheduler in the top-of-rack (ToR) switch with intra-server schedulers on each server. The in-network inter-server scheduler is programmed to realize power-of-k-choices, ensure request affinity, and track server loads accurately and efficiently. (iii) NetVRM: A network management system that supports dynamic register memory sharing in the network. It orchestrates the register memory allocation between multiple concurrent network applications to optimize the multiplexing benefits. This goal is achieved with three major features: a virtual register memory abstraction, a dynamic memory allocation algorithm, and a domain-specific programming language extension. (iv) NeuroPlan: Automated and efficient network planning with deep reinforcement learning (RL). It leverages a two-stage hybrid approach that first uses deep RL to prune a large and complex search space and then uses an Integer Linear Programming (ILP) solver to find the final solution. Such an automated approach avoids human efforts to design heuristic algorithms manually and reduces network plan cost efficiently. We have done theoretical analysis, built testbeds, and evaluated these systems with prototype experiments and simulations under realistic setups from production networks