A Quantitative Theory of Bottleneck Structures for Data Networks

The conventional view of the congestion control problem in data networks is based on the principle that a flow's performance is uniquely determined by the state of its bottleneck link, regardless of the topological properties of the network. However, recent work has shown that the behavior of congestion-controlled networks is better explained by models that account for the interactions between bottleneck links. These interactions are captured by a latent \textit{bottleneck structure}, a model describing the complex ripple effects that changes in one part of the network exert on the other parts. In this paper, we present a \textit{quantitative} theory of bottleneck structures (QTBS), a mathematical and engineering framework comprising a family of polynomial-time algorithms that can be used to reason about a wide variety of network optimization problems, including routing, capacity planning and flow control. QTBS can contribute to traffic engineering by making clear predictions about the relative performance of alternative flow routes, and by providing numerical recommendations for the optimal rate settings of traffic shapers. A particularly novel result in the domain of capacity planning indicates that previously established rules for the design of folded-Clos networks are suboptimal when flows are congestion controlled. We show that QTBS can be used to derive the optimal rules for this important class of topologies, and empirically demonstrate the correctness and efficacy of these results using the BBR and Cubic congestion-control algorithms

Introducing v0.5 of the AI Safety Benchmark from MLCommons

This paper introduces v0.5 of the AI Safety Benchmark, which has been created by the MLCommons AI Safety Working Group. The AI Safety Benchmark has been designed to assess the safety risks of AI systems that use chat-tuned language models. We introduce a principled approach to specifying and constructing the benchmark, which for v0.5 covers only a single use case (an adult chatting to a general-purpose assistant in English), and a limited set of personas (i.e., typical users, malicious users, and vulnerable users). We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0.5 benchmark. We plan to release version 1.0 of the AI Safety Benchmark by the end of 2024. The v1.0 benchmark will provide meaningful insights into the safety of AI systems. However, the v0.5 benchmark should not be used to assess the safety of AI systems. We have sought to fully document the limitations, flaws, and challenges of v0.5. This release of v0.5 of the AI Safety Benchmark includes (1) a principled approach to specifying and constructing the benchmark, which comprises use cases, types of systems under test (SUTs), language and context, personas, tests, and test items; (2) a taxonomy of 13 hazard categories with definitions and subcategories; (3) tests for seven of the hazard categories, each comprising a unique set of test items, i.e., prompts. There are 43,090 test items in total, which we created with templates; (4) a grading system for AI systems against the benchmark; (5) an openly available platform, and downloadable tool, called ModelBench that can be used to evaluate the safety of AI systems on the benchmark; (6) an example evaluation report which benchmarks the performance of over a dozen openly available chat-tuned language models; (7) a test specification for the benchmark

Introducing v0.5 of the AI Safety Benchmark from MLCommons

This paper introduces v0.5 of the AI Safety Benchmark, which has been created by the MLCommons AI Safety Working Group. The AI Safety Benchmark has been designed to assess the safety risks of AI systems that use chat-tuned language models. We introduce a principled approach to specifying and constructing the benchmark, which for v0.5 covers only a single use case (an adult chatting to a general-purpose assistant in English), and a limited set of personas (i.e., typical users, malicious users, and vulnerable users). We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0.5 benchmark. We plan to release version 1.0 of the AI Safety Benchmark by the end of 2024. The v1.0 benchmark will provide meaningful insights into the safety of AI systems. However, the v0.5 benchmark should not be used to assess the safety of AI systems. We have sought to fully document the limitations, flaws, and challenges of v0.5. This release of v0.5 of the AI Safety Benchmark includes (1) a principled approach to specifying and constructing the benchmark, which comprises use cases, types of systems under test (SUTs), language and context, personas, tests, and test items; (2) a taxonomy of 13 hazard categories with definitions and subcategories; (3) tests for seven of the hazard categories, each comprising a unique set of test items, i.e., prompts. There are 43,090 test items in total, which we created with templates; (4) a grading system for AI systems against the benchmark; (5) an openly available platform, and downloadable tool, called ModelBench that can be used to evaluate the safety of AI systems on the benchmark; (6) an example evaluation report which benchmarks the performance of over a dozen openly available chat-tuned language models; (7) a test specification for the benchmark

Resolving Constrained Existential Queries over Context-Sensitive Analyses

A context-sensitive analysis is an analysis in which program elements are interpreted with respect to the context in which they occur. For analyses on imperative languages, this often refers to considering the behavior of called procedures with respect to the calling-stack contexts that precede them. Algorithms for performing or approximating these types of analyses make up the core of interprocedural program analysis and are pervasive; having applications in program optimization, checkpointing, and model checking. This paper presents an abstraction of a popular form of context-sensitive analysis based on iteratively encapsulating the cumulative effect of a recurring piece of code. Given an analysis fitting this abstraction, a technique is presented for resolving queries of the form: Is there an occurring context, subject to a given stack-context constraint, in which a particular set of facts holds at a particular location? This practical technique, based on manipulating regular languages, is capable not only of answering queries of this form, but also of generating a compact mechanism for dynamically applying the output of the analysis to contexts as they occur. A comprehensive example is presented along with performance data on a case study code. Finally, a selection of potential applications is discussed

An Optimizing Compiler for Batches of Temporal Logic Formulas

Model checking based on validating temporal logic formulas has proven practical and effective for numerous software engineering applications. As systems based on this approach have become more mainstream, a need has arisen to deal effectively with large batches of formulas over a common model. Presently, most systems validate formulas one at a time, with little or no interaction between validation of separate formulas. This is the case despite the fact that, for a wide range of applications, a certain level of redundancy between domain-related formulas can be anticipated. This paper presents an optimizing compiler for batches of temporal logic formulas. A component of the Carnauba model checking system, this compiler addresses the need to handle batches of temporal logic formulas by leveraging th

An End-to-End System for Model Checking over Context-Sensitive Analyses

interpretation frameworks. Journal of Logic and Computation, 2(4):511--547, August 1992

Man vs. Machine : Comparing Handwritten and Compiler-generated Application-Level Checkpointing

The contributions of this paper are the following. We describe the implementation of the $C^3$ system for semi-automatic application-level checkpointing of C programs. The system has (i) a pre-compiler that instruments C programs so that they can save their states at program execution points specified by the user, and (ii) a novel memory allocator that manages the heap as a collection of pools. We describe two static analyses for reducing the overhead of saving and restoring the application state. The first one optimizes stack variables, while the second one optimizes heap data structures. To benchmark our system, we compare the overheads introduced by our semi-automatic approach with the overhead of handwritten application-level checkpointing in an n-body code written by Joshua Barnes. Except for very small problem sizes, these overheads are comparable. We highlight various algorithmic challenges in the optimization of application-level checkpointing that should provide grist for the mills of the PLDI community

Efficient Computation of Interprocedural Control Dependence

Control dependence information is useful for a wide range of software maintenance and testing tasks. For example, program slicers use it to determine statements and predicates that might affect the value of a particular variable at a particular program location. In the intraprocedural context an optimal algorithm is known for computing control dependence which unfortunately relies critically on the underlying intraprocedural postdominance relation being tree-structured. Hence, this algorithm is not directly applicable to the interprocedural case where the transitive reduction of the postdominance relation can be a directed acyclic graph (DAG), with nodes having multiple immediate dominators. In this paper we present two efficient, conceptually simple algorithms for computing the interprocedural postdominance relation that can be used to compute interprocedural control dependence. For an interprocedural control flow graph $G=(V,E)$, our reachability based algorithm takes time and space $O(|V|^2 + |V||E|)$. Unlike other algorithms, it does not perform confluence operations on whole bit-vectors and can be tuned to concentrate on the interprocedural rather than intraprocedural relations in a program thus allowing it to scale better to larger programs