26 research outputs found
Modeling and Testing Implementations of Protocols with Complex Messages
This paper presents a new language called APSL for formally describing
protocols to facilitate automated testing. Many real world communication
protocols exchange messages whose structures are not trivial, e.g. they may
consist of multiple and nested fields, some could be optional, and some may
have values that depend on other fields. To properly test implementations of
such a protocol, it is not sufficient to only explore different orders of
sending and receiving messages. We also need to investigate if the
implementation indeed produces correctly formatted messages, and if it responds
correctly when it receives different variations of every message type. APSL's
main contribution is its sublanguage that is expressive enough to describe
complex message formats, both text-based and binary. As an example, this paper
also presents a case study where APSL is used to model and test a subset of
Courier IMAP email server
Testing by Dualization
Software engineering requires rigorous testing to guarantee the product's
quality. Semantic testing of functional correctness is challenged by
nondeterminism in behavior, which makes testers difficult to write and reason
about.
This thesis presents a language-based technique for testing interactive
systems. I propose a theory for specifying and validating nondeterministic
behaviors, with guaranteed soundness and correctness. I then apply the theory
to testing practices, and show how to derive specifications into interactive
tester programs. I also introduce a language design for producing test inputs
that can effectively detect and reproduce invalid behaviors.
I evaluate the methodology by specifying and testing real-world systems such
as web servers and file synchronizers, demonstrating the derived testers'
ability to find disagreements between the specification and the implementation
Improving the Network Scalability of Erlang
As the number of cores grows in commodity architectures so does the likelihood of failures. A distributed actor model potentially facilitates the development of reliable and scalable software on these architectures. Key components include lightweight processes which âshare nothingâ and hence can fail independently. Erlang is not only increasingly widely used, but the underlying actor model has been a beacon for programming language design, influencing for example Scala, Clojure and Cloud Haskell.
While the Erlang distributed actor model is inherently scalable, we demonstrate that it is limited by some pragmatic factors. We address two network scalability issues here: globally registered process names must be updated on every node (virtual machine) in the system, and any Erlang nodes that communicate maintain an active connection. That is, there is a fully connected O(n2) network of n nodes.
We present the design, implementation, and initial evaluation of a conservative extension of Erlang â Scalable Distributed (SD) Erlang. SD Erlang partitions the global namespace and connection network using s_groups. An s_group is a set of nodes with its own process namespace and with a fully connected network within the s_group, but only individual connections outside it. As a node may belong to more than one s_group it is possible to construct arbitrary connection topologies like trees or rings.
We present an operational semantics for the s_group functions, and outline the validation of conformance between the implementation and the semantics using the QuickCheck automatic testing tool. Our preliminary evaluation in comparison with distributed Erlang shows that SD Erlang dramatically improves network scalability even if the number of global operations is tiny (0.01%). Moreover, even in the absence of global operations the reduced connection maintenance overheads mean that SD Erlang scales better beyond 80 nodes (1920 cores)
Testing By Dualization
Software engineering requires rigorous testing to guarantee the product\u27s quality. Semantic testing of functional correctness is challenged by nondeterminism in behavior, which makes testers difficult to write and reason about.
This thesis presents a language-based technique for testing interactive systems. I propose a theory for specifying and validating nondeterministic behaviors, with guaranteed soundness and correctness. I then apply the theory to testing practices, and show how to derive specifications into interactive tester programs. I also introduce a language design for producing test inputs that can effectively detect and reproduce invalid behaviors.
I evaluate the methodology by specifying and testing real-world systems such as web servers and file synchronizers, demonstrating the derived testers\u27 ability to find disagreements between the specification and the implementation
A Formal Semantics for the SmartFrog Configuration Language
System configuration languages are now widely used to drive the deployment and evolution of large computing infrastructures. Most such languages are highly informal, making it difficult to reason about configurations, and introducing an important source of failure. We claim that a more rigorous approach to the development and specification of these languages will help to avoid these difficulties and bring a number of additional benefits. In order to test this claim, we present a formal semantics for the core of the SmartFrog configuration language. We demonstrate how this can be used to prove important properties such as termination of the compilation process. To show that this also contributes to the practical development of clear and correct compilers, we present three independent implementations, and verify their equivalence with each other, and with the semantics. Supported by an extended example from a real configuration scenario, we demonstrate how the process of developing the semantics has improved understanding of the language, highlighted problem areas, and suggested alternative interpretations. This leads us to advocate this approach for the future development of practical configuration languages
Recommended from our members
Engineering with logic: Rigorous test-oracle specification and validation for TCP/IP and the Sockets API
Conventional computer engineering relies on test-and-debug development processes, with the behavior of common interfaces described (at best) with prose specification documents. But prose specifications cannot be used in test-and-debug development in any automated way, and prose is a poor medium for expressing complex (and loose) specifications.
The TCP/IP protocols and Sockets API are a good example of this: they play a vital role in modern communication and computation, and interoperability between implementations is essential. But what exactly they are is surprisingly obscure: their original development focused on ârough consensus and running code,â augmented by prose RFC specifications that do not precisely define what it means for an implementation to be correct. Ultimately, the actual standard is the de facto one of the common implementations, including, for example, the 15Â 000 to 20Â 000 lines of the BSD implementationâoptimized and multithreaded C code, time dependent, with asynchronous event handlers, intertwined with the operating system, and security critical.
This article reports on work done in the
Netsem
project to develop lightweight mathematically rigorous techniques that can be applied to such systems: to specify their behavior precisely (but loosely enough to permit the required implementation variation) and to test whether these specifications and the implementations correspond with specifications that are
executable as test oracles
. We developed post hoc specifications of TCP, UDP, and the Sockets API, both of the service that they provide to applications (in terms of TCP bidirectional stream connections) and of the internal operation of the protocol (in terms of TCP segments and UDP datagrams), together with a testable abstraction function relating the two. These specifications are rigorous, detailed, readable, with broad coverage, and rather accurate. Working within a general-purpose proof assistant (HOL4), we developed
language idioms
(within higher-order logic) in which to write the specifications: operational semantics with nondeterminism, time, system calls, monadic relational programming, and so forth. We followed an
experimental semantics
approach, validating the specifications against several thousand traces captured from three implementations (FreeBSD, Linux, and WinXP). Many differences between these were identified, as were a number of bugs. Validation was done using a special-purpose
symbolic model checker
programmed above HOL4.
Having demonstrated that our logic-based engineering techniques suffice for handling real-world protocols, we argue that similar techniques could be applied to future critical software infrastructure at design time, leading to cleaner designs and (via specification-based testing) more robust and predictable implementations. In cases where specification looseness can be controlled, this should be possible with lightweight techniques, without the need for a general-purpose proof assistant, at relatively little cost.EPSRC Programme Grant EP/K008528/1 REMS: Rigorous Engineering for Mainstream Systems
EPSRC Leadership Fellowship EP/H005633 (Sewell)
Royal Society University Research Fellowship (Sewell)
St Catharine's College Heller Research Fellowship (Wansbrough),
EPSRC grant GR/N24872 Wide-area programming: Language, Semantics and Infrastructure Design
EPSRC grant EP/C510712 NETSEM: Rigorous Semantics for Real
Systems
EC FET-GC project IST-2001-33234 PEPITO Peer-to-Peer Computing: Implementation and Theory
CMI UROP internship support (Smith)
EC Thematic Network IST-2001-38957 APPSEM 2
NICTA was funded by the Australian Government's Backing Australia's Ability initiative, in part through the Australian Research Council