17 research outputs found
Muteria: An Extensible and Flexible Multi-Criteria Software Testing Framework
Program based test adequacy criteria (TAC), such as statement, branch coverage and mutation give objectives for software testing. Many techniques and tools have been developed to improve each phase of the TAC-based software testing process. Nonetheless, The engineering effort required to integrate these tools and techniques into the software testing process limits their use and creates an overhead to the users. Especially for system testing with languages like C, where test cases are not always well structured in a framework.
In response to these challenges, this paper presents Muteria, a TAC-based software testing framework. Muteria enables the integration of multiple software testing tools.
Muteria abstracts each phase of the TAC-based software testing process to provide tool drivers interfaces for the implementation of tool drivers. Tool drivers enable Muteria to call the corresponding tools during the testing process. An initial set of drivers for KLEE, Shadow and SEMu test-generation tools, Gcov, and coverage.py code coverage tools, and Mart mutant generation tool for C and Python programming language were implemented with an average of 345 lines of Python code. Moreover, the user configuration file required to measure code coverage and mutation score on a sample C programs, using the Muteria framework, consists of less than 15 configuration variables.
Users of the Muteria framework select, in a configuration file, the tools and TACs to measure. The Muteria framework uses the user configuration to run the testing process and report the outcome.
Users interact with Muteria through its Application Programming Interface and Command Line Interface. Muteria can benefit to researchers as a laboratory to execute experiments, and to software practitioners
LIPIcs
Fault-tolerant distributed algorithms play an important role in many critical/high-availability applications. These algorithms are notoriously difficult to implement correctly, due to asynchronous communication and the occurrence of faults, such as the network dropping messages or computers crashing. Nonetheless there is surprisingly little language and verification support to build distributed systems based on fault-tolerant algorithms. In this paper, we present some of the challenges that a designer has to overcome to implement a fault-tolerant distributed system. Then we review different models that have been proposed to reason about distributed algorithms and sketch how such a model can form the basis for a domain-specific programming language. Adopting a high-level programming model can simplify the programmer's life and make the code amenable to automated verification, while still compiling to efficiently executable code. We conclude by summarizing the current status of an ongoing language design and implementation project that is based on this idea
Model Checking Boot Code from AWS Data Centers
This paper describes our experience with symbolic model checking in an industrial setting. We have proved that the initial boot code running in data centers at Amazon Web Services is memory safe, an essential step in establishing the security of any data center. Standard static analysis tools cannot be easily used on boot code without modification owing to issues not commonly found in higher-level code, including memory-mapped device interfaces, byte-level memory access, and linker scripts. This paper describes automated solutions to these issues and their implementation in the C Bounded Model Checker (CBMC). CBMC is now the first source-level static analysis tool to extract the memory layout described in a linker script for use in its analysis
On Solving Word Equations Using SAT
We present Woorpje, a string solver for bounded word equations (i.e.,
equations where the length of each variable is upper bounded by a given
integer). Our algorithm works by reformulating the satisfiability of bounded
word equations as a reachability problem for nondeterministic finite automata,
and then carefully encoding this as a propositional satisfiability problem,
which we then solve using the well-known Glucose SAT-solver. This approach has
the advantage of allowing for the natural inclusion of additional linear length
constraints. Our solver obtains reliable and competitive results and,
remarkably, discovered several cases where state-of-the-art solvers exhibit a
faulty behaviour
An Empirical Study on Mutation, Statement and Branch Coverage Fault Revelation that Avoids the Unreliable Clean Program Assumption
Many studies suggest using coverage concepts, such as branch coverage, as the starting point of testing, while others as the most prominent test quality indicator. Yet the relationship between coverage and fault-revelation remains unknown, yielding uncertainty and controversy. Most previous studies rely on the Clean Program Assumption, that a test suite will obtain similar
coverage for both faulty and fixed (‘clean’) program versions. This assumption may appear intuitive, especially for bugs that denote small semantic deviations. However, we present evidence
that the Clean Program Assumption does not always hold, thereby raising a critical threat to the validity of previous results. We then conducted a study using a robust experimental methodology that avoids this threat to validity, from which our primary finding is that strong mutation testing has the highest fault revelation of four widely-used criteria. Our findings also revealed that fault revelation starts to increase significantly only once relatively high levels of coverage are attained
Capacity: Cryptographically-Enforced In-Process Capabilities for Modern ARM Architectures (Extended Version)
In-process compartmentalization and access control have been actively
explored to provide in-place and efficient isolation of in-process security
domains. Many works have proposed compartmentalization schemes that leverage
hardware features, most notably using the new page-based memory isolation
feature called Protection Keys for Userspace (PKU) on x86. Unfortunately, the
modern ARM architecture does not have an equivalent feature. Instead, newer ARM
architectures introduced Pointer Authentication (PA) and Memory Tagging
Extension (MTE), adapting the reference validation model for memory safety and
runtime exploit mitigation. We argue that those features have been
underexplored in the context of compartmentalization and that they can be
retrofitted to implement a capability-based in-process access control scheme.
This paper presents Capacity, a novel hardware-assisted intra-process access
control design that embraces capability-based security principles. Capacity
coherently incorporates the new hardware security features on ARM that already
exhibit inherent characteristics of capability. It supports the life-cycle
protection of the domain's sensitive objects -- starting from their import from
the file system to their place in memory. With intra-process domains
authenticated with unique PA keys, Capacity transforms file descriptors and
memory pointers into cryptographically-authenticated references and completely
mediates reference usage with its program instrumentation framework and an
efficient system call monitor. We evaluate our Capacity-enabled NGINX web
server prototype and other common applications in which sensitive resources are
isolated into different domains. Our evaluation shows that Capacity incurs a
low-performance overhead of approximately 17% for the single-threaded and
13.54% for the multi-threaded webserver.Comment: Accepted at ACM CCS 202