108,082 research outputs found
Automatically Generating Test Cases for Safety-Critical Software via Symbolic Execution
Automated test generation based on symbolic execution can be beneficial for
systematically testing safety-critical software, to facilitate test engineers
to pursue the strict testing requirements mandated by the certification
standards, while controlling at the same time the costs of the testing process.
At the same time, the development of safety-critical software is often
constrained with programming languages or coding conventions that ban
linguistic features which are believed to downgrade the safety of the programs,
e.g., they do not allow dynamic memory allocation and variable-length arrays,
limit the way in which loops are used, forbid recursion, and bound the
complexity of control conditions. As a matter of facts, these linguistic
features are also the main efficiency-blockers for the test generation
approaches based on symbolic execution at the state of the art. This paper
contributes new evidence of the effectiveness of generating test cases with
symbolic execution for a significant class of industrial safety
critical-systems. We specifically focus on Scade, a largely adopted model-based
development language for safety-critical embedded software, and we report on a
case study in which we exploited symbolic execution to automatically generate
test cases for a set of safety-critical programs developed in Scade. To this
end, we introduce a novel test generator that we developed in a recent
industrial project on testing safety-critical railway software written in
Scade, and we report on our experience of using this test generator for testing
a set of Scade programs that belong to the development of an on-board signaling
unit for high-speed rail. The results provide empirically evidence that
symbolic execution is indeed a viable approach for generating high-quality test
suites for the safety-critical programs considered in our case study
Unit Testing of Flash Memory Device Driver through a SAT-Based Model Checker
Flash memory has become virtually indispensable in most mobile devices. In order for mobile devices to success-fully provide services to users, it is essential that flash mem-ory be controlled correctly through the device driver soft-ware. However, as is typical for embedded software, con-ventional testing methods often fail to detect hidden flaws in the complex device driver software. This deficiency in-curs significant development and operation overhead to the manufacturers. Model checking techniques have been proposed to compensate weaknesses of conventional testing methods through exhaustive analyses. These techniques, however, require significant manual efforts to create an abstract tar-get model and, thus, are not widely applied in industry. In this project, we applied model checking technique based on Boolean satisfiability (SAT) solver. One advantage of SAT-based model checking is that a target C code can be analyzed directly without an abstract model, thereby en-abling fully-automated and bit-level accurate verification. In this project, we have applied CBMC, a SAT-based soft-ware model checker, for unit testing of the Samsung One-NAND device driver. Through this project, we detected sev-eral bugs that had not been discovered previously.
SmartUnit: Empirical Evaluations for Automated Unit Testing of Embedded Software in Industry
In this paper, we aim at the automated unit coverage-based testing for
embedded software. To achieve the goal, by analyzing the industrial
requirements and our previous work on automated unit testing tool CAUT, we
rebuild a new tool, SmartUnit, to solve the engineering requirements that take
place in our partner companies. SmartUnit is a dynamic symbolic execution
implementation, which supports statement, branch, boundary value and MC/DC
coverage. SmartUnit has been used to test more than one million lines of code
in real projects. For confidentiality motives, we select three in-house real
projects for the empirical evaluations. We also carry out our evaluations on
two open source database projects, SQLite and PostgreSQL, to test the
scalability of our tool since the scale of the embedded software project is
mostly not large, 5K-50K lines of code on average. From our experimental
results, in general, more than 90% of functions in commercial embedded software
achieve 100% statement, branch, MC/DC coverage, more than 80% of functions in
SQLite achieve 100% MC/DC coverage, and more than 60% of functions in
PostgreSQL achieve 100% MC/DC coverage. Moreover, SmartUnit is able to find the
runtime exceptions at the unit testing level. We also have reported exceptions
like array index out of bounds and divided-by-zero in SQLite. Furthermore, we
analyze the reasons of low coverage in automated unit testing in our setting
and give a survey on the situation of manual unit testing with respect to
automated unit testing in industry.Comment: In Proceedings of 40th International Conference on Software
Engineering: Software Engineering in Practice Track, Gothenburg, Sweden, May
27-June 3, 2018 (ICSE-SEIP '18), 10 page
Semantics-based Automated Web Testing
We present TAO, a software testing tool performing automated test and oracle
generation based on a semantic approach. TAO entangles grammar-based test
generation with automated semantics evaluation using a denotational semantics
framework. We show how TAO can be incorporated with the Selenium automation
tool for automated web testing, and how TAO can be further extended to support
automated delta debugging, where a failing web test script can be
systematically reduced based on grammar-directed strategies. A real-life
parking website is adopted throughout the paper to demonstrate the effectivity
of our semantics-based web testing approach.Comment: In Proceedings WWV 2015, arXiv:1508.0338
CTGEN - a Unit Test Generator for C
We present a new unit test generator for C code, CTGEN. It generates test
data for C1 structural coverage and functional coverage based on
pre-/post-condition specifications or internal assertions. The generator
supports automated stub generation, and data to be returned by the stub to the
unit under test (UUT) may be specified by means of constraints. The typical
application field for CTGEN is embedded systems testing; therefore the tool can
cope with the typical aliasing problems present in low-level C, including
pointer arithmetics, structures and unions. CTGEN creates complete test
procedures which are ready to be compiled and run against the UUT. In this
paper we describe the main features of CTGEN, their technical realisation, and
we elaborate on its performance in comparison to a list of competing test
generation tools. Since 2011, CTGEN is used in industrial scale test campaigns
for embedded systems code in the automotive domain.Comment: In Proceedings SSV 2012, arXiv:1211.587
Functional Requirements-Based Automated Testing for Avionics
We propose and demonstrate a method for the reduction of testing effort in
safety-critical software development using DO-178 guidance. We achieve this
through the application of Bounded Model Checking (BMC) to formal low-level
requirements, in order to generate tests automatically that are good enough to
replace existing labor-intensive test writing procedures while maintaining
independence from implementation artefacts. Given that existing manual
processes are often empirical and subjective, we begin by formally defining a
metric, which extends recognized best practice from code coverage analysis
strategies to generate tests that adequately cover the requirements. We then
formulate the automated test generation procedure and apply its prototype in
case studies with industrial partners. In review, the method developed here is
demonstrated to significantly reduce the human effort for the qualification of
software products under DO-178 guidance
Ada(R) Test and Verification System (ATVS)
The Ada Test and Verification System (ATVS) functional description and high level design are completed and summarized. The ATVS will provide a comprehensive set of test and verification capabilities specifically addressing the features of the Ada language, support for embedded system development, distributed environments, and advanced user interface capabilities. Its design emphasis was on effective software development environment integration and flexibility to ensure its long-term use in the Ada software development community
- …