35,335 research outputs found
Automatically Generating Test Cases for Safety-Critical Software via Symbolic Execution
Automated test generation based on symbolic execution can be beneficial for
systematically testing safety-critical software, to facilitate test engineers
to pursue the strict testing requirements mandated by the certification
standards, while controlling at the same time the costs of the testing process.
At the same time, the development of safety-critical software is often
constrained with programming languages or coding conventions that ban
linguistic features which are believed to downgrade the safety of the programs,
e.g., they do not allow dynamic memory allocation and variable-length arrays,
limit the way in which loops are used, forbid recursion, and bound the
complexity of control conditions. As a matter of facts, these linguistic
features are also the main efficiency-blockers for the test generation
approaches based on symbolic execution at the state of the art. This paper
contributes new evidence of the effectiveness of generating test cases with
symbolic execution for a significant class of industrial safety
critical-systems. We specifically focus on Scade, a largely adopted model-based
development language for safety-critical embedded software, and we report on a
case study in which we exploited symbolic execution to automatically generate
test cases for a set of safety-critical programs developed in Scade. To this
end, we introduce a novel test generator that we developed in a recent
industrial project on testing safety-critical railway software written in
Scade, and we report on our experience of using this test generator for testing
a set of Scade programs that belong to the development of an on-board signaling
unit for high-speed rail. The results provide empirically evidence that
symbolic execution is indeed a viable approach for generating high-quality test
suites for the safety-critical programs considered in our case study
SmartUnit: Empirical Evaluations for Automated Unit Testing of Embedded Software in Industry
In this paper, we aim at the automated unit coverage-based testing for
embedded software. To achieve the goal, by analyzing the industrial
requirements and our previous work on automated unit testing tool CAUT, we
rebuild a new tool, SmartUnit, to solve the engineering requirements that take
place in our partner companies. SmartUnit is a dynamic symbolic execution
implementation, which supports statement, branch, boundary value and MC/DC
coverage. SmartUnit has been used to test more than one million lines of code
in real projects. For confidentiality motives, we select three in-house real
projects for the empirical evaluations. We also carry out our evaluations on
two open source database projects, SQLite and PostgreSQL, to test the
scalability of our tool since the scale of the embedded software project is
mostly not large, 5K-50K lines of code on average. From our experimental
results, in general, more than 90% of functions in commercial embedded software
achieve 100% statement, branch, MC/DC coverage, more than 80% of functions in
SQLite achieve 100% MC/DC coverage, and more than 60% of functions in
PostgreSQL achieve 100% MC/DC coverage. Moreover, SmartUnit is able to find the
runtime exceptions at the unit testing level. We also have reported exceptions
like array index out of bounds and divided-by-zero in SQLite. Furthermore, we
analyze the reasons of low coverage in automated unit testing in our setting
and give a survey on the situation of manual unit testing with respect to
automated unit testing in industry.Comment: In Proceedings of 40th International Conference on Software
Engineering: Software Engineering in Practice Track, Gothenburg, Sweden, May
27-June 3, 2018 (ICSE-SEIP '18), 10 page
A Symbolic Java Virtual Machine for Test Case Generation
Quality management is becoming a more and more important part of the software development process. As software testing is currently understood as the core function of the quality managment, developers start using software testing tools to facilitate their work. However most existing tools just manage given sets of test cases and check them against pre-defined testing criteria. The necessary test case discovery is usually up to user, who has to generate the test cases in an ad-hoc approach with only minimal support from the software. GlassTT, the tool we present in this paper, faces this problem by creating the needed test cases for a given criterion for a Java class file. It uses a symbolic Java virtual machine to generate constraints representing the conditions for the control flow under consideration. The system can employ a parameterizeable test criterion such as data-flow coverage. The constraint solvers embedded in the tool are capable of solving linear and non-linear constraints, which is sufficient for almost all constraints encountered in the execution of Java programs. They are encapsulated in an incremental constraint solver manager, which dynamically chooses an appropriate constraint solver and enables the efficient communication between constraint solvers and symbolical virtual machine
CTGEN - a Unit Test Generator for C
We present a new unit test generator for C code, CTGEN. It generates test
data for C1 structural coverage and functional coverage based on
pre-/post-condition specifications or internal assertions. The generator
supports automated stub generation, and data to be returned by the stub to the
unit under test (UUT) may be specified by means of constraints. The typical
application field for CTGEN is embedded systems testing; therefore the tool can
cope with the typical aliasing problems present in low-level C, including
pointer arithmetics, structures and unions. CTGEN creates complete test
procedures which are ready to be compiled and run against the UUT. In this
paper we describe the main features of CTGEN, their technical realisation, and
we elaborate on its performance in comparison to a list of competing test
generation tools. Since 2011, CTGEN is used in industrial scale test campaigns
for embedded systems code in the automotive domain.Comment: In Proceedings SSV 2012, arXiv:1211.587
Incremental bounded model checking for embedded software
Program analysis is on the brink of mainstream usage in embedded systems development. Formal verification of behavioural requirements, finding runtime errors and test case generation are some of the most common applications of automated verification tools based on bounded model checking (BMC). Existing industrial tools for embedded software use an off-the-shelf bounded model checker and apply it iteratively to verify the program with an increasing number of unwindings. This approach unnecessarily wastes time repeating work that has already been done and fails to exploit the power of incremental SAT solving. This article reports on the extension of the software model checker CBMC to support incremental BMC and its successful integration with the industrial embedded software verification tool BTC EMBEDDED TESTER. We present an extensive evaluation over large industrial embedded programs, mainly from the automotive industry. We show that incremental BMC cuts runtimes by one order of magnitude in comparison to the standard non-incremental approach, enabling the application of formal verification to large and complex embedded software. We furthermore report promising results on analysing programs with arbitrary loop structure using incremental BMC, demonstrating its applicability and potential to verify general software beyond the embedded domain
Debugging Memory Issues In Embedded Linux: A Case Study
Debugging denotes the process of detecting root causes of unexpected
observable behaviors in programs, such as a program crash, an unexpected output
value being produced or an assertion violation. Debugging of program errors is
a difficult task and often takes a significant amount of time in the software
development life cycle. In the context of embedded software, the probability of
bugs is quite high. Due to requirements of low code size and less resource
consumption, embedded softwares typically do away with a lot of sanity checks
during development time. This leads to high chance of errors being uncovered in
the production code at run time. In this paper we propose a methodology for
debugging errors in BusyBox, a de-facto standard for Linux in embedded systems.
Our methodology works on top of Valgrind, a popular memory error detector and
Daikon, an invariant analyzer. We have experimented with two published errors
in BusyBox and report our findings in this paper.Comment: In proceedings of IEEE TechSym 2011, 14-16 January, 2011, IIT
kharagpur, Indi
- …