35 research outputs found
Erlang Code Evolution Control
During the software lifecycle, a program can evolve several times for
different reasons such as the optimisation of a bottle-neck, the refactoring of
an obscure function, etc. These code changes often involve several functions or
modules, so it can be difficult to know whether the correct behaviour of the
previous releases has been preserved in the new release. Most developers rely
on a previously defined test suite to check this behaviour preservation. We
propose here an alternative approach to automatically obtain a test suite that
specifically focusses on comparing the old and new versions of the code. Our
test case generation is directed by a sophisticated combination of several
already existing tools such as TypEr, CutEr, and PropEr; and other ideas such
as allowing the programmer to chose an expression of interest that must
preserve the behaviour, or the recording of the sequences of values to which
this expression is evaluated. All the presented work has been implemented in an
open-source tool that is publicly available on GitHub.Comment: Pre-proceedings paper presented at the 27th International Symposium
on Logic-Based Program Synthesis and Transformation (LOPSTR 2017), Namur,
Belgium, 10-12 October 2017 (arXiv:1708.07854
How Time-Fault Ratio helps in Test Case Prioritization for Regression Testing
Regression testing analyzes whether the maintenance of the software has adversely affected its normal functioning. Regression testing is generally performed under the strict time constraints. Due to limited time budget, it is not possible to test the software with all available test cases. Thus, the reordering of the test cases, on the basis of their effectiveness, is always needed. A test prioritization technique, which prioritizes the test cases on the basis of their Time -Fault Ratio (TFR), has been proposed in this paper. The technique tends to maximize the fault detection as the faults are exposed in the ascending order of their detection times. The proposed technique may be used at any stage of software development
Pair-Wise Time-Aware Test Case Prioritization for Regression Testing
After maintenance, software requires regression testing for its validation. Prioritization of test cases for regression testing is required as software is tested under strict time and other constraints. A Pair-wise time-aware Test Case Prioritization (PTCP) technique has been proposed in this paper that determines the effectiveness of a test case on the basis of total number of faults present in software, number of faults detected till time, and the time of execution of different test cases. It selects that test case which determines maximum new faults, not yet detected, within minimum time. Thus prioritized test suite contains those test cases which are effective and tend to minimize repetitive faults detection. Through two comparative studies, it has been observed that with least wastage of time, the proposed technique performed equally well as other two parallel prioritizing techniques, Average Percentage of Fault Detection (APFD) based prioritization, and Optimal Test Case Prioritization (OTCP)
A new concept of effective regression test generation in a C++ specific environment
During regression testing test cases from an existing test suite are run against a modified version of a program in order to assure that the underlying modifications do not cause any side effects that would demolish the integrity and consistency of the system. Since the ultimate goal of a regression test set is to effectively test all modifications and reveal errors in the earliest possible stage, the maintenance of a relevant test set containing effective test cases is of utmost importance. In this paper we present an efficient, C++ specific framework to automatically manage the regression test suite. Our two main contributions are a new interpretation of reliable test cases and a dynamic forward impact analyzer method that eases the transformation of existing tests to meet the definition of reliability. Using this approach we complement the test set with test cases that pass through a modification and have an impact on at least one output. Our approach is designed to be applicable to large-scale applications
A Simple and Practical Approach to Unit Testing: The JML and JUnit Way
Writing unit test code is labor-intensive, hence it is often not done as an integral part of programming. However, unit testing is a practical approach to increasing the correctness and quality of software; for example, the Extreme Programming approach relies on frequent unit testing. In this paper we present a new approach that makes writing unit tests easier. It uses a formal specification language\u27s runtime assertion checker to decide whether methods are working correctly, thus automating the writing of unit test oracles. These oracles can be easily combined with hand-written test data. Instead of writing testing code, the programmer writes formal specifications (e.g., pre- and postconditions). This makes the programmer\u27s task easier, because specifications are more concise and abstract than the equivalent test code, and hence more readable and maintainable. Furthermore, by using specifications in testing, specification errors are quickly discovered, so the specifications are more likely to provide useful documentation and inputs to other tools. We have implemented this idea using the Java Modeling Language (JML) and the JUnit testing framework, but the approach could be easily implemented with other combinations of formal specification languages and unit test tools
A Simple and Practical Approach to Unit Testing: The JML and JUnit Way
Writing unit test code is labor-intensive, hence it is often not done as an integral part of programming. However, unit testing is a practical approach to increasing the correctness and quality of software; for example, the Extreme Programming approach relies on frequent unit testing. In this paper we present a new approach that makes writing unit tests easier. It uses a formal specification language\u27s runtime assertion checker to decide whether methods are working correctly, thus automating the writing of unit test oracles. These oracles can be easily combined with hand-written test data. Instead of writing testing code, the programmer writes formal specifications (e.g., pre- and postconditions). This makes the programmer\u27s task easier, because specifications are more concise and abstract than the equivalent test code, and hence more readable and maintainable. Furthermore, by using specifications in testing, specification errors are quickly discovered, so the specifications are more likely to provide useful documentation and inputs to other tools. We have implemented this idea using the Java Modeling Language (JML) and the JUnit testing framework, but the approach could be easily implemented with other combinations of formal specification languages and unit test tools
The JML and JUnit Way of Unit Testing and its Implementation
Writing unit test code is labor-intensive, hence it is often not done as an integral part of programming. However, unit testing is a practical approach to increasing the correctness and quality of software; for example, Extreme Programming relies on frequent unit testing. In this paper we present a new approach that makes writing unit tests easier. It uses a formal specification language\u27s runtime assertion checker to decide whether methods are working correctly; thus code to decide whether tests pass or fail is automatically produced from specifications. Our tool combines this testing code with hand-written test data to execute tests. Therefore, instead of writing testing code, the programmer writes formal specifications (e.g., pre- and postconditions). This makes the programmer\u27s task easier, because specifications are more concise and abstract than the equivalent test code, and hence more readable and maintainable. Furthermore, by using specifications in testing, specification errors are quickly discovered, so the specifications are more likely to provide useful documentation and inputs to other tools. In this paper we describe an implementation using the Java Modeling Language (JML) and the JUnit testing framework, but the approach could be easily implemented with other combinations of formal specification languages and unit testing tools
Mapping the Structure and Evolution of Software Testing Research Over the Past Three Decades
Background: The field of software testing is growing and rapidly-evolving.
Aims: Based on keywords assigned to publications, we seek to identify
predominant research topics and understand how they are connected and have
evolved.
Method: We apply co-word analysis to map the topology of testing research as
a network where author-assigned keywords are connected by edges indicating
co-occurrence in publications. Keywords are clustered based on edge density and
frequency of connection. We examine the most popular keywords, summarize
clusters into high-level research topics, examine how topics connect, and
examine how the field is changing.
Results: Testing research can be divided into 16 high-level topics and 18
subtopics. Creation guidance, automated test generation, evolution and
maintenance, and test oracles have particularly strong connections to other
topics, highlighting their multidisciplinary nature. Emerging keywords relate
to web and mobile apps, machine learning, energy consumption, automated program
repair and test generation, while emerging connections have formed between web
apps, test oracles, and machine learning with many topics. Random and
requirements-based testing show potential decline.
Conclusions: Our observations, advice, and map data offer a deeper
understanding of the field and inspiration regarding challenges and connections
to explore.Comment: To appear, Journal of Systems and Softwar
Automated, Cost-effective, and Update-driven App Testing
Apps' pervasive role in our society led to the definition of test automation
approaches to ensure their dependability. However, state-of-the-art approaches
tend to generate large numbers of test inputs and are unlikely to achieve more
than 50% method coverage. In this paper, we propose a strategy to achieve
significantly higher coverage of the code affected by updates with a much
smaller number of test inputs, thus alleviating the test oracle problem. More
specifically, we present ATUA, a model-based approach that synthesizes App
models with static analysis, integrates a dynamically-refined state abstraction
function and combines complementary testing strategies, including (1) coverage
of the model structure, (2) coverage of the App code, (3) random exploration,
and (4) coverage of dependencies identified through information retrieval. Its
model-based strategy enables ATUA to generate a small set of inputs that
exercise only the code affected by the updates. In turn, this makes common test
oracle solutions more cost-effective as they tend to involve human effort. A
large empirical evaluation, conducted with 72 App versions belonging to nine
popular Android Apps, has shown that ATUA is more effective and less effort
intensive than state-of-the-art approaches when testing App updates