3 research outputs found
Supporting Regression Test Scoping with Visual Analytics
Test managers have to repeatedly select test cases for test activities during evolution of large software systems. Researchers have widely studied automated test scoping, but have not fully investigated decision support with human interaction. We previously proposed the introduction of visual analytics for this purpose. Aim: In this empirical study we investigate how to design such decision support. Method: We explored the use of visual analytics using heat maps of historical test data for test scoping support by letting test managers evaluate prototype visualizations in three focus groups with in total nine industrial test experts. Results: All test managers in the study found the visual analytics useful for supporting test planning. However, our results show that different tasks and contexts require different types of visualizations. Conclusion: Important properties for test planning support are: ability to overview testing from different perspectives, ability to filter and zoom to compare subsets of the testing with respect to various attributes and the ability to manipulate the subset under analysis by selecting and deselecting test cases. Our results may be used to support the introduction of visual test analytics in practice
Boundary Value Exploration for Software Analysis
For software to be reliable and resilient, it is widely accepted that tests
must be created and maintained alongside the software itself. One safeguard
from vulnerabilities and failures in code is to ensure correct behavior on the
boundaries between sub-domains of the input space. So-called boundary value
analysis (BVA) and boundary value testing (BVT) techniques aim to exercise
those boundaries and increase test effectiveness. However, the concepts of BVA
and BVT themselves are not clearly defined and it is not clear how to identify
relevant sub-domains, and thus the boundaries delineating them, given a
specification. This has limited adoption and hindered automation. We clarify
BVA and BVT and introduce Boundary Value Exploration (BVE) to describe
techniques that support them by helping to detect and identify boundary inputs.
Additionally, we propose two concrete BVE techniques based on
information-theoretic distance functions: (i) an algorithm for boundary
detection and (ii) the usage of software visualization to explore the behavior
of the software under test and identify its boundary behavior. As an initial
evaluation, we apply these techniques on a much used and well-tested date
handling library. Our results reveal questionable behavior at boundaries
highlighted by our techniques. In conclusion, we argue that the boundary value
exploration that our techniques enable is a step towards automated boundary
value analysis and testing which can foster their wider use and improve test
effectiveness and efficiency