A partial oracle for uniformity statistics

This paper investigates the problem of testing implementations of uniformity statistics. In this paper we used Metamorphic Testing to address the oracle problem, of checking the output of one or more test executions, for uniformity statistics. We defined a partial oracle that uses regression analysis (a Regression Model based Metamorphic Relation). We investigated the effectiveness of our partial oracle. We found that the technique can achieve mutation scores ranging from 77.78% to 100%, and tends towards higher mutation scores in this range. These results are promising, and suggest that the Regression Model based Metamorphic Relation approach is a viable method of alleviating the oracle problem in implementations of uniformity statistics, and potentially other classes of statistics e.g. correlation statistics

Towards Automatic Generation of Amplified Regression Test Oracles

Regression testing is crucial in ensuring that pure code refactoring does not adversely affect existing software functionality, but it can be expensive, accounting for half the cost of software maintenance. Automated test case generation reduces effort but may generate weak test suites. Test amplification is a promising solution that enhances tests by generating additional or improving existing ones, increasing test coverage, but it faces the test oracle problem. To address this, we propose a test oracle derivation approach that uses object state data produced during System Under Test (SUT) test execution to amplify regression test oracles. The approach monitors the object state during test execution and compares it to the previous version to detect any changes in relation to the SUT's intended behaviour. Our preliminary evaluation shows that the proposed approach can enhance the detection of behaviour changes substantially, providing initial evidence of its effectiveness.Comment: 8 pages, 1 figur