Adaptive Random Testing in Detecting Layout Faults of Web Applications

As part of a software testing process, output verification poses a challenge when the output is not numeric or textual, such as graphical. The industry practice of using human oracles (testers) to observe and verify the correctness of the actual results is both expensive and error-prone. In particular, this practice is usually unsustainable when developing web applications - the most popular software of our era. This is because web applications change frequently due to the fast-evolving requirements amid popular demand. To improve the cost effectiveness of browser output verification, in this study we design failure-based testing techniques and evaluate the effectiveness and efficiency thereof in the context of web testing. With a novel application of the concept of adaptive random sequence (ARS), our approach leverages peculiar characteristics of failure patterns found in browser layout rendering. An empirical study shows that the use of failure patterns and inclination to guide the testing flow leads to more cost-effective results than other classic methods. This study extends the application of ARSs from the input space of programs to their output space, and also shows that adaptive random testing (ART) can outperform random testing (RT) in both failure detection effectiveness (in terms of F-measure) and failure detection efficiency (in terms of execution time)

Testing model transformation programs using metamorphic testing

Model transformations are crucial for the success of Model Driven Engineering. Testing is a prevailing technique of verifying the correctness of model transformation programs. A major challenge in model transformation testing is the oracle problem, which refers to the difficulty or high cost in determining the correctness of the output models. Metamorphic Testing alleviates the oracle problem by making use of the relationships among the inputs and outputs of multiple executions of the target function. This paper investigates the effectiveness and feasibility of metamorphic testing in testing model transformation programs. Empirical results show that metamorphic testing is an effective testing method for model transformation programs

A similarity metric for the inputs of OO programs and its application in adaptive random testing

Random testing (RT) has been identified as one of the most popular testing techniques, due to its simplicity and ease of automation. Adaptive random testing (ART) has been proposed as an enhancement to RT, improving its fault-detection effectiveness by evenly spreading random test inputs across the input domain. To achieve the even spreading, ART makes use of distance measurements between consecutive inputs. However, due to the nature of object-oriented software (OOS), its distance measurement can be particularly challenging: Each input may involve multiple classes, and interaction of objects through method invocations. Two previous studies have reported on how to test OOS at a single-class level using ART. In this study, we propose a new similarity metric to enable multiclass level testing using ART. When generating test inputs (for multiple classes, a series of objects, and a sequence of method invocations), we use the similarity metric to calculate the distance between two series of objects, and between two sequences of method invocations. We integrate this metric with ART and apply it to a set of open-source OO programs, with the empirical results showing that our approach outperforms other RT and ART approaches in OOS testing

Metamorphic testing for cybersecurity

Metamorphic testing (MT) can enhance security testing by providing an alternative to using a testing oracle, which is often unavailable or impractical. The authors report how MT detected previously unknown bugs in real-world critical applications such as code obfuscators, giving evidence that software testing requires diverse perspectives to achieve greater cybersecurity

A revisit of three studies related to random testing

Software testing is an approach that ensures the quality of software through execution, with a goal being to reveal failures and other problems as quickly as possible. Test case selection is a fundamental issue in software testing, and has generated a large body of research, especially with regards to the effectiveness of random testing (RT), where test cases are randomly selected from the software’s input domain. In this paper, we revisit three of our previous studies. The first study investigated a sufficient condition for partition testing (PT) to outperform RT, and was motivated by various controversial and conflicting results suggesting that sometimes PT performed better than RT, and sometimes the opposite. The second study aimed at enhancing RT itself, and was motivated by the fact that RT continues to be a fundamental and popular testing technique. This second study enhanced RT fault detection effectiveness by making use of the common observation that failure-causing inputs tend to cluster together, and resulted in a new family of RT techniques: adaptive random testing (ART), which is random testing with an even spread of test cases across the input domain. Following the successful use of failure-causing region contiguity insights to develop ART, we conducted a third study on how to make use of other characteristics of failure-causing inputs to develop more effective test case selection strategies. This third study revealed how best to approach testing strategies when certain characteristics of the failure-causing inputs are known, and produced some interesting and important results. In revisiting these three previous studies, we explore their unexpected commonalities, and identify diversity as a key concept underlying their effectiveness. This observation further prompted us to examine whether or not such a concept plays a role in other areas of software testing, and our conclusion is that, yes, diversity appears to be one of the most important concepts in the field of software testing

Exercise training with negative pressure ventilation improves exercise capacity in patients with severe restrictive lung disease: a prospective controlled study

BACKGROUND: Exercise training is of benefit for patients with restrictive lung disease. However, it tends to be intolerable for those with severe disease. We examined whether providing ventilatory assistance by using negative pressure ventilators (NPV) during exercise training is feasible for such patients and the effects of training. METHODS: 36 patients with restrictive lung disease were prospectively enrolled for a 12-week multidisciplinary rehabilitation program. During this program, half of them (n:18; 60.3 ± 11.6 years; 6 men; FVC: 32.5 ± 11.7% predicted ) received regular sessions of exercise training under NPV, whilst the 18 others (59.6 ± 12.3 years; 8 men; FVC: 37.7 ± 10.2% predicted) did not. Exercise capacity, pulmonary function, dyspnea and quality of life were measured. The primary endpoint was the between-group difference in change of 6 minute-walk distance (6MWD) after 12 weeks of rehabilitation. RESULTS: All patients in the NPV-exercise group were able to tolerate and completed the program. The between-group differences were significantly better in the NPV-exercise group in changes of 6MWD (34.1 ± 12.7 m vs. -32.5 ± 17.5 m; P = 0.011) and St George Score (−14.5 ± 3.6 vs. 11.8 ± 6.0; P < 0.01). There was an improvement in dyspnea sensation (Borg’s scale, from 1.4 ± 1.5 point to 0.8 ± 1.3 point, P = 0.049) and a small increase in FVC (from 0.85 ± 0.09 L to 0.91 ± 0.08 L, P = 0.029) in the NPV-exercise group compared to the control group. CONCLUSION: Exercise training with NPV support is feasible for patients with severe restrictive lung diseases, and improves exercise capacity and health-related quality of life

Test case prioritization for object-oriented software: an adaptive random sequence approach based on clustering

Test case prioritization (TCP) attempts to improve fault detection effectiveness by scheduling the important test cases to be executed earlier, where the importance is determined by some criteria or strategies. Adaptive random sequences (ARSs) can be used to improve the effectiveness of TCP based on white-box information (such as code coverage information) or black-box information (such as test input information). To improve the testing effectiveness for object-oriented software in regression testing, in this paper, we present an ARS approach based on clustering techniques using black-box information. We use two clustering methods: (1) clustering test cases according to the number of objects and methods, using the K-means and K-medoids clustering algorithms; and (2) clustered based on an object and method invocation sequence similarity metric using the K-medoids clustering algorithm. Our approach can construct ARSs that attempt to make their neighboring test cases as diverse as possible. Experimental studies were also conducted to verify the proposed approach, with the results showing both enhanced probability of earlier fault detection, and higher effectiveness than random prioritization and method coverage TCP technique