Code coverage of adaptive random testing

Random testing is a basic software testing technique that can be used to assess the software reliability as well as to detect software failures. Adaptive random testing has been proposed to enhance the failure-detection capability of random testing. Previous studies have shown that adaptive random testing can use fewer test cases than random testing to detect the first software failure. In this paper, we evaluate and compare the performance of adaptive random testing and random testing from another perspective, that of code coverage. As shown in various investigations, a higher code coverage not only brings a higher failure-detection capability, but also improves the effectiveness of software reliability estimation. We conduct a series of experiments based on two categories of code coverage criteria: structure-based coverage, and fault-based coverage. Adaptive random testing can achieve higher code coverage than random testing with the same number of test cases. Our experimental results imply that, in addition to having a better failure-detection capability than random testing, adaptive random testing also delivers a higher effectiveness in assessing software reliability, and a higher confidence in the reliability of the software under test even when no failure is detected

How can non-technical end users effectively test their spreadsheets?

Purpose – An alarming number of spreadsheet faults have been reported in the literature, indicating that effective and easy-to-apply spreadsheet testing techniques are not available for “non-technical,” end-user programmers. The purpose of this paper is to alleviate the problem by introducing a metamorphic testing (MT) technique for spreadsheets. Design/methodology/approach – The paper discussed four common challenges encountered by end-user programmers when testing a spreadsheet. The MT technique was then discussed and how it could be used to solve the common challenges was explained. An experiment involving several “real-world” spreadsheets was performed to determine the viability and effectiveness of MT. Findings – The experiment confirmed that MT is highly effective in spreadsheet fault detection, and yet MT is a general technique that can be easily used by end-user programmers to test a large variety of spreadsheet applications. Originality/value – The paper provides a detailed discussion of some common challenges of spreadsheet testing encountered by end-user programmers. To the best of the authors knowledge, the paper is the first that includes an empirical study of how effective MT is in spreadsheet fault detection from an end-user programmer's perspective

How effectively does metamorphic testing alleviate the oracle problem?

In software testing, something which can verify the correctness of test case execution results is called an oracle. The oracle problem occurs when either an oracle does not exist, or exists but is too expensive to be used. Metamorphic testing is a testing approach which uses metamorphic relations, properties of the software under test represented in the form of relations among inputs and outputs of multiple executions, to help verify the correctness of a program. This paper presents new empirical evidence to support this approach, which has been used to alleviate the oracle problem in various applications and to enhance several software analysis and testing techniques. It has been observed that identification of a sufficient number of appropriate metamorphic relations for testing, even by inexperienced testers, was possible with a very small amount of training. Furthermore, the cost-effectiveness of the approach could be enhanced through the use of more diverse metamorphic relations. The empirical studies presented in this paper clearly show that a small number of diverse metamorphic relations, even those identified in an ad hoc manner, had a similar fault-detection capability to a test oracle, and could thus effectively help alleviate the oracle problem

Exercise-Induced Changes in Exhaled NO Differentiates Asthma With or Without Fixed Airway Obstruction From COPD With Dynamic Hyperinflation.

Asthmatic patients with fixed airway obstruction (FAO) and patients with chronic obstructive pulmonary disease (COPD) share similarities in terms of irreversible pulmonary function impairment. Exhaled nitric oxide (eNO) has been documented as a marker of airway inflammation in asthma, but not in COPD. To examine whether the basal eNO level and the change after exercise may differentiate asthmatics with FAO from COPD, 27 normal subjects, 60 stable asthmatics, and 62 stable COPD patients were studied. Asthmatics with FAO (n = 29) were defined as showing a postbronchodilator FEV(1)/forced vital capacity (FVC) ≤70% and FEV(1) less than 80% predicted after inhaled salbutamol (400 μg). COPD with dynamic hyperinflation (n = 31) was defined as a decrease in inspiratory capacity (ΔIC%) after a 6 minute walk test (6MWT). Basal levels of eNO were significantly higher in asthmatics and COPD patients compared to normal subjects. The changes in eNO after 6MWT were negatively correlated with the percent change in IC (r = −0.380, n = 29, P = 0.042) in asthmatics with FAO. Their levels of basal eNO correlated with the maximum mid-expiratory flow (MMEF % predicted) before and after 6MWT. In COPD patients with air-trapping, the percent change of eNO was positively correlated to ΔIC% (rs = 0.404, n = 31, P = 0.024). We conclude that asthma with FAO may represent residual inflammation in the airways, while dynamic hyperinflation in COPD may retain NO in the distal airspace. eNO changes after 6MWT may differentiate the subgroups of asthma or COPD patients and will help toward delivery of individualized therapy for airflow obstruction

Automated functional testing of online search services

Search services are the main interface through which people discover information on the Internet. A fundamental challenge in testing search services is the lack of oracles. The sheer volume of data on the Internet prohibits testers from verifying the results. Furthermore, it is difficult to objectively assess the ranking quality because different assessors can have very different opinions on the relevance of a Web page to a query. This paper presents a novel method for automatically testing search services without the need of a human oracle. The experimental findings reveal that some commonly used search engines, including Google, Yahoo!, and Live Search, are not as reliable as what most users would expect. For example, they may fail to find pages that exist in their own repositories, or rank pages in a way that is logically inconsistent. Suggestions are made for search service providers to improve their service quality. Copyright © 2010 John Wiley & Sons, Ltd. A novel method for automatically testing search services without the need of a human oracle is presented. The experimental findings reveal that some commonly used search engines, including Google, Yahoo!, and Live Search, are not as reliable as what most users would expect. For example, they may fail to find pages that exist in their own repositories, or rank pages in a way that is logically inconsistent. Suggestions are made for search service providers to improve their service quality. Copyright © 2010 John Wiley & Sons, Ltd.link_to_subscribed_fulltex

A Mobile App Platform for Discovering Learning Profiles and Analytics

published_or_final_versio

Exploring Characteristics of Fine-Grained Behaviors of Learning Mathematics in Tablet-Based E-Learning Activities

Attributes of teaching and learning contexts provide rich information about how students participate in learning activities. By tracking and analyzing snapshots of these attributes captured continuously throughout the duration of the learning activities, teachers can identify individual students who need special attention and apply different pedagogical actions to them. This paper describes the results of the work-in-progress study in exploring characteristics of fine-grained behaviors of learning mathematics in tablet-based e-learning activities. An experimental platform called SkyApp is built. Through SkyApp, teachers can create e-learning activities and track learning records of students after the delivery of the activities. SkyApp supports capturing, storing and analyzing of fine-grained behaviors of students. Pilot tests have been done in two primary schools for eight months. The review of the tests demonstrates the potential in performing learning analytics. By applying clustering algorithms on multiple learning metrics of marks, time and number of attempts for students in solving mathematics questions, classification of students by learning characteristics of performance and engagement can be formulated. [For the complete proceedings, see ED579335.]published_or_final_versio

Human Competitiveness of Genetic Programming in Spectrum-Based Fault Localisation: Theoretical and Empirical Analysis

We report on the application of Genetic Programming to Software Fault Localisation, a problem in the area of Search-Based Software Engineering (SBSE). We give both empirical and theoretical evidence for the human competitiveness of the evolved fault localisation formulæ under the single fault scenario, compared to those generated by human ingenuity and reported in many papers, published over more than a decade. Though there have been previous human competitive results claimed for SBSE problems, this is the first time that evolved solutions have been formally proved to be human competitive. We further prove that no future human investigation could outperform the evolved solutions. We complement these proofs with an empirical analysis of both human and evolved solutions, which indicates that the evolved solutions are not only theoretically human competitive, but also convey similar practical benefits to human-evolved counterparts