Analyzing the Relationships between Inspections and Testing to Provide a Software Testing Focus

Context: Quality assurance effort, especially testing effort, is frequently a major cost factor during software development. Consequently, one major goal is often to reduce testing effort. One promising way to improve the effectiveness and efficiency of software quality assurance is the use of data from early defect detection activities to provide a software testing focus. Studies indicate that using a combination of early defect data and other product data to focus testing activities outperforms the use of other product data only. One of the key challenges is that the use of data from early defect detection activities (such as inspections) to focus testing requires a thorough understanding of the relationships between these early defect detection activities and testing. An aggravating factor is that these relationships are highly context-specific and need to be evaluated for concrete environments. Objective: The underlying goal of this paper is to help companies get a better understanding of these relationships for their own environment, and to provide them with a methodology for finding relationships in their own environments. Method: This article compares three different strategies for evaluating assumed relationships between inspections and testing. We compare a confidence counter, different quality classes, and the F-measure including precision and recall. Results: One result of this case-study-based comparison is that evaluations based on the aggregated F- measures are more suitable for industry environments than evaluations based on a confidence counter. Moreover, they provide more detailed insights about the validity of the relationships. Conclusion: We have confirmed that inspection results are suitable data for controlling testing activities. Evaluated knowledge about relationships between inspections and testing can be used in the integrated inspection and testing approach In2Test to focus testing activities. Product data can be used in addition. However, the assumptions have to be evaluated in each new context.Peer reviewe

Fault detection and correction modeling of software systems

Ph.DDOCTOR OF PHILOSOPH

The Application of Subjective Estimates of Effectiveness to Controlling Software Inspections

One of the recently proposed tools for controlling software inspections is capture-recapture models. These are models that can be used to estimate the number of remaining defects in a software document after an inspection. Based on this information one can decide whether to reinspect a document to ensure that it is below a prespecified defect density threshold, and that the inspection process itself has attained a minimal level of effectiveness. This line of work has also recently been extended with other techniques, such as the Detection Profile Method. In this paper we investigate an alternative approach: the use of subjective estimates of effectiveness by the inspectors for making the reinspection decision. We performed a study with 30 professional software engineers and found that the median relative error of the engineers' subjective estimates of defect content to be zero, and that the reinspection decision based on that estimate is consistently more correct than the default decision of never reinspecting. This means that subjective estimates provide a good basis for ensuring product quality and inspection process effectiveness during software inspections. Since a subjective estimation procedure can be easily integrated into existing inspection processes, it represents a good starting point for practitioners before introducing more objective decision making criteria by means of capture-recapture models or the Defect Detection Profile Method

The Application of Subjective Estimates of Effectiveness to Controlling Software Inspections

One of the recently proposed tools for controlling software inspections is capture-recapture models. These are models that can be used to estimate the number of remaining defects in a software document after an inspection. Based on this information one can decide whether to reinspect a document to ensure that it is below a prespecified defect density threshold, and that the inspection process itself has attained a minimal level of effectiveness. This line of work has also recently been extended with other techniques, such as the detection profile method (DPM). In this paper, we investigate an alternative approach: the use of subjective estimates of effectiveness by the inspectors for making the reinspection decision. We performed a study with 30 professional software engineers and found that the median relative error of the engineers' subjective estimates of defect content to be zero, and that the reinspection decision based on that estimate is consistently more correct than the default decision of never reinspecting. This means that subjective estimates provide a good basis for ensuring product quality and inspection process effectiveness during software inspections. Since a subjective estimation procedure can be easily integrated into existing inspection processes, it epresents a good starting point for practitioners before introducing more objective decision making criteria by means of capture-recapture models or the defect detection profile method