A formal evaluation of data flow path selection criteria

A number of path selection criteria have been proposed throughout the years. Unfortunately, little work has been done on comparing these criteria. To determine what would be an effective path selection criterion for revealing errors in programs, we have undertaken an evaluation of these criteria. This paper reports on the results of our evaluation of path selection criteria based on data flow relationships. We show how these criteria relate to each other, thereby demonstrating some of their strengths and weaknesses. In addition, we suggest minor changes to some criteria that improve their performance. We conclude with a discussion of the major limitations of these criteria and directions for future research

Estudio de la Efectividad de Tres Técnicas de Evaluación de Código: Resultados de una Serie de Experimentos.

Hasta la fecha se han evaluado distintas técnicas de verificación y validación teórica y empíricamente. La mayoría de las evaluaciones empíricas se han llevado a cabo sin sujetos, abstrayendo el efecto del sujeto sobre la técnica a la hora de aplicarla. Hemos evaluado mediante un experimento con sujetos la efectividad de tres técnicas de verificación y validación de código: partición en clases de equivalencia, cobertura de decisión y lectura de código mediante abstracciones sucesivas, estudiando la capacidad de las técnicas para la detección de fallos en tres programas distintos. Hemos replicado el experimento ocho veces en cuatro entornos distintos. Los resultados arrojan diferencias entre las técnicas y señalan variables contextuales del proyecto software que deberían considerarse cuando se quiera elegir o aplicar una técnica de verificación y validación

Comparing the effectiveness of equivalence partitioning, branch testing and code reading by stepwise abstraction applied by subjects

Some verification and validation techniques have been evaluated both theoretically and empirically. Most empirical studies have been conducted without subjects, passing over any effect testers have when they apply the techniques. We have run an experiment with students to evaluate the effectiveness of three verification and validation techniques (equivalence partitioning, branch testing and code reading by stepwise abstraction). We have studied how well able the techniques are to reveal defects in three programs. We have replicated the experiment eight times at different sites. Our results show that equivalence partitioning and branch testing are equally effective and better than code reading by stepwise abstraction. The effectiveness of code reading by stepwise abstraction varies significantly from program to program. Finally, we have identified project contextual variables that should be considered when applying any verification and validation technique or to choose one particular technique

Estudio de la Efectividad de Tres Técnicas de Evaluación de Código: Resultados de una Serie de Experimentos

Hasta la fecha se han evaluado distintas técnicas de verificación y validación teórica y empíricamente. La mayoría de las evaluaciones empíricas se han llevado a cabo sin sujetos, abstrayendo el efecto del sujeto sobre la técnica a la hora de aplicarla. Hemos evaluado mediante un experimento con sujetos la efectividad de tres técnicas de verificación y validación de código: partición en clases de equivalencia, cobertura de decisión y lectura de código mediante abstracciones sucesivas, estudiando la capacidad de las técnicas para la detección de fallos en tres programas distintos. Hemos replicado el experimento ocho veces en cuatro entornos distintos. Los resultados arrojan diferencias entre las técnicas y señalan variables contextuales del proyecto software que deberían considerarse cuando se quiera elegir o aplicar una técnica de verificación y validación.Ministerio de Ciencia e Innovación TIN2008-0055

Error flow in computer programs

White box program analysis has been applied to program testing for some time, but this analysis is primarily grounded in program syntax, while errors arise from incorrect program semantics. We introduce a semantically-based technique called error flow analysis, which is used to investigate the behavior of a program at the level of data state transitions. Error flow analysis is based on a model of program execution as a composition of functions that each map a prior data state into a subsequent data state. According to the fault/failure model, failure occurs when a fault causes an infection in the data state which then propagates to output. A faulty program may also produce coincidentally correct output for a given input if the fault resists infection, or an infection is cancelled by subsequent computation. We investigate this phenomenon using dynamic error flow analysis to track the infection and propagation of errors in the data states of programs with seeded faults. This information is gathered for a particular fault over many inputs on a path-by-path basis to estimate execution, infection, and failure rates as well as characteristics of error flow behavior for the fault. Those paths that exhibit high failure rates would be more desirable to test for this fault than those with lower failure rates, and we look for error flow characteristics that correlate with high failure rate. We present the results of dynamic error flow experiments on several programs, and suggest ways in which error flow information can be used in program analysis and testing

Análise de mutantes e critérios restritos no contexto de teste de software : resultados de uma avalição empírica

Orientadora: Silvia Regina VergilioDissertação (mestrado) - Universidade Federal do Paran