Relational-model based change management for non-functional requirements: approach and experiment

In software industry, many organizations either focus their traceability efforts on Functional Requirements (FRs) or else fail entirely to implement an effective traceability process. NonFunctional Requirements (NFRs) such as security, safety, performance, and reliability are treated in a rather ad hoc fashion and are rarely traced. This is mainly because of the unique nature of NFRs. They are subjective, relative and they tend to become scattered among multiple modules when they are mapped from the requirements domain to the solution space. Furthermore, NFRs can often interact, in the sense that attempts to achieve one NFR can help or hinder the achievement of other NFRs at particular software functionality. Such an interaction creates an extensive network of interdependencies and tradeoffs among NFRs which is not easy to trace. In a previous work, we proposed a conceptualization of NFRs through the NFRs Ontology. In this paper, we extend the previous work by proposing a change management mechanism for tracing the impact of NFRs on the other constructs in the ontology such as FR or NFR operationalization and vice versa, and providing a traceability mechanism using Datalog expressions to imp lement queries on a relational model-based representation for the ontology. The proposed traceability queries are then evaluated through a multiproject variation quasi-experiment on regression testing conducted in the industry

Estudio de la Efectividad de Tres Técnicas de Evaluación de Código: Resultados de una Serie de Experimentos.

Hasta la fecha se han evaluado distintas técnicas de verificación y validación teórica y empíricamente. La mayoría de las evaluaciones empíricas se han llevado a cabo sin sujetos, abstrayendo el efecto del sujeto sobre la técnica a la hora de aplicarla. Hemos evaluado mediante un experimento con sujetos la efectividad de tres técnicas de verificación y validación de código: partición en clases de equivalencia, cobertura de decisión y lectura de código mediante abstracciones sucesivas, estudiando la capacidad de las técnicas para la detección de fallos en tres programas distintos. Hemos replicado el experimento ocho veces en cuatro entornos distintos. Los resultados arrojan diferencias entre las técnicas y señalan variables contextuales del proyecto software que deberían considerarse cuando se quiera elegir o aplicar una técnica de verificación y validación

A systematic review on regression test selection techniques

Regression testing is verifying that previously functioning software remains after a change. With the goal of finding a basis for further research in a joint industry-academia research project, we conducted a systematic review of empirical evaluations of regression test selection techniques. We identified 27 papers reporting 36 empirical studies, 21 experiments and 15 case studies. In total 28 techniques for regression test selection are evaluated. We present a qualitative analysis of the findings, an overview of techniques for regression test selection and related empirical evidence. No technique was found clearly superior since the results depend on many varying factors. We identified a need for empirical studies where concepts are evaluated rather than small variations in technical implementations

A test case generation framework based on UML statechart diagram

Early software fault detection offers more flexibility to correct errors in the early development stages. Unfortunately, existing studies in this domain are not sufficiently comprehensive in describing the major processes of the automated test case generation. Furthermore, the algorithms used for test case generation are not provided or well described. Current studies also hardly address loops and parallel paths issues, and achieved low coverage criteria. Therefore, this study proposes a test case generation framework that generates minimized and prioritized test cases from UML statechart diagram with higher coverage criteria. This study, conducted a review of the previous research to identify the issues and gaps related to test case generation, model-based testing, and coverage criteria. The proposed framework was designed from the gathered information based on the reviews and consists of eight components that represent a comprehensive test case generation processes. They are relation table, relation graph, consistency checking, test path minimization, test path prioritization, path pruning, test path generation, and test case generation. In addition, a prototype to implement the framework was developed. The evaluation of the framework was conducted in three phases: prototyping, comparison with previous studies, and expert review. The results reveal that the most suitable coverage criteria for UML statechart diagram are all-states coverage, all-transitions coverage, alltransition-pairs coverage, and all-loop-free-paths coverage. Furthermore, this study achieves higher coverage criteria in all coverage criteria, except for all-state coverage, when compared with the previous studies. The results of the experts’ review show that the framework is practical, easy to implement due to it is suitability to generate the test cases. The proposed algorithms provide correct results, and the prototype is able to generate test case effectively. Generally, the proposed system is well accepted by experts owing to its usefulness, usability, and accuracy. This study contributes to both theory and practice by providing an early alternative test case generation framework that achieves high coverage and can effectively generate test cases from UML statechart diagrams. This research adds new knowledge to the software testing field, especially for testing processes in the model-based techniques, testing activity, and testing tool support

An improved method for test case prioritization by incorporating historical test case data

AbstractTest case prioritization reorders test cases from a previous version of a software system for the current release to optimize regression testing. We have previously introduced a technique for test case prioritization using historical test case performance data. The technique was based on a test case prioritization equation, which directly computes the priority of each test case using the historical information of the test case using an equation with constant coefficients. This technique was compared just with random ordering approach. In this paper, we present an enhancement of the aforementioned technique in two ways. First, we propose a new prioritization equation with variable coefficients gained according to the available historical performance data, which acts as a feedback from the previous test sessions. Second, a family of comprehensive empirical studies has been conducted to evaluate the performance of the technique. We have compared the proposed technique with our previous technique and the technique proposed by Kim and Porter. The experimental results demonstrate the effectiveness of the proposed technique in accelerating the rate of fault detection in history-based test case prioritization

Estudio de la Efectividad de Tres Técnicas de Evaluación de Código: Resultados de una Serie de Experimentos

Hasta la fecha se han evaluado distintas técnicas de verificación y validación teórica y empíricamente. La mayoría de las evaluaciones empíricas se han llevado a cabo sin sujetos, abstrayendo el efecto del sujeto sobre la técnica a la hora de aplicarla. Hemos evaluado mediante un experimento con sujetos la efectividad de tres técnicas de verificación y validación de código: partición en clases de equivalencia, cobertura de decisión y lectura de código mediante abstracciones sucesivas, estudiando la capacidad de las técnicas para la detección de fallos en tres programas distintos. Hemos replicado el experimento ocho veces en cuatro entornos distintos. Los resultados arrojan diferencias entre las técnicas y señalan variables contextuales del proyecto software que deberían considerarse cuando se quiera elegir o aplicar una técnica de verificación y validación.Ministerio de Ciencia e Innovación TIN2008-0055

The empire infrastructure for testing experimentation

It is widely believed that there is not enough experimentation in the field of computer science. One area in particular in which additional experimentation is needed involves regression testing of large, real software. Regression testing is the expensive process of validating changes made to previously tested software. One reason why there is not enough experimentation in this area is the substantial infrastructure required for such experimentation. We have developed an infrastructure designed for performing various testing experiments on a large, real test subject. This paper describes the requirements, design, and implementation issues associated with the development of our infrastructure. We also performed an experiment to validate the infrastructure, in which we investigated the affects test granularity has on regression test selection techniques. The experiment shows the usefulness of our infrastructure. We believe that our infrastructure can serve as a prototype for future development of similar infrastructures involving other large test subjects

Exploring regression testing and software product line testing - research and state of practice

In large software organizations with a product line development approach a selective testing of product variants is necessary in order to keep pace with the decreased development time for new products, enabled by the systematic reuse. The close relationship between products in product line indicates an option to reduce the testing effort due to redundancy. In many cases test selection is performed manually, based on test leaders’ expertise. This makes the cost and quality of the testing highly dependent on the skills and experience of the test leaders. There is a need in industry for systematic approaches to test selection. The goal of our research is to improve the control of the testing and reduce the amount of redundant testing in the product line context by applying regression test selection strategies. In this thesis, the state of art of regression testing and software product line testing are explored. Two extensive systematic reviews are conducted as well as an industrial survey of regression testing state of practice and an industrial evaluation of a pragmatic regression test selection strategy. Regression testing is not an isolated one-off activity, but rather an activity of varying scope and preconditions, strongly dependent on the context in which it is applied. Several techniques for regression test selection are proposed and evaluated empirically but in many cases the context is too specific for a technique to be easily applied directly by software developers. In order to improve the possibility for generalizing empirical results on regression test selection, guidelines for reporting the testing context are discussed in this thesis. Software product line testing is a relatively new research area. The understanding about challenges is well established but when looking for solutions to these challenges, we mostly find proposals, and empirical evaluations are sparse. Regression test selection strategies proposed in literature are not easily applicable in the product line context. Instead, control may be increased by increased visibility of the effects of testing and proper measurements of software quality. Focus of our future work will be on how to guide the planning and assessment of regression testing activities in large, complex reuse based systems, by visualizing the quality achieved in different parts of the system and evaluating the effects of different selection strategies when applied in various regression testing situations