Software industry experiments: a systematic literature review

There is no specialized survey of experiments conducted in the software industry. Goal: Identify the major features of software industry experiments, such as time distribution, independent and dependent variables, subject types, design types and challenges. Method: Systematic literature review, taking the form of a scoping study. Results: We have identified 10 experiments and five quasi-experiments up to July 2012. Most were run as of 2003. The main features of these studies are that they test technologies related to quality and management and analyse outcomes related to effectiveness and effort. Most experiments have a factorial design. The major challenges faced by experimenters are to minimize the cost of running the experiment for the company and to schedule the experiment so as not to interfere with production processes

Open BOK on Software Engineering Educational Context: A Systematic Literature Review

In this review, a Systematic Literature Review (SLR) on Open Body of Knowledge (BOK) is presented. Moreover, the theoretical base to build a model for knowledge description was created, and it was found that there is a lack of guidelines to describe knowledge description because of the dramatically increasing number of requirements to produce an Open BOK, the difficulty of comparing related BOK contents, and the fact that reusing knowledge description is a very laborious task. In this sense, this review can be considered as a first step in building a model that can be used for describing knowledge description in Open BOK. Finally, in order to improve the educational context, a comparison among BOK, structure, and evolution is conducted.This work is supported partially by RTI2018-096846-B-C21 (MCIU/AEI/FEDER, UE) and ADIAN grant IT980-16 (BasqueGovernment)

Variation Factors in the Design and Analysis of Replicated Controlled Experiments - Three (Dis)similar Studies on Inspections versus Unit Testing

Background. In formal experiments on software engineering, the number of factors that may impact an outcome is very high. Some factors are controlled and change by design, while others are are either unforeseen or due to chance. Aims. This paper aims to explore how context factors change in a series of for- mal experiments and to identify implications for experimentation and replication practices to enable learning from experimentation. Method. We analyze three experiments on code inspections and structural unit testing. The first two experiments use the same experimental design and instrumentation (replication), while the third, conducted by different researchers, replaces the programs and adapts defect detection methods accordingly (reproduction). Experimental procedures and location also differ between the experiments. Results. Contrary to expectations, there are significant differences between the original experiment and the replication, as well as compared to the reproduction. Some of the differences are due to factors other than the ones designed to vary between experiments, indicating the sensitivity to context factors in software engineering experimentation. Conclusions. In aggregate, the analysis indicates that reducing the complexity of software engineering experiments should be considered by researchers who want to obtain reliable and repeatable empirical measures