A Unified Checklist for Observational and Experimental Research in Software Engineering (Version 1)

Current checklists for empirical software engineering cover either experimental research or case study research but ignore the many commonalities that exist across all kinds of empirical research. Identifying these commonalities, and explaining why they exist, would enhance our understanding of empirical research in general and of the differences between experimental and case study research in particular. In this report we design a unified checklist for empirical research, and identify commonalities and differences between experimental and case study research. We design the unified checklist as a specialization of the general engineering cycle, which itself is a special case of the rational choice cycle. We then compare the resulting empirical research cycle with two checklists for experimental research, and with one checklist for case study research. The resulting checklist identifies important questions to be answered in experimental and case study research design and reports. The checklist provides insights in two different types of empirical research design and their relationships. Its limitations are that it ignores other research methods such as meta-research or surveys. It has been tested so far only in our own research designs and in teaching empirical methods. Future work includes expanding the comparison with other methods and application in more cases, by others than ourselves

Preliminary Survey on Empirical Research Practices in Requirements Engineering

Context and Motivation:\ud Based on published output in the premium RE conferences and journals, we observe a growing body of research using both quantitative and qualitative research methods to help understand which RE technique, process or tool work better in which context. Also, more and more empirical studies in RE aim at comparing and evaluating alternative techniques that are solutions to common problems. However, until now there have been few meta studies of the current state of knowledge about common practices carried out by researchers and practitioners in empirical RE. Also, surprisingly little has been published on how RE researchers perceive the usefulness of these best practices.\ud \ud Objective:\ud The goal of our study is to improve our understanding of what empirical practices are performed by researchers and practitioners in RE, for the purpose of understanding the extent to which the research methods of empirical software engineering are adopted in the RE community.\ud \ud Method:\ud We surveyed the practices that participants of the REFSQ conference have been using in their empirical research projects. The survey was part of the REFSQ 2012 Empirical Track.\ud \ud Conclusions:\ud We found that there are 15 commonly used practices out of a set of 27. The study has two implications: first it presents a list of practices that are commonly used in the RE community, and a list of practices that still remain to be practiced. Researchers may now make an informed decision on how to extend the practices they use in producing and executing their research designs, so that their designs get better. Second, we found that senior researchers and PhD students do not always converge in their perceptions about the usefulness of research practices. Whether this is all right and whether something needs to be done in the face of this finding remains an open question

Supporting Defect Causal Analysis in Practice with Cross-Company Data on Causes of Requirements Engineering Problems

[Context] Defect Causal Analysis (DCA) represents an efficient practice to improve software processes. While knowledge on cause-effect relations is helpful to support DCA, collecting cause-effect data may require significant effort and time. [Goal] We propose and evaluate a new DCA approach that uses cross-company data to support the practical application of DCA. [Method] We collected cross-company data on causes of requirements engineering problems from 74 Brazilian organizations and built a Bayesian network. Our DCA approach uses the diagnostic inference of the Bayesian network to support DCA sessions. We evaluated our approach by applying a model for technology transfer to industry and conducted three consecutive evaluations: (i) in academia, (ii) with industry representatives of the Fraunhofer Project Center at UFBA, and (iii) in an industrial case study at the Brazilian National Development Bank (BNDES). [Results] We received positive feedback in all three evaluations and the cross-company data was considered helpful for determining main causes. [Conclusions] Our results strengthen our confidence in that supporting DCA with cross-company data is promising and should be further investigated.Comment: 10 pages, 8 figures, accepted for the 39th International Conference on Software Engineering (ICSE'17

Can using Fagan Inspections improve the quality of specification in 2011? A Case Study

In this paper, we explore why Fagan Inspections have become obsolete in the software industry, given the body of evidence which supports their use to improve the quality of software artefacts and the software development process. Since the late 1970’s, much has been written about how Fagan Inspections improve the quality of both processes and outputs of the software development process. The literature indicates that the Fagan Inspection technique can improve quality of software (or other software development artefacts) by a reduction in defects of 60 – 90%. However, recent literature suggests that inspection techniques in general and Fagan Inspections in particular, are no longer used. A study in 1998 found that respondents used inspections either irregularly or not at all. Teams often review artefacts informally, but believe that they are performing an inspection or formal review. The lack of rigour in the review process results in reduced benefits and more defects in the artefacts. To explore this situation, we conducted a case study with a local enterprise and we report on the early findings. These suggest that the introduction of Fagan Inspections may have a number of benefits before they have even been introduced fully, including recognition of flaws in the current development process, development of technical knowledge relating to the software process domain, and improved team relations and a ‘quality’ culture. In addition, the personnel using Fagan Inspection gain experience in the production of ‘quality’ artefacts

Design science methodology for information systems and software engineering

Textbook on design science methodology. http://link.springer.com/book/10.1007/978-3-662-43839-