Experiences in Using Practitioner’s Checklists to Evaluate the Relevance of Experiments Reported in Requirements Engineering

Background: Requirements Engineering (RE) researchers recognize that for RE methods to be adopted in industry, practitioners should be able to evaluate the relevance of a study to their practice. Kitchenham et al proposed a set of perspective-based checklists, which demonstrated to be a useful instrument for this purpose. Specifically, the checklist from the practitioner’s perspective seems to be a good candidate for evaluating the relevance of RE studies to RE practice. However, little is known about the applicability of the checklist to the area of RE. Moreover, this checklist also requires a greater analysis about its reliability. Aim: The aim of this report is to propose a perspective-based checklist to the RE community that allows evaluating the relevance of experimental studies in RE from the practitioner’s/consultant’s viewpoint. Method: Our research followed an iterative design-science based approach in which we first analyzed the problems with a previously published checklist and developed an operationalized proposal for a new checklist to counter these problems. We performed a reliability evaluation of this new checklist. The research was performed with two practitioners and 24 papers that report experimental results on comprehensibility of software requirements specifications. Results: This report gives first-hand experiences of practitioners in evaluating the relevance of primary studies in RE, by using a perspective-based checklist. With respect to the reliability of the adjusted checklist, 9 of out 19 questions show an acceptable proportion of agreement (between two practitioners). Conclusions: Based on our experience, the contextualization and operationalization of a perspective-based checklist helps to make it more useful for the practitioners. However, to increase the reliability of the checklist, more reviewers are required and more discussion cycles are necessary. Our plan is to involve at least two more practitioners in order to improve the reliability of the practitioner checklist proposed

Assessing the effectiveness of sequence diagrams in the comprehension of functional requirements: results from a family of five experiments

Modeling is a fundamental activity within the requirements engineering process and concerns the construction of abstract descriptions of requirements that are amenable to interpretation and validation. The choice of a modeling technique is critical whenever it is necessary to discuss the interpretation and validation of requirements. This is particularly true in the case of functional requirements and stakeholders with divergent goals and different backgrounds and experience. This paper presents the results of a family of experiments conducted with students and professionals to investigate whether the comprehension of functional requirements is influenced by the use of dynamic models that are represented by means of the UML sequence diagrams. The family contains five experiments performed in different locations and with 112 participants of different abilities and levels of experience with UML. The results show that sequence diagrams improve the comprehension of the modeled functional requirements in the case of high ability and more experienced participants.The authors wish to thank all the participants in the experiments. This research was partially supported by the MULTIPLE project (with ref. TIN2009-13838).Abrahao Gonzales, SM.; Gravino, .C.; Insfrán Pelozo, CE.; Scaniello, .G.; Tortora, .G. (2013). Assessing the effectiveness of sequence diagrams in the comprehension of functional requirements: results from a family of five experiments. IEEE Transactions on Software Engineering. 39(3):327-342. https://doi.org/10.1109/TSE.2012.27S32734239

ON THE THEORETICAL FOUNDATIONS OF RESEARCH INTO THE UNDERSTANDABILITY OF BUSINESS PROCESS MODELS

Against the background of the growing significance of Business Process Management (BPM) for Information Systems (IS) research and practice, especially the field of Business Process Modeling gains more and more importance. Business process models support communication about as well as the coordination of processes and have become a widely adopted tool in practice. As the understandability of business process models plays a crucial role in communication processes, more and more studies on process model understandability have been conducted in IS research. This article aims at investigating underlying theories of research into business process model understandability by means of an in-depth analysis of 126 systematically retrieved research articles on the topic. It shows in how far process model understandability research is multi-theoretically founded. Identified theories differ regarding addressed subject matters, their coverage, their focus as well as the underlying notion of model understanding, which is exemplarily demonstrated and discussed in this article. Moreover, implications of the findings are discussed and an outlook on future business process model understandability research and on the integration potential of theories in this field is given