Guidelines for conducting interactive rapid reviews in software engineering -- from a focus on technology transfer to knowledge exchange

Evidence-based software engineering (EBSE) aims to improve research utilization in practice. It relies on systematic methods (like systematic literature reviews, systematic mapping studies, and rapid reviews) to identify, appraise, and synthesize existing research findings to answer questions of interest. However, the lack of practitioners' involvement in the design, execution, and reporting of these methods indicates a lack of appreciation for knowledge exchange between researchers and practitioners. Within EBSE, the main reason for conducting these systematic studies is to answer the practitioner's questions and impact practice. However, in many cases, academics have undertaken these studies without any direct involvement of practitioners.This report focuses on the rapid review guidelines and presents practical advice on conducting these with practitioner involvement to facilitate knowledge co-creation.Based on a literature review of rapid reviews and stakeholders engagement in medicine and our experience of using secondary studies in software engineering, we propose extensions to an existing proposal for rapid reviews in software engineering to increase researchers-practitioners knowledge exchange. We refer to the extended method as an interactive rapid review.An interactive rapid review is a streamlined approach to conduct agile literature reviews in close collaboration between researchers and practitioners in software engineering. This report describes the process and discusses possible usage scenarios and some reflections from the proposal's ongoing evaluation.The proposed guidelines will potentially boost knowledge co-creation through active researcher-practitioner interaction by streamlining practitioners' involvement and recognizing the need for an agile process

‘Follow the Moon’ Development: Writing a Systematic Literature Review on Global Software Engineering Education

This presentation reflects on method and practice in Computer Science Education Research, through introducing the process of conducting a Systematic Literature Review. While Systematic Literature Reviews are an established research method within the Software Engineering discipline, they are a relatively unfamiliar research approach within Computer Science Education. Yet research disciplines can be strengthened by borrowing and adapting methods from other fields. I reflect on the rationale and underlying philosophy behind Systematic Reviews, and the implications for conducting a rigorous study and the quality of the resulting outputs. This chronicle of the journey of an ITiCSE working group, outlines the process we adopted and reflects on the methodological and logistical challenges we had to overcome in producing a review titled Challenges and Recommendations for the Design and Conduct of Global Software Engineering Courses. I conclude by discussing how systematic literature reviews can be adapted to an undergraduate teaching setting

Audience and Content Areas of Online Software Engineering Education and Training: A Systematic Review

Online courses and instruction are a popular and rapidly expanding medium for learning. However, no holistic summary exists which classifies who or what is being taught and how online courses are being researched in the field of software engineering education and training (SEET). More important, prior research does not cover what gaps exist in the literature: which areas of SEET being ignored altogether in research. This paper reports the results of a systematic review of the existing literature for online SEET and provides analysis of the audiences and content areas being researched. Grounded in established guidelines for systematic reviews in software engineering, the studies that are identified are then coded using three standards for software engineering education content areas. As a result of the systematic review, more than 9000 search results were analyzed. Inclusion and exclusion of studies in predetermined stages resulted in more than 30 studies being coded for audience and/or software engineering content areas. The result is a comprehensive picture of the current state of research in online SEET and an identification of the gaps to be addressed

Guidelines for the Search Strategy to Update Systematic Literature Reviews in Software Engineering

Context: Systematic Literature Reviews (SLRs) have been adopted within Software Engineering (SE) for more than a decade to provide meaningful summaries of evidence on several topics. Many of these SLRs are now potentially not fully up-to-date, and there are no standard proposals on how to update SLRs in SE. Objective: The objective of this paper is to propose guidelines on how to best search for evidence when updating SLRs in SE, and to evaluate these guidelines using an SLR that was not employed during the formulation of the guidelines. Method: To propose our guidelines, we compare and discuss outcomes from applying different search strategies to identify primary studies in a published SLR, an SLR update, and two replications in the area of effort estimation. These guidelines are then evaluated using an SLR in the area of software ecosystems, its update and a replication. Results: The use of a single iteration forward snowballing with Google Scholar, and employing as a seed set the original SLR and its primary studies is the most cost-effective way to search for new evidence when updating SLRs. Furthermore, the importance of having more than one researcher involved in the selection of papers when applying the inclusion and exclusion criteria is highlighted through the results. Conclusions: Our proposed guidelines formulated based upon an effort estimation SLR, its update and two replications, were supported when using an SLR in the area of software ecosystems, its update and a replication. Therefore, we put forward that our guidelines ought to be adopted for updating SLRs in SE.Comment: Author version of manuscript accepted for publication at the Information and Software Technology Journa

Tool support for systematic reviews in software engineering

Background: Systematic reviews have become an established methodology in software engineering. However, they are labour intensive, error prone and time consuming. These and other challenges have led to the development of tools to support the process. However, there is limited evidence about their usefulness. Aim: To investigate the usefulness of tools to support systematic reviews in software engineering and develop an evaluation framework for an overall support tool. Method: A literature review, taking the form of a mapping study, was undertaken to identify and classify tools supporting systematic reviews in software engineering. Motivated by its results, a feature analysis was performed to independently compare and evaluate a selection of tools which aimed to support the whole systematic review process. An initial version of an evaluation framework was developed to carry out the feature analysis and later refined based on its results. To obtain a deeper understanding of the technology, a survey was undertaken to explore systematic review tools in other domains. Semi-structured interviews with researchers in healthcare and social science were carried out. Quantitative and qualitative data was collected, analysed and used to further refine the framework. Results: The literature review showed an encouraging growth of tools to support systematic reviews in software engineering, although many had received limited evaluation. The feature analysis provided new insight into the usefulness of tools, determined the strongest and weakest candidate and established the feasibility of an evaluation framework. The survey provided knowledge about tools used in other domains, which helped further refine the framework. Conclusions: Tools to support systematic reviews in software engineering are still immature. Their potential, however, remains high and it is anticipated that the need for tools within the community will increase. The evaluation framework presented aims to support the future development, assessment and selection of appropriate tools

Code Smells and Refactoring: A Tertiary Systematic Review of Challenges and Observations

In this paper, we present a tertiary systematic literature review of previous surveys, secondary systematic literature reviews, and systematic mappings. We identify the main observations (what we know) and challenges (what we do not know) on code smells and refactoring. We show that code smells and refactoring have a strong relationship with quality attributes, i.e., with understandability, maintainability, testability, complexity, functionality, and reusability. We argue that code smells and refactoring could be considered as the two faces of a same coin. Besides, we identify how refactoring affects quality attributes, more than code smells. We also discuss the implications of this work for practitioners, researchers, and instructors. We identify 13 open issues that could guide future research work. Thus, we want to highlight the gap between code smells and refactoring in the current state of software-engineering research. We wish that this work could help the software-engineering research community in collaborating on future work on code smells and refactoring