Identifying, categorizing and mitigating threats to validity in software engineering secondary studies

Context: Secondary studies are vulnerable to threats to validity. Although, mitigating these threats is crucial for the credibility of these studies, we currently lack a systematic approach to identify, categorize and mitigate threats to validity for secondary studies. Objective: In this paper, we review the corpus of secondary studies, with the aim to identify: (a) the trend of reporting threats to validity, (b) the most common threats to validity and corresponding mitigation actions, and (c) possible categories in which threats to validity can be classified. Method: To achieve this goal we employ the tertiary study research method that is used for synthesizing knowledge from existing secondary studies. In particular, we collected data from more than 100 studies, published until December 2016 in top quality software engineering venues (both journals and conference). Results: Our results suggest that in recent years, secondary studies are more likely to report their threats to validity. However, the presentation of such threats is rather ad hoc, e.g., the same threat may be presented with a different name, or under a different category. To alleviate this problem, we propose a classification schema for reporting threats to validity and possible mitigation actions. Both the classification of threats and the associated mitigation actions have been validated by an empirical study, i.e., Delphi rounds with experts. Conclusion: Based on the proposed schema, we provide a checklist, which authors of secondary studies can use for identifying and categorizing threats to validity and corresponding mitigation actions, while readers of secondary studies can use the checklist for assessing the validity of the reported results

How tertiary studies perform quality assessment of secondary studies in software engineering

Best Paper Award a l’Experimental Software Engineering Track (ESELAW) de la XXIV Ibero-American Conference on Software Engineering, CIbSE 2021Context: Tertiary studies are becoming increasingly popular in software engineering as an instrument to synthesise evidence on a research topic in a systematic way. In order to understand and contextualize their findings, it is important to assess the quality of the selected secondary studies. Objective: This paper aims to provide a state of the art on the assessment of secondary studies’ quality as conducted in tertiary studies in the area of software engineering, reporting the frameworks used as instruments, the facets examined in these frameworks, and the purposes of the quality assessment. Method: We designed this study as a systematic mapping responding to four research questions derived from the objective above. We applied a rigorous search protocol over the Scopus digital library, resulting in 47 papers after application of inclusion and exclusion criteria. The extracted data was synthesised using content analysis. Results: A majority of tertiary studies perform quality assessment. It is not often used for excluding studies, but to support some kind of investigation. The DARE quality assessment framework is the most frequently used, with customizations in some cases to cover missing facets. We outline the first steps towards building a new framework to address the shortcomings identified. Conclusion: This paper is a step forward establishing a foundation for researchers in two different ways. As authors of tertiary studies, understanding the different possibilities in which they can perform quality assessment of secondary studies. As readers, having an instrument to understand the methodological rigor upon which tertiary studies may claim their findings.Peer ReviewedAward-winningPostprint (author's final draft

Systems Interoperability Types: A Tertiary Study

Interoperability has been a focus of attention over at least four decades, with the emergence of several interoperability types (or levels), diverse models, frameworks, and solutions, also as a result of a continuous effort from different domains. The current heterogeneity in technologies such as blockchain, IoT and new application domains such as Industry 4.0 brings not only new interaction possibilities but also challenges for interoperability. Moreover, confusion and ambiguity in the current understanding of interoperability types exist, hampering stakeholders' communication and decision making. This work presents an updated panorama of software-intensive systems interoperability with particular attention to its types. For this, we conducted a tertiary study that scrutinized 37 secondary studies published from 2012 to 2023, from which we found 36 interoperability types associated with 117 different definitions, besides 13 interoperability models and six frameworks in various domains. This panorama reveals that the concern with interoperability has migrated from technical to social-technical issues going beyond the software systems' boundary and still requiring solving many open issues. We also address the urgent actions and also potential research opportunities to leverage interoperability as a multidisciplinary research field to achieve low-coupled, cost-effective, and interoperable systems.Comment: 33 page

Computer Game Scenario Representation: A Systematic Mapping Study

Background: Game scenario is an important factor for achieving player enjoyment; consisting a key business success factor. Additionally, the production of early design artifacts is crucial for the success of the development process. However, representing scenarios is a non-trivial task: (a) multiple aspects of the game need to be visualized; and (b) there is a plethora of representation approaches, out of which the game designer needs to select from. Aim: The goal of this work is to provide a panorama of the current scenario representation approaches, to aid game engineers in selecting the most fitting scenario representation approach and understand the existing designing options. Method: We have performed a Systematic Mapping Study, using 4 digital libraries, since the main goal can be achieved through study classification. By following an established search and filtering process, we have identified 717 articles, and analyzed in detail 95. Results: Diagrams are the most common generic approach to represent scenario; Game story is the most usual part of the scenario being represented; Characters are the most common component; and Transitions are the most usual connectors. Conclusion: Researchers may get useful information for empirically investigating several game engineering aspects; whereas game engineers can efficiently select the most fitting approach

Association of Surgical Site Infections with Hospital Size and Urban and Rural Locations in California

Approximately 20% of health care-associated infections (HAIs) result from surgical site infections (SSIs), increasing health costs, morbidity, and mortality. The mandates of public reporting of HAIs through the Centers of Medicare and Medicaid Services can have consequential effects on hospitals’ reimbursement, a significant concern to health care administrators. The primary purpose of this quantitative study was to determine if urban and rural locations and the size of hospitals affect the rate of SSIs in California. The two research questions addressed whether there is an association between the rate of SSIs in urban and rural locations and the size of hospitals in California. The Donabedian model was used as the conceptual framework to examine quality of care by focusing on implementation, physical facility, and processes. Data from the California Department of Public Health was analyzed using Mann-Whitney U and Kruskal-Wallis tests to determine the association between the rate of SSIs in rural and urban locations and the size of hospitals. The 2019 data set contained 28 types of surgical procedures from 336 rural and urban hospitals in 52 California counties. A higher rate of SSIs was evident in urban than rural hospitals. The rate of SSIs was also higher in large-sized hospitals than in medium and small hospitals. Research recommendations include exploring the rate of SSIs in different regions using data obtained from hospitals. This study may serve as a guide for health care administrators to understand the interplay of the size and location of hospitals and prevalence of SSIs. This understanding may also highlight community disparities leading to positive social change through the development of programs to improve quality care, reduce financial burden, and attain patient satisfaction