Applying Bayesian Analysis Guidelines to Empirical Software Engineering Data: The Case of Programming Languages and Code Quality

Statistical analysis is the tool of choice to turn data into information, and then information into empirical knowledge. To be valid, the process that goes from data to knowledge should be supported by detailed, rigorous guidelines, which help ferret out issues with the data or model, and lead to qualified results that strike a reasonable balance between generality and practical relevance. Such guidelines are being developed by statisticians to support the latest techniques for Bayesian data analysis. In this article, we frame these guidelines in a way that is apt to empirical research in software engineering. To demonstrate the guidelines in practice, we apply them to reanalyze a GitHub dataset about code quality in different programming languages. The dataset's original analysis (Ray et al., 2014) and a critical reanalysis (Berger at al., 2019) have attracted considerable attention -- in no small part because they target a topic (the impact of different programming languages) on which strong opinions abound. The goals of our reanalysis are largely orthogonal to this previous work, as we are concerned with demonstrating, on data in an interesting domain, how to build a principled Bayesian data analysis and to showcase some of its benefits. In the process, we will also shed light on some critical aspects of the analyzed data and of the relationship between programming languages and code quality. The high-level conclusions of our exercise will be that Bayesian statistical techniques can be applied to analyze software engineering data in a way that is principled, flexible, and leads to convincing results that inform the state of the art while highlighting the boundaries of its validity. The guidelines can support building solid statistical analyses and connecting their results, and hence help buttress continued progress in empirical software engineering research

The Impact of Requirements on Systems Development Speed: A Multiple-Case Study in Automotive

Automotive\ua0manufacturers have historically adopted rigid\ua0requirements\ua0engineering processes. This allowed them to meet safety-critical\ua0requirements\ua0when producing\ua0a\ua0highly complex and differentiated product out of the integration of thousands of physical and software components. Nowadays, few software-related domains are as rapidly changing as the\ua0automotive\ua0industry.\ua0In\ua0particular, the needs of improving\ua0development\ua0speed\ua0are increasingly pushing companies\ua0in\ua0this domain toward new ways of developing software.\ua0In\ua0this paper, we investigate how the goal to increase\ua0development\ua0speed\ua0impacts how\ua0requirements\ua0are managed\ua0in\ua0the\ua0automotive\ua0domain. We start from\ua0a\ua0manager perspective, which we then complement with\ua0a\ua0more general perspective. We used\ua0a\ua0qualitative\ua0multiple-case\ua0study, organized\ua0in\ua0two steps.\ua0In\ua0the first step, we had 20 semi-structured interviews, at two\ua0automotive\ua0manufacturers. Our sampling strategy focuses on manager roles, complemented with technical specialists.\ua0In\ua0the second step, we validated our results with 12 more interviews, covering nine additional respondents and three recurring from the first step.\ua0In\ua0addition to validating our qualitative model, the second step of interviews broadens our perspective with technical experts and change managers. Our respondents indicate and rank six aspects of the current\ua0requirements\ua0engineering approach that\ua0impact\ua0development\ua0speed. These aspects include the negative\ua0impact\ua0of\ua0a\ua0requirements\ua0style dominated by safety concerns as well as decomposition of\ua0requirements\ua0over many levels of abstraction. Furthermore, the use of\ua0requirements\ua0as part of legal contracts with suppliers is seen as hindering fast collaboration. Six additional suggestions for potential improvements include domain-specific tooling, model-based\ua0requirements, test automation, and\ua0a\ua0combination of lightweight upfront\ua0requirements\ua0engineering preceding\ua0development\ua0with precise specifications post-development. Out of these 12 aspects, seven can likely be addressed as part of an ongoing agile transformation. We offer an empirical account of expectations and needs for new\ua0requirements\ua0engineering approaches\ua0in\ua0the\ua0automotive\ua0domain, necessary to coordinate hundreds of collaborating organizations developing software-intensive and potentially safety-critical\ua0systems

Effective Communication in Globally Distributed Scrum: A Model and Practical Guidance

A trend in information systems development is for globally distributed teams to use agile methods and frameworks such as Scrum. In globally distributed (GD) software development, a known challenge is effective team communication. Researchers, however, cannot evaluate effective communication in GD teams using Scrum unless they know what effective communication means in that context. This qualitative study contributes a theoretical model of effective communication in GD Scrum teams and practical guidance for practitioners. Ten industry professionals working in GD Scrum teams were interviewed to capture their understanding of effective communication. Qualitative content analysis was used to analyse the interviews and form a basis for the model and the practical guidance. This novel model consists of communication transparency, communication quality, and communication discipline, which together lead to the alignment of team understanding (i.e., a team-level shared mental model). This theoretical model lays the ground for future research into the effect of Scrum practices on communication in GD contexts, and the effect of communication on team and project success. For practitioners, this study contributes 11 practical actions that professionals recommend for improving and sustaining effective communication

Improving Requirements-Test Alignment by Prescribing Practices that Mitigate Communication Gaps

The communication of requirements within software development is vital for project success. Requirements engineering and testing are two processes that when aligned can enable the discovery of issues and misunderstandings earlier, rather than later, and avoid costly and time-consuming rework and delays. There are a number of practices that support requirements-test alignment. However, each organisation and project is different and there is no one-fits-all set of practices. The software process improvement method called Gap Finder is designed to increase requirements-test alignment. The method contains two parts: an assessment part and a prescriptive part. It detects potential communication gaps between people and between artefacts (the assessment part), and identifies practices for mitigating these gaps (the prescriptive part). This paper presents the design and formative evaluation of the prescriptive part; an evaluation of the assessment part was published previously. The Gap Finder method was constructed using a design science research approach and is built on the Theory of Distances for Software Engineering, which in turn is grounded in empirical evidence from five case companies. The formative evaluation was performed through a case study in which Gap Finder was applied to an on-going development project. A qualitative and mixed-method approach was taken in the evaluation, including ethnographically-informed observations. The results show that Gap Finder can detect relevant communication gaps and seven of the nine prescribed practices were deemed practically relevant for mitigating these gaps. The project team found the method to be useful and supported joint reflection and improvement of their requirements communication. Our findings demonstrate that an empirically-based theory can be used to improve software development practices and provide a foundation for further research on factors that affect requirements communicatio

Effective Communication in Globally Distributed Scrum: A Model and Practical Guidance

A trend in information systems development is for globally distributed teams to use agile methods and frameworks such as Scrum. In globally distributed (GD) software development, a known challenge is effective team communication. Researchers, however, cannot evaluate effective communication in GD teams using Scrum unless they know what effective communication means in that context. This qualitative study contributes a theoretical model of effective communication in GD Scrum teams and practical guidance for practitioners. Ten industry professionals working in GD Scrum teams were interviewed to capture their understanding of effective communication. Qualitative content analysis was used to analyse the interviews and form a basis for the model and the practical guidance. This novel model consists of communication transparency, communication quality, and communication discipline, which together lead to the alignment of team understanding (i.e., a team-level shared mental model). This theoretical model lays the ground for future research into the effect of Scrum practices on communication in GD contexts, and the effect of communication on team and project success. For practitioners, this study contributes 11 practical actions that professionals recommend for improving and sustaining effective communication

Development of a Concept for Assessing Shared Understanding by Means of Vision Videos in Requirements Engineering Based on a Systematic Literature Review

Ein essentieller Aspekt für ein erfolgreiches Softwareprojekt ist das Erzeugen eines gemeinsamen Verständnisses zwischen den Stakeholdern und den Entwicklern. Dieses gemeinsame Verständnis ist eine Voraussetzung für die erfolgreiche Kommunikation der Anforderungen, die an das zu entwickelnde System gestellt werden. In der Literatur existieren hierfür bereits diverse Ansätze die beschreiben, mit welchen Methoden ein gemeinsames Verständnis erzeugt werden kann. Einige dieser Ansätze sehen dabei die Nutzung von Videos vor, da durch die visuelle Darstellung der Anforderungen an das System mögliche Sprachbarrieren umgangen werden, die sonst bei einer traditionellen textuellen Kommunikation zu Problemen führen können. Die Darstellung von visionären Nutzungsszenarien eines Systems in sogenannten Vision Videos stellt dabei eine mögliche Umsetzung dieses Ansatzes dar. In diesem Zusammenhang existieren zwar einige Publikationen die zeigen, dass durch Vision Videos ein gemeinsames Verständnis erzeugt werden kann, jedoch gibt es noch keinen Ansatz, in dem eine allgemeingültige Methode zur Beurteilung oder Messung von gemeinsamen Verständnis definiert wird. Aus diesem Grund wird innerhalb dieser Arbeit nach Methoden, Metriken und Heuristiken gesucht, mit denen gemeinsames Verständnis gemessen beziehungsweise beurteilt werden kann, das auf Basis von Vision Videos erzeugt wurde. Zu diesem Zweck wird eine systematische Literatursuche durchgeführt, um geeignete Methoden, Metriken und Heuristiken in der Literatur zu identifizieren. Die identifizierten Methoden, Metriken und Heuristiken werden anschließend in den Kontext von Vision Videos übertragen, um sie für diese Arbeit nutzbar zu machen. Zuletzt liefert diese Arbeit ein Experimentdesign, in dem konkrete Ansatzpunkte für die Nutzung der Methoden, Metriken und Heuristiken zur Messung und Beurteilung von gemeinsamen Verständnis beschrieben werden.An essential aspect for a successful software project is the creation of a shared understanding between stakeholders and developers. This shared understanding is a prerequisite for the successful communication of the requirements that are posed to the system to be developed. There are many approaches in literature that describe methods with which it is possible to create a shared understanding. Some of those approaches intend to use videos, since their visual representation of the requirements of the system can bypass language barriers that would pose problems for traditional textual communication. The representation of visionary scenarios of use of a system in so called vision videos presents a possible implementation of this approach. While there are some publications in this context that show that vision videos can create shared understanding, there are no approaches that define methods for the assessment or measurement of shared understanding. For that reason, within this work methods, metrics and heuristics are sought which can assess or measure shared understanding that was created via vision videos. For this purpose, a systematic literature review is conducted to identify fitting methods, metrics and heuristics. Furthermore, these methods, metrics and heuristics are transferred into the context of vision videos to make them usable for this work. Finally, this work presents an experiment design which describes concrete approaches for the use of the methods, metrics and heuristics to measure and assess shared understanding

A influência de fatores na produtividade do desenvolvimento de software de acordo com um modelo de estruturas teóricas

This work presents an evidence-based model describing the effects of a set of factors on software development productivity, obtained through an evidence synthesis method in Software Engineering. Thus, the relationships among this set and the software development productivity (observed phenomena) are described as results of combining theoretical structures capable of expressing and dealing with differences between different effects and uncertainties varying according to the types of studies found in the literature. Besides, to evaluate the model found, its findings are confronted with a survey capturing the practitioners’ perception (managers and leaders of software projects in Brazilian organizations). The degree of agreement between research (the model) and practice (the practitioners’ perception) shows that scientific knowledge does not differ considerably from the reality experienced by software projects when both of them refer to the influence of factors on software development productivity. The impression that research and practice on the theme go through different paths persists. According to this work, the reasons for this impression are more related to the use of non-standardized and, perhaps, inappropriate measures used to perceive and monitor the influence of factors as well as to measure the software development productivityEste trabalho apresenta um modelo baseado em evidências que descreve efeitos de alguns fatores na produtividade do desenvolvimento de software, obtidos através de um método de síntese de evidências em Engenharia de Software. Deste modo, as relações entre um conjunto de fatores e a produtividade do desenvolvimento de software (fenômenos observados) são descritas como resultados da combinação de estruturas teóricas capazes de expressar e tratar diferenças entre efeitos e incertezas variadas de acordo com os tipos de estudos primários encontrados na literatura. Além disso, para avaliar o modelo encontrado, seus achados são confrontados com uma pesquisa de opinião realizada para capturar a percepção de profissionais da prática (gestores e líderes de projetos de software em organizações brasileiras). O grau de concordância entre a pesquisa (o modelo) e a prática (a percepção dos profissionais) demonstra que, aparentemente, o conhecimento científico não diverge consideravelmente da realidade vivenciada pelos projetos de software no Brasil, quando ambos se referem à influência de fatores na produtividade do desenvolvimento de software. Persiste a impressão, entretanto, de que a pesquisa e a prática no tema percorrem caminhos distintos. De acordo com este trabalho, a impressão do distanciamento parece estar relacionadas à questão do uso de medidas não-padronizadas e, talvez, inapropriadas para mensurar os fatores e a produtividade do desenvolvimento de softwar