On the effects of programming and testing skills on external quality and productivity in a test-driven development context

Background: In previous studies, a model was proposed that investigated how the developers’ unit testing effort impacted their productivity as well as the external quality of the software they developed. Goal: The aim of this study is to enhance the proposed model by considering two additional factors related to the expertise of developers: programming and unit testing skills. The possibility of including such skills in a model that represents the relationship that testing effort has with the developer’s productivity and the product’s external quality was investigated. Method: Data collected from a test-first development task in academic setting was used in order to gauge the relationship between testing effort, external quality, and productivity. Furthermore, Analysis of Covariance (ANCOVA) was utilized to check the impact of developers’ skills on productivity and quality. Result: The results obtained in previous studies were confirmed: there exists a positive effect of testing effort on productivity, but not on quality. Moreover, the developers’ skills have an impact on productivity but none on external quality. Conclusion: Productivity improves with testing effort, a result consistent across previous, similar studies. The role of existing skills is a relevant factor in studying the effects of developers’ unit testing effort on productivity. Nevertheless, more investigations are needed regarding the relationship between unit testing effort and external quality.This research is partially supported by the Academy of Finland with decision no: 278354. The rst author would like to acknowledge ISACA Finland Chapter for the support provided to complete this work

Teaching Software Engineering with Free Open Source Software Development: An Experience Report

We report on the design and delivery of a senior Software Engineering course within the limits of a Computer Science program. The course is structured around a collaboration with a large, active Free Open Source Software project. We show how this structure allows us to (a) incorporate principles of Project Based Learning and of Service Learning, reaping the benefits of these pedagogies, (b) effectively, using a hands-on approach, teach a number of essential topics in Software Engineering, (c) provide the students with a capstone project experience, given the lack of one in our curriculum, and (d) use the project as a powerful motivating factor for the students. We outline the experiences of the course instructor, of the teaching assistants team, and of the students of the course. We also describe the experience of the lead developers of this open source project, and report on the benefits and costs (time commitment) to the project

Towards an operationalization of test-driven development skills: An industrial empirical study

Abstract Context: The majority of the empirical studies on Test-driven development (TDD) are concerned with verifying or refuting the effectiveness of the technique over a traditional approach, and they tend to neglect whether the subjects possess the necessary skills to apply TDD, though they argue such skills are necessary. Objective: We evaluate a set of minimal, a priori and in process skills necessary to apply TDD. We determine whether variations in external quality (i.e., number of defects) and productivity (i.e., number of features implemented) can be associated with different clusters of the TDD skills? set. Method: We executed a quasi-experiment involving 30 practitioners from industry. We first grouped the participants according to their TDD skills? set (consisting of a priori experience on programming and testing as well as in-process TDD conformance) into three levels (Low-Medium-High) using k-means clustering. We then applied ANOVA to compare the clusters in terms of external quality and productivity, and conducted post-hoc pairwise analysis. Results: We did not observe a statistically significant difference between the clusters either for external software quality ( F ( 2 , 27 = 1.44 , p = . 260 ), or productivity ( F ( 2 , 27 ) = 3.02 , p = . 065 ). However, the analysis of the effect sizes and their confidence intervals shows that the TDD skills? set is a factor that could account for up to 28% of the external quality, and 38% for productivity. Conclusion: We have reason to conclude that focusing on the improvement of TDD skills? set investigated in this study could benefit software developers in improving their baseline productivity and the external quality of the code they produce. However, replications are needed to overcome the issues related with the statistical power of this study. We suggest practical insights for future work to investigate the phenomenon further.hisresearchispartiallysupportedbytheAcademyofFinlandwithdecisionno.:278354,andbyFinnishDistinguishedProfessor(Fi.Di.Pro.) programme, ESEIL

DevOps in practice : A multiple case study of five companies

Context: DevOps is considered important in the ability to frequently and reliably update a system in operational state. DevOps presumes cross-functional collaboration and automation between software development and operations. DevOps adoption and implementation in companies is non-trivial due to required changes in technical, organisational and cultural aspects. Objectives: This exploratory study presents detailed descriptions of how DevOps is implemented in practice. The context of our empirical investigation is web application and service development in small and medium sized companies. Method: A multiple-case study was conducted in five different development contexts with successful DevOps implementations since its benefits, such as quick releases and minimum deployment errors, were achieved. Data was mainly collected through interviews with 26 practitioners and observations made at the companies. Data was analysed by first coding each case individually using a set of predefined themes and thereafter perform a cross-case synthesis. Results: Our analysis yielded some of the following results: (I) software development team attaining ownership and responsibility to deploy software changes in production is crucial in DevOps. (ii) toolchain usage and support in deployment pipeline activities accelerates the delivery of software changes, bug fixes and handling of production incidents. (ii) the delivery speed to production is affected by context factors, such as manual approvals by the product owner (iii) steep learning curve for new skills is experienced by both software developers and operations staff, who also have to cope with working under pressure. Conclusion: Our findings contributes to the overall understanding of DevOps concept, practices and its perceived impacts, particularly in small and medium sized companies. We discuss two practical implications of the results.Peer reviewe

Missing steps in a staircase: a qualitative study of the perspectives of key stakeholders on the use of adaptive designs in confirmatory trials

Background Despite the promising benefits of adaptive designs (ADs), their routine use, especially in confirmatory trials, is lagging behind the prominence given to them in the statistical literature. Much of the previous research to understand barriers and potential facilitators to the use of ADs has been driven from a pharmaceutical drug development perspective, with little focus on trials in the public sector. In this paper, we explore key stakeholders’ experiences, perceptions and views on barriers and facilitators to the use of ADs in publicly funded confirmatory trials. Methods Semi-structured, in-depth interviews of key stakeholders in clinical trials research (CTU directors, funding board and panel members, statisticians, regulators, chief investigators, data monitoring committee members and health economists) were conducted through telephone or face-to-face sessions, predominantly in the UK. We purposively selected participants sequentially to optimise maximum variation in views and experiences. We employed the framework approach to analyse the qualitative data. Results We interviewed 27 participants. We found some of the perceived barriers to be: lack of knowledge and experience coupled with paucity of case studies, lack of applied training, degree of reluctance to use ADs, lack of bridge funding and time to support design work, lack of statistical expertise, some anxiety about the impact of early trial stopping on researchers’ employment contracts, lack of understanding of acceptable scope of ADs and when ADs are appropriate, and statistical and practical complexities. Reluctance to use ADs seemed to be influenced by: therapeutic area, unfamiliarity, concerns about their robustness in decision-making and acceptability of findings to change practice, perceived complexities and proposed type of AD, among others. Conclusions There are still considerable multifaceted, individual and organisational obstacles to be addressed to improve uptake, and successful implementation of ADs when appropriate. Nevertheless, inferred positive change in attitudes and receptiveness towards the appropriate use of ADs by public funders are supportive and are a stepping stone for the future utilisation of ADs by researchers

Automated recommendation, reuse, and generation of unit tests for software systems

This thesis presents a body of work relating to the automated discovery, reuse, and generation of unit tests for software systems with the goal of improving the efficiency of the software engineering process and the quality of the produced software. We start with a novel approach to test-to-code traceability link establishment, called TCTracer, which utilises multilevel information and an ensemble of static and dynamic techniques to achieve state-of-the-art accuracy when establishing links between tests and tested functions and test classes and tested classes. This approach is utilised to provide test-to-code traceability links which facilitate multiple other parts of the work. We then move on to test reuse where we first define an abstract framework, called Rashid, for using connections between artefacts to identify new artefacts for reuse and utilise this framework in Relatest, an approach for producing test recommendations for new functions. Relatest instantiates Rashid by using TCTracer to establish connections between tests and functions and code similarity measures to establish connections between similar functions. This information is used to create lists of recommendations for new functions. We then present an investigation into the automated transplantation of tests which attempts to remove the manual effort required to transform Relatest recommendations and insert them into another project. Finally, we move on to test generation where we utilise neural networks to generate unit test code by learning from existing function-to-test pairs. The first approach, TestNMT, investigates using recurrent neural networks to generate whole JUnit tests and the second approach, ReAssert, utilises a transformer-based architecture to generate JUnit asserts. In total, this thesis addresses the problem by developing approaches for the discovery, reuse, and utilisation of existing functions and tests, including the establishment of relationships between these artefacts, developing mechanisms to aid automated test reuse and learning from existing tests to generate new tests

How Scrum Adds Value to Achieving Software Quality?

Scrum remains the most popular agile software development method implementation for a variety of reasons; one important motive is to improve software quality. Yet many organizations fail to achieve quality improvements through the use of Scrum, and existing research sheds little light on the value-add of Scrum for software quality. More specifically, (1) how notions of software quality among Scrum practitioners relate to established quality perspectives, (2) how Scrum helps teams to achieve higher software quality and (3) why some teams fail to meet the objective of higher quality. We addressed these gaps through a two-phased qualitative study based on 39 interviews and two in-depth case studies. We find that Scrum practitioners emphasize established notions of external quality comprising of conformity to business needs and absence of defects, while they also value internal quality, especially sustainable software design. Our results show that Scrum helps teams achieve both dimensions of quality by promoting some social antecedents (collaboration, psychological safety, accountability, transparency) and process-induced advantages (iterative development, formal inspection, and adaptation). Our findings unveil how these factors contribute to achieving software quality and under what conditions their effects can fail to materialize. These conditions include inconsistent Scrum implementations, cultural constraints, team tensions, and inaccessibility of end-users. In addition, the complexity of the project aggravates the impact of these conditions. Taken together, these findings show that Scrum can complement established quality assurance and software engineering practices by promoting a social environment that is conducive to creating high-quality software. Based on our findings, we provide specific recommendations for how practitioners can create such an environment