How reliable are systematic reviews in empirical software engineering?

BACKGROUND – the systematic review is becoming a more commonly employed research instrument in empirical software engineering. Before undue reliance is placed on the outcomes of such reviews it would seem useful to consider the robustness of the approach in this particular research context. OBJECTIVE – the aim of this study is to assess the reliability of systematic reviews as a research instrument. In particular we wish to investigate the consistency of process and the stability of outcomes. METHOD – we compare the results of two independent reviews under taken with a common research question. RESULTS – the two reviews find similar answers to the research question, although the means of arriving at those answers vary. CONCLUSIONS – in addressing a well-bounded research question, groups of researchers with similar domain experience can arrive at the same review outcomes, even though they may do so in different ways. This provides evidence that, in this context at least, the systematic review is a robust research method

On Integrating Student Empirical Software Engineering Studies with Research and Teaching Goals

Background: Many empirical software engineering studies use students as subjects and are conducted as part of university courses. Aim: We aim at reporting our experiences with using guidelines for integrating empirical studies with our research and teaching goals. Method: We document our experience from conducting three studies with graduate students in two software architecture courses. Results: Our results show some problems that we faced when following the guidelines and deviations we made from the original guidelines. Conclusions: Based on our results we propose recommendations for empirical software engineering studies that are integrated in university courses.

Preface

This CEUR volume contains the research proposals accepted for presentation at the 13th International Doctoral Symposium on Empirical Software Engineering (IDoESE 2015), held in Beijing, China, the 21st of October 2015, as an event integrated in the Empirical Software Engineering International Week (ESEIW), which remarkably included the world-leading Empirical Software Engineering and Measurement conference (ESEM 2015). The objective of the doctoral symposium is to provide junior researchers with the opportunity to present their work to the empirical software engineering community and receive valuable feedback from experienced researchers in that community. The symposium also aims at facilitating the exchange of ideas among young researchers. To do so, experienced members of the empirical software engineering community serve as symposium advisors and provide feedback to students presenting their work

Happy software developers solve problems better: psychological measurements in empirical software engineering

For more than 30 years, it has been claimed that a way to improve software developers' productivity and software quality is to focus on people and to provide incentives to make developers satisfied and happy. This claim has rarely been verified in software engineering research, which faces an additional challenge in comparison to more traditional engineering fields: software development is an intellectual activity and is dominated by often-neglected human aspects. Among the skills required for software development, developers must possess high analytical problem-solving skills and creativity for the software construction process. According to psychology research, affects-emotions and moods-deeply influence the cognitive processing abilities and performance of workers, including creativity and analytical problem solving. Nonetheless, little research has investigated the correlation between the affective states, creativity, and analytical problem-solving performance of programmers. This article echoes the call to employ psychological measurements in software engineering research. We report a study with 42 participants to investigate the relationship between the affective states, creativity, and analytical problem-solving skills of software developers. The results offer support for the claim that happy developers are indeed better problem solvers in terms of their analytical abilities. The following contributions are made by this study: (1) providing a better understanding of the impact of affective states on the creativity and analytical problem-solving capacities of developers, (2) introducing and validating psychological measurements, theories, and concepts of affective states, creativity, and analytical-problem-solving skills in empirical software engineering, and (3) raising the need for studying the human factors of software engineering by employing a multidisciplinary viewpoint.Comment: 33 pages, 11 figures, published at Peer

Empirical Software Engineering in Industry Short Courses

This paper reports on a pilot project that incorporated small empirical studies in three industry short courses. These laboratory experiments were one component of a larger leveled study on the effects of test-driven development (TDD) on internal software quality. The approach is proposed to have pedagogical value to student-developers by improving their understanding and appreciation for empirical evidence, to instructors by providing feedback through surveys and exercises, and to the community at large by reporting results of the studies. Pre-experiment surveys in the three pilot experiments revealed large differences in programmer opinions of TDD. Possible correlations to development environment and programmer experience will be proposed. Post-experiment surveys revealed improvements in programmer opinions of TDD following the experiment exercises. Crafting sufficiently small but interesting assignments proved to be challenging. Few complete solutions were submitted and some developers were unwilling to submit their partial solutions. Positive observations will be made regarding the use of experiments in short courses. For instance, participating in the study encourages analytical thinking, prompts developers to evaluate alternative approaches, and instills the value of empirical evidence. Ethical concerns regarding threats to validity are raised and addressed. The authors find that ethical considerations not only support performing such studies, but encourage it as the duty of software professionals

Introducing empirical software engineering methods in education

Empirical methods are important in software engineering. It is important to be able to evaluate new techniques and methods in a structured way before they are introduced in the software process. This paper presents how empirical methods may be taught by letting students take part in the execution and analysis of empirical investigations in projects. The projects that the students carry out include a role-play where teachers play the roles of managers for the students and responsible for ordering the work of the students. It is found that the project is well received and that it is probable that the students reach the related learning objectives of the cours

DISTANCE: a framework for software measure construction.

In this paper we present a framework for software measurement that is specifically suited to satisfy the measurement needs of empirical software engineering research. The framework offers an approach to measurement that builds upon the easily imagined, detected and visualised concepts of similarity and dissimilarity between software entities. These concepts are used both to model the software attributes of interest and to define the corresponding software measures. Central to the framework is a process model that embeds constructive procedures for attribute modelling and measure construction into a goal-oriented approach to empirical software engineering studies. The underlying measurement theoretic principles of our approach ensure the construct validity of the resulting measures. The approach was tested on a popular suite of object-oriented design measures. We further show that our measure construction method compares favourably to related work.Software;