Qualitative software engineering research -- reflections and guidelines

Researchers are increasingly recognizing the importance of human aspects in software development and since qualitative methods are used to, in-depth, explore human behavior, we believe that studies using such techniques will become more common. Existing qualitative software engineering guidelines do not cover the full breadth of qualitative methods and knowledge on using them found in the social sciences. The aim of this study was thus to extend the software engineering research community's current body of knowledge regarding available qualitative methods and provide recommendations and guidelines for their use. With the support of an epistemological argument and a literature review, we suggest that future research would benefit from (1) utilizing a broader set of research methods, (2) more strongly emphasizing reflexivity, and (3) employing qualitative guidelines and quality criteria. We present an overview of three qualitative methods commonly used in social sciences but rarely seen in software engineering research, namely interpretative phenomenological analysis, narrative analysis, and discourse analysis. Furthermore, we discuss the meaning of reflexivity in relation to the software engineering context and suggest means of fostering it. Our paper will help software engineering researchers better select and then guide the application of a broader set of qualitative research methods.Comment: 30 page

Consequences of Unhappiness While Developing Software

The growing literature on affect among software developers mostly reports on the linkage between happiness, software quality, and developer productivity. Understanding the positive side of happiness -- positive emotions and moods -- is an attractive and important endeavor. Scholars in industrial and organizational psychology have suggested that also studying the negative side -- unhappiness -- could lead to cost-effective ways of enhancing working conditions, job performance, and to limiting the occurrence of psychological disorders. Our comprehension of the consequences of (un)happiness among developers is still too shallow, and is mainly expressed in terms of development productivity and software quality. In this paper, we attempt to uncover the experienced consequences of unhappiness among software developers. Using qualitative data analysis of the responses given by 181 questionnaire participants, we identified 49 consequences of unhappiness while doing software development. We found detrimental consequences on developers' mental well-being, the software development process, and the produced artifacts. Our classification scheme, available as open data, will spawn new happiness research opportunities of cause-effect type, and it can act as a guideline for practitioners for identifying damaging effects of unhappiness and for fostering happiness on the job.Comment: 6 pages. To be presented at the Second International Workshop on Emotion Awareness in Software Engineering, colocated with the 39th International Conference on Software Engineering (ICSE'17). Extended version of arXiv:1701.02952v2 [cs.SE

How Do You Feel, Developer? An Explanatory Theory of the Impact of Affects on Programming Performance

Affects---emotions and moods---have an impact on cognitive activities and the working performance of individuals. Development tasks are undertaken through cognitive processes, yet software engineering research lacks theory on affects and their impact on software development activities. In this paper, we report on an interpretive study aimed at broadening our understanding of the psychology of programming in terms of the experience of affects while programming, and the impact of affects on programming performance. We conducted a qualitative interpretive study based on: face-to-face open-ended interviews, in-field observations, and e-mail exchanges. This enabled us to construct a novel explanatory theory of the impact of affects on development performance. The theory is explicated using an established taxonomy framework. The proposed theory builds upon the concepts of events, affects, attractors, focus, goals, and performance. Theoretical and practical implications are given.Comment: 24 pages, 2 figures. Postprin

The Effects of the Quantification of Faculty Productivity: Perspectives from the Design Science Research Community

In recent years, efforts to assess faculty research productivity have focused more on the measurable quantification of academic outcomes. For benchmarking academic performance, researchers have developed different ranking and rating lists that define so-called high-quality research. While many scholars in IS consider lists such as the Senior Scholar’s basket (SSB) to provide good guidance, others who belong to less-mainstream groups in the IS discipline could perceive these lists as constraining. Thus, we analyzed the perceived impact of the SSB on information systems (IS) academics working in design science research (DSR) and, in particular, how it has affected their research behavior. We found the DSR community felt a strong normative influence from the SSB. We conducted a content analysis of the SSB and found evidence that some of its journals have come to accept DSR more. We note the emergence of papers in the SSB that outline the role of theory in DSR and describe DSR methodologies, which indicates that the DSR community has rallied to describe what to expect from a DSR manuscript to the broader IS community and to guide the DSR community on how to organize papers for publication in the SSB

Evolution of statistical analysis in empirical software engineering research: Current state and steps forward

Software engineering research is evolving and papers are increasingly based on empirical data from a multitude of sources, using statistical tests to determine if and to what degree empirical evidence supports their hypotheses. To investigate the practices and trends of statistical analysis in empirical software engineering (ESE), this paper presents a review of a large pool of papers from top-ranked software engineering journals. First, we manually reviewed 161 papers and in the second phase of our method, we conducted a more extensive semi-automatic classification of papers spanning the years 2001--2015 and 5,196 papers. Results from both review steps was used to: i) identify and analyze the predominant practices in ESE (e.g., using t-test or ANOVA), as well as relevant trends in usage of specific statistical methods (e.g., nonparametric tests and effect size measures) and, ii) develop a conceptual model for a statistical analysis workflow with suggestions on how to apply different statistical methods as well as guidelines to avoid pitfalls. Lastly, we confirm existing claims that current ESE practices lack a standard to report practical significance of results. We illustrate how practical significance can be discussed in terms of both the statistical analysis and in the practitioner's context.Comment: journal submission, 34 pages, 8 figure