208,823 research outputs found
Measuring Time - A Systematic Survey of Controlled Experiments in Software Engineering
To conduct controlled experiments is the classical method for identifying cause-effect relationships. This helps researchers evaluate and validate their research results. This master thesis quantitatively systematically surveys the present practices of reporting and using time in 113 controlled software engineering experiments published in 103 scientific articles in the period 1993-2002. The survey can be regarded as an extension of the study of Sjøberg et al. [6], who investigated several other aspects of the same collection of controlled experiments.
The need for an analysis of time in controlled SE-experiments was recognized based on the results of an earlier survey conducted by Jo E. Hannay at SIMULA Research Laboratory. He identified that time often is reported as dependent variable. Due to the dependent variable’s important role in experiments as provider of the effect construct in the causal relationship, it was relevant to find out how time is used.
The investigation of time in controlled SE-experiments focuses on the following aspects; the overall frequency of reporting time, the use of time, the recording of time, the specification of time in terms of time units, and the validity threats that time can constitute. These aspects were regarded as the most important features of time, and hence, would give a thorough and appropriate picture of time in controlled SE-experiments.
The main results of the survey of the aspects of time, is first of all, that time is reported in a substantial majority of controlled SE-experiments. Second, in most of the controlled SE-experiments time is used as dependent variable, or more precise, as measure of subjects’ experimental tasks. However, in an extensive number of controlled SE-experiments time is not explicitly described as dependent variable, although it is used in that sense. Third, time measures are found to be recorded by subjects, experimenters and tools. An interesting finding in relation to this aspect is that the largest proportion of controlled SE-experiments did not explicitly report their time recording. Fourth, a majority of controlled SE-experiments measure the time in minutes. Last, but not least, very few controlled SE-experiments were found to address time in relation to validity threats.
The main conclusions and recommendations of the research, is first of all, that SE-researchers documenting controlled experiments should explicitly describe time as dependent variable in the experimental design, when time is used as dependent variable. This is obviously important due to the vital role of a dependent variable in experimentation. Second, SE-researchers should consistently describe how time is recorded in controlled experiments, due to validations of the experiment results and to possible replications. Third, SE-researchers should specify time measures in minutes, because it provides an appropriate granularity of the time data. And fourth, aspects of time constitute threats to both internal and external validity, and SE-researchers should therefore assess time in relation to possible validity threats of the controlled experiments.
The hope of this master thesis is that the results, discussions, conclusions and recommendations of the research provide useful insight to SE-researchers, so that they can improve the design of time in their controlled experiments. This is vital in order to advance a state-of-the art practice in empirical software engineering research
Empirical software engineering experts on the use of students and professionals in experiments
[Context] Controlled experiments are an important empirical method to generate and validate theories. Many software engineering experiments are conducted with students. It is often claimed that the use of students as participants in experiments comes at the cost of low external validity while using professionals does not. [Objective] We believe a deeper understanding is needed on the external validity of software engineering experiments conducted with students or with professionals. We aim to gain insight about the pros and cons of using students and professionals in experiments. [Method] We performed an unconventional, focus group approach and a follow-up survey. First, during a session at ISERN 2014, 65 empirical researchers, including the seven authors, argued and discussed the use of students in experiments with an open mind. Afterwards, we revisited the topic and elicited experts' opinions to foster discussions. Then we derived 14 statements and asked the ISERN attendees excluding the authors, to provide their level of agreement with the statements. Finally, we analyzed the researchers' opinions and used the findings to further discuss the statements. [Results] Our survey results showed that, in general, the respondents disagreed with us about the drawbacks of professionals. We, on the contrary, strongly believe that no population (students, professionals, or others) can be deemed better than another in absolute terms. [Conclusion] Using students as participants remains a valid simplification of reality needed in laboratory contexts. It is an effective way to advance software engineering theories and technologies but, like any other aspect of study settings, should be carefully considered during the design, execution, interpretation, and reporting of an experiment. The key is to understand which developer population portion is being represented by the participants in an experiment. Thus, a proposal for describing experimental participants is put forward.Peer reviewe
Project Quality of Offshore Virtual Teams Engaged in Software Requirements Analysis: An Exploratory Comparative Study
The off-shore software development companies in countries such as India use a global delivery model in which initial requirement analysis phase of software projects get executed at client locations to leverage frequent and deep interaction between user and developer teams. Subsequent phases such as design, coding and testing are completed at off-shore locations. Emerging trends indicate an increasing interest in off-shoring even requirements analysis phase using computer mediated communication. We conducted an exploratory research study involving students from Management Development Institute (MDI), India and Marquette University (MU), USA to determine quality of such off-shored requirements analysis projects. Our findings suggest that project quality of teams engaged in pure off-shore mode is comparable to that of teams engaged in collocated mode. However, the effect of controls such as user project monitoring on the quality of off-shored projects needs to be studied further
An Approach for the Empirical Validation of Software Complexity Measures
Software metrics are widely accepted tools to control and assure software quality. A large number of software metrics with a variety of content can be found in the literature; however most of them are not adopted in industry as they are seen as irrelevant to needs, as they are unsupported, and the major reason behind this is due to improper
empirical validation. This paper tries to identify possible root causes for the improper empirical validation of the software metrics. A practical model for the empirical validation of software metrics is proposed along with root causes. The model is validated by applying it to recently proposed and well known metrics
Remote laboratories in teaching and learning – issues impinging on widespread adoption in science and engineering education
This paper discusses the major issues that impinge on the widespread adoption of remote controlled laboratories in science and engineering education. This discussion largely emerges from the work of the PEARL project and is illustrated with examples and evaluation data from the project. Firstly the rationale for wanting to offer students remote experiments is outlined. The paper deliberately avoids discussion of technical implementation issues of remote experiments but instead focuses on issues that impinge on the specification and design of such facilities. This includes pedagogic, usability and accessibility issues. It compares remote experiments to software simulations. It also considers remote experiments in the wider context for educational institutions and outlines issues that will affect their decisions as to whether to adopt this approach. In conclusion it argues that there are significant challenges to be met if remote laboratories are to achieve a widespread presence in education but expresses the hope that this delineation of the issues is a contribution towards meeting these challenges
- …