A systematic review of software development cost estimation studies

This paper aims to provide a basis for the improvement of software estimation research through a systematic review of previous work. The review identifies 304 software cost estimation papers in 76 journals and classifies the papers according to research topic, estimation approach, research approach, study context and data set. A web-based library of these cost estimation papers is provided to ease the identification of relevant estimation research results. The review results combined with other knowledge provide support for recommendations for future software cost estimation research, including: 1) Increase the breadth of the search for relevant studies, 2) Search manually for relevant papers within a carefully selected set of journals when completeness is essential, 3) Conduct more studies on estimation methods commonly used by the software industry, and, 4) Increase the awareness of how properties of the data sets impact the results when evaluating estimation methods

Reflective Designs — An Overview

AbstractWe render runtime system adaptations by design-level concepts such that running systems can be adapted and examined at a higher level of abstraction. The overall idea is to express design decisions as applications of design operators to be carried out at runtime. Design operators can implement design patterns for use at runtime. Applications of design operators are made explicit as design elements in the running system such that they can be traced, reconfigured, and made undone.Our approach enables Reflective Designs: on one side, design operators employ reflection to perform runtime adaptations; on the other side, design elements provide an additional reflection protocol to examine and configure performed adaptations. Our approach helps understanding the development and the maintenance of the class of software systems that cannot tolerate downtime or frequent shutdown-revise-startup cycles.We have accumulated a class library for programming with Reflective Designs in Squeak/Smalltalk. This library employs reflection and dynamic aspect-oriented programming. We have also implemented tool support for navigating in a system that is adapted continuously at runtime.Note: This extended abstract summarises our full paper [Hirschfeld, R. and R. Lämmel, Reflective Designs, IEE Proceedings Software (2004), Special Issue on Reusable Software Libraries. To appear. Available at http://homepages.cwi.nl/~ralf/rd/]

Human vs. Algorithm

We consider the roles of algorithm and human and their inter-relationships. As a vehicle for some of our ideas we describe an empirical investigation of software professionals using analogy-based tools and unaided search in order to solve various prediction problems. We conclude that there exist a class of software engineering problems which might be characterised as high value and low frequency where the human-algorithm interaction must be considered carefully if they are to be successfully deployed in industry

Making Software Cost Data Available for Meta-Analysis

In this paper we consider the increasing need for meta-analysis within empirical software engineering. However, we also note that a necessary precondition to such forms of analysis is to have both the results in an appropriate format and sufficient contextual information to avoid misleading inferences. We consider the implications in the field of software project effort estimation and show that for a sample of 12 seemingly similar published studies, the results are difficult to compare let alone combine. This is due to different reporting conventions. We argue that a protocol is required and make some suggestions as to what it should contain

Software project economics: A roadmap

The objective of this paper is to consider research progress in the field of software project economics with a view to identifying important challenges and promising research directions. I argue that this is an important sub-discipline since this will underpin any cost-benefit analysis used to justify the resourcing, or otherwise, of a software project. To accomplish this I conducted a bibliometric analysis of peer reviewed research articles to identify major areas of activity. My results indicate that the primary goal of more accurate cost prediction systems remains largely unachieved. However, there are a number of new and promising avenues of research including: how we can combine results from primary studies, integration of multiple predictions and applying greater emphasis upon the human aspects of prediction tasks. I conclude that the field is likely to remain very challenging due to the people-centric nature of software engineering, since it is in essence a design task. Nevertheless the need for good economic models will grow rather than diminish as software becomes increasingly ubiquitous

An Analysis of Data Sets Used to Train and Validate Cost Prediction Systems

OBJECTIVE - the aim of this investigation is to build up a picture of the nature and type of data sets being used to develop and evaluate different software project effort prediction systems. We believe this to be important since there is a growing body of published work that seeks to assess different prediction approaches. Unfortunately, results – to date – are rather inconsistent so we are interested in the extent to which this might be explained by different data sets. METHOD - we performed an exhaustive search from 1980 onwards from three software engineering journals for research papers that used project data sets to compare cost prediction systems. RESULTS - this identified a total of 50 papers that used, one or more times, a total of 74 unique project data sets. We observed that some of the better known and publicly accessible data sets were used repeatedly making them potentially disproportionately influential. Such data sets also tend to be amongst the oldest with potential problems of obsolescence. We also note that only about 70% of all data sets are in the public domain and this can be particularly problematic when the data set description is incomplete or limited. Finally, extracting relevant information from research papers has been time consuming due to different styles of presentation and levels of contextural information. CONCLUSIONS - we believe there are two lessons to learn. First, the community needs to consider the quality and appropriateness of the data set being utilised; not all data sets are equal. Second, we need to assess the way results are presented in order to facilitate meta-analysis and whether a standard protocol would be appropriate

Usability and open source software.

Open source communities have successfully developed many pieces of software although most computer users only use proprietary applications. The usability of open source software is often regarded as one reason for this limited distribution. In this paper we review the existing evidence of the usability of open source software and discuss how the characteristics of open-source development influence usability. We describe how existing human-computer interaction techniques can be used to leverage distributed networked communities, of developers and users, to address issues of usability

New ideas and emerging research: evaluating prediction system accuracy

BACKGROUND: Prediction e.g. of project cost is an important concern in software engineering. PROBLEM: Although many empirical validations of software engineering prediction systems have been published, no one approach dominates and sense-making of conflicting empirical results is proving challenging. METHOD: We propose a new approach to evaluating competing prediction systems based upon an unbiased statistic (Standardised Accuracy), analysis of results relative to the baseline technique of guessing and calculation of effect sizes. RESULTS: Two empirical studies are revisited and the published results are shown to be misleading when re-analysed using our new approach. CONCLUSION: Biased statistics such as MMRE are deprecated. By contrast our approach leads to valid results. Such steps will greatly assist in performing future meta-analyses

Effect of user experience on technology acceptance: the case of foss

Free and open source software (FOSS) movement essentially arises like answer to the evolution occurred in the market from the software, characterized by the closing of the source code. Furthermore, some FOSS characteristics, such as (1) the advance of this movement and (2) the attractiveness that contributes the voluntary and cooperative work, have increased the interest of the users towards free software. Traditionally, research in FOSS has focused on identifying individual personal motives for participating in the development of a FOSS project, analyzing specific FOSS solutions, or the FOSS movement itself. Nevertheless, the advantages of the FOSS for users and the effect of the demographic dimensions on user acceptance for FOSS have been two research topics with little attention. Specifically, this paper’s aim is to focus on the influence of the user experience with FOSS the FOSS acceptance. Based on the literature, user experience is an essential demographic dimension for explaining the Information Systems acceptance. With this purpose, the authors have developed a research model based on the Technological Acceptance Model (TAM).El movimiento asociado al software de código abierto (FOSS) surge como una respuesta a la evolución acontecida en el mercado del software, caracterizado por el cierre del código fuente. Además, algunas características del FOSS como (1) el avance de este movimiento y (2) el atractivo que suscita debido a que se construye bajo la filosofía de trabajo voluntario y cooperativo, ha incrementado el interés de los usuarios hacia FOSS. Tradicionalmente las investigaciones en FOSS han estado centradas en identificar las motivaciones personales de participar en el desarrollo de un proyecto FOSS. Mientras que las ventajas del FOSS para los usuarios y el efecto de las dimensiones demográficas en la aceptación de FOSS han sido dos tópicos de investigación con poca atención. Concretamente, este artículo se centra en analizar la influencia de la experiencia con FOSS en la propia aceptación de FOSS. Basado en la literatura, la experiencia del usuario es una dimensión demográfica esencial para explicar la aceptación de los Sistemas de Información. Con este propósito, los autores han desarrollado un modelo de investigación basado en el Metamodelo de Aceptación de la Tecnología (TAM)