Empirical research on Software Effort Estimation Accuracy

To improve the software development process is named by both the European Union and the United States government as an important task for society. The constant problem with effort overruns and estimation inaccuracy is a main part of the software development problem. Empirical research on software effort estimation is a key part of the continuing effort by researchers and practitioners to improve the way in which software development projects are carried out. As part of this effort, a study on eighteen of the latest projects at a Norwegian software consultancy was done. The study was done by interviewing the project managers responsible for the projects, having them provide key project data, and their assessments of different project properties related to effort estimation. The study focused on answering research questions related to: • The effect the contractor-customer relationship and customer properties have on estimation accuracy • The effect utilizing experience data has on estimation accuracy • The role of estimation accuracy when assessing software project success The analysis of the collected empirical data showed that reduced effort overruns was associated with increased contact frequency with the customer and contracts that share the risk between contractor and customer. Utilization of experience data, and the use of checklists, was also found to have a positive impact on estimation accuracy. There was not found any strong correlation between project managers’ project success assessment and estimation accuracy, indicating that estimation accuracy and project manager success assessment contribute with two different, but important viewpoints when software project success is to be assessed. In addition to the empirical study and its results, the thesis presents a review of existing group combination techniques for software effort estimation. The review was motivated by recent studies that have suggested that to do software estimation as a group is beneficial. The review presents techniques that vary largely as to how they structure the interaction among the group members, and how their opinions are aggregated. A thorough discussion on the argumentation behind the techniques, and the consequences they have is given in the review. The empirical data collected during the work with this thesis suggests different ways in which software contractors could improve their estimation ability and reduce their effort overruns. The conclusions of this thesis is, that to increase estimation accuracy, software contractors should: (i) involve the customer, and nurture the customer relationship, (ii) add some repeatable structure to the estimation process, but be careful not to add too much structure, (iii) gather and utilize experience data in the estimation process and (iv) evaluate projects when they are done. In doing the evaluation both objective data on effort, schedule and functionality compliance and subjective assessments of project success from key stakeholders, as customer, user, project manager, developers and management should be gathered

An Empirical investigation into software effort estimation by analogy

Most practitioners recognise the important part accurate estimates of development effort play in the successful management of major software projects. However, it is widely recognised that current estimation techniques are often very inaccurate, while studies (Heemstra 1992; Lederer and Prasad 1993) have shown that effort estimation research is not being effectively transferred from the research domain into practical application. Traditionally, research has been almost exclusively focused on the advancement of algorithmic models (e.g. COCOMO (Boehm 1981) and SLIM (Putnam 1978)), where effort is commonly expressed as a function of system size. However, in recent years there has been a discernible movement away from algorithmic models with non-algorithmic systems (often encompassing machine learning facets) being actively researched. This is potentially a very exciting and important time in this field, with new approaches regularly being proposed. One such technique, estimation by analogy, is the focus of this thesis. The principle behind estimation by analogy is that past experience can often provide insights and solutions to present problems. Software projects are characterised in terms of collectable features (such as the number of screens or the size of the functional requirements) and stored in a historical case base as they are completed. Once a case base of sufficient size has been cultivated, new projects can be estimated by finding similar historical projects and re-using the recorded effort. To make estimation by analogy feasible it became necessary to construct a software tool, dubbed ANGEL, which allowed the collection of historical project data and the generation of estimates for new software projects. A substantial empirical validation of the approach was made encompassing approximately 250 real historical software projects across eight industrial data sets, using stepwise regression as a benchmark. Significance tests on the results accepted the hypothesis (at the 1% confidence level) that estimation by analogy is a superior prediction system to stepwise regression in terms of accuracy. A study was also made of the sensitivity of the analogy approach. By growing project data sets in a pseudo time-series fashion it was possible to answer pertinent questions about the approach, such as, what are the effects of outlying projects and what is the minimum data set size? The main conclusions of this work are that estimation by analogy is a viable estimation technique that would seem to offer some advantages over algorithmic approaches including, improved accuracy, easier use of categorical features and an ability to operate even where no statistical relationships can be found

Object-oriented software development effort prediction using design patterns from object interaction analysis

Software project management is arguably the most important activity in modern software development projects. In the absence of realistic and objective management, the software development process cannot be managed in an effective way. Software development effort estimation is one of the most challenging and researched problems in project management. With the advent of object-oriented development, there have been studies to transpose some of the existing effort estimation methodologies to the new development paradigm. However, there is not in existence a holistic approach to estimation that allows for the refinement of an initial estimate produced in the requirements gathering phase through to the design phase. A SysML point methodology is proposed that is based on a common, structured and comprehensive modeling language (OMG SysML) that factors in the models that correspond to the primary phases of object-oriented development into producing an effort estimate. This dissertation presents a Function Point-like approach, named Pattern Point, which was conceived to estimate the size of object-oriented products using the design patterns found in object interaction modeling from the late OO analysis phase. In particular, two measures are proposed (PP1 and PP2) that are theoretically validated showing that they satisfy wellknown properties necessary for size measures. An initial empirical validation is performed that is meant to assess the usefulness and effectiveness of the proposed measures in predicting the development effort of object-oriented systems. Moreover, a comparative analysis is carried out; taking into account several other size measures. The experimental results show that the Pattern Point measure can be effectively used during the OOA phase to predict the effort values with a high degree of confidence. The PP2 metric yielded the best results with an aggregate PRED (0.25) = 0.874

A systematic review of software development cost estimation studies

This paper aims to provide a basis for the improvement of software estimation research through a systematic review of previous work. The review identifies 304 software cost estimation papers in 76 journals and classifies the papers according to research topic, estimation approach, research approach, study context and data set. A web-based library of these cost estimation papers is provided to ease the identification of relevant estimation research results. The review results combined with other knowledge provide support for recommendations for future software cost estimation research, including: 1) Increase the breadth of the search for relevant studies, 2) Search manually for relevant papers within a carefully selected set of journals when completeness is essential, 3) Conduct more studies on estimation methods commonly used by the software industry, and, 4) Increase the awareness of how properties of the data sets impact the results when evaluating estimation methods

Software project economics: A roadmap

The objective of this paper is to consider research progress in the field of software project economics with a view to identifying important challenges and promising research directions. I argue that this is an important sub-discipline since this will underpin any cost-benefit analysis used to justify the resourcing, or otherwise, of a software project. To accomplish this I conducted a bibliometric analysis of peer reviewed research articles to identify major areas of activity. My results indicate that the primary goal of more accurate cost prediction systems remains largely unachieved. However, there are a number of new and promising avenues of research including: how we can combine results from primary studies, integration of multiple predictions and applying greater emphasis upon the human aspects of prediction tasks. I conclude that the field is likely to remain very challenging due to the people-centric nature of software engineering, since it is in essence a design task. Nevertheless the need for good economic models will grow rather than diminish as software becomes increasingly ubiquitous