Estimating software project effort using analogies

Accurate project effort prediction is an important goal for the software engineering community. To date most work has focused upon building algorithmic models of effort, for example COCOMO. These can be calibrated to local environments. We describe an alternative approach to estimation based upon the use of analogies. The underlying principle is to characterise projects in terms of features (for example, the number of interfaces, the development method or the size of the functional requirements document). Completed projects are stored and then the problem becomes one of finding the most similar projects to the one for which a prediction is required. Similarity is defined as Euclidean distance in n-dimensional space where n is the number of project features. Each dimension is standardised so all dimensions have equal weight. The known effort values of the nearest neighbours to the new project are then used as the basis for the prediction. The process is automated using a PC based tool known as ANGEL. The method is validated on nine different industrial datasets (a total of 275 projects) and in all cases analogy outperforms algorithmic models based upon stepwise regression. From this work we argue that estimation by analogy is a viable technique that, at the very least, can be used by project managers to complement current estimation techniques

An Empirical Evaluation of Effort Prediction Models Based on Functional Size Measures

Software development effort estimation is among the most interesting issues for project managers, since reliable estimates are at the base of good planning and project control. Several different techniques have been proposed for effort estimation, and practitioners need evidence, based on which they can choose accurate estimation methods. The work reported here aims at evaluating the accuracy of software development effort estimates that can be obtained via popular techniques, such as those using regression models and those based on analogy. The functional size and the development effort of twenty software development projects were measured, and the resulting dataset was used to derive effort estimation models and evaluate their accuracy. Our data analysis shows that estimation based on the closest analogues provides better results for most models, but very bad estimates in a few cases. To mitigate this behavior, the correction of regression toward the mean proved effective. According to the results of our analysis, it is advisable that regression to the mean correction is used when the estimates are based on closest analogues. Once corrected, the accuracy of analogy-based estimation is not substantially different from the accuracy of regression based models

Analogy-based model for software project effort estimation

Accurate effort estimation of software development plays an important role to predict how much effort should be prepared during the works of a software project so that it can be completed on time and budget. Some sectors, e.g. banking sectors, were renowned fields of software projects, not only due to its huge size of project, but also extremely expensive and takes a long time to completion. Project estimation is essential for software development project able to run on time and budget with maximum quality. This study aims to investigate the accuracy of software project effort estimation with the Analogy method using three parameters: Euclidean, Manhattan and Minkowski distance. Analogy based estimation consists several stage included similarity measure, analogy adaptation, estimation calculation and model evaluation. The results showed that the best combination of Analogy methods was using Manhattan distance with an accuracy of 50% MMRE, 28% MdMRE and Pred(25) 48%. Thus, we can concluded that this model can be used to predict accurately

An Empirical investigation into software effort estimation by analogy

Most practitioners recognise the important part accurate estimates of development effort play in the successful management of major software projects. However, it is widely recognised that current estimation techniques are often very inaccurate, while studies (Heemstra 1992; Lederer and Prasad 1993) have shown that effort estimation research is not being effectively transferred from the research domain into practical application. Traditionally, research has been almost exclusively focused on the advancement of algorithmic models (e.g. COCOMO (Boehm 1981) and SLIM (Putnam 1978)), where effort is commonly expressed as a function of system size. However, in recent years there has been a discernible movement away from algorithmic models with non-algorithmic systems (often encompassing machine learning facets) being actively researched. This is potentially a very exciting and important time in this field, with new approaches regularly being proposed. One such technique, estimation by analogy, is the focus of this thesis. The principle behind estimation by analogy is that past experience can often provide insights and solutions to present problems. Software projects are characterised in terms of collectable features (such as the number of screens or the size of the functional requirements) and stored in a historical case base as they are completed. Once a case base of sufficient size has been cultivated, new projects can be estimated by finding similar historical projects and re-using the recorded effort. To make estimation by analogy feasible it became necessary to construct a software tool, dubbed ANGEL, which allowed the collection of historical project data and the generation of estimates for new software projects. A substantial empirical validation of the approach was made encompassing approximately 250 real historical software projects across eight industrial data sets, using stepwise regression as a benchmark. Significance tests on the results accepted the hypothesis (at the 1% confidence level) that estimation by analogy is a superior prediction system to stepwise regression in terms of accuracy. A study was also made of the sensitivity of the analogy approach. By growing project data sets in a pseudo time-series fashion it was possible to answer pertinent questions about the approach, such as, what are the effects of outlying projects and what is the minimum data set size? The main conclusions of this work are that estimation by analogy is a viable estimation technique that would seem to offer some advantages over algorithmic approaches including, improved accuracy, easier use of categorical features and an ability to operate even where no statistical relationships can be found

Using Functional Complexity Measures in Software Development Effort Estimation

Several definitions of measures that aim at representing the size of software requirements are currently available. These measures have gained a quite relevant role, since they are one of the few types of objective measures upon which effort estimation can be based. However, traditional Functional Size Measures do not take into account the amount and complexity of elaboration required, concentrating instead on the amount of data accessed or moved. This is a problem since the amount and complexity of the required data elaboration affect the implementation effort, but are not adequately represented by the current size measures, including the standardized ones. Recently, a few approaches to measuring aspects of user requirements that are supposed to be related with functional complexity and/or data elaboration have been proposed by researchers. In this paper, we take into consideration some of these proposed measures and compare them with respect to their ability to predict the development effort, especially when used in combination with measures of functional size. A few methods for estimating software development effort \u2013both based on model building and on analogy\u2013 are experimented with, using different types of functional size and elaboration complexity measures. All the most significant models obtained were based on a notion of computation density that is based on the number of computation flows in functional processes. When using estimation by analogy, considering functional complexity in the selection of analogue projects improved accuracy in all the evaluated cases. In conclusion, it appears that functional complexity is a factor that affects development effort; accordingly, whatever method is used for effort estimation, it is advisable to take functional complexity into due consideration

Optimization of fuzzy analogy in software cost estimation using linguistic variables

One of the most important objectives of software engineering community has been the increase of useful models that beneficially explain the development of life cycle and precisely calculate the effort of software cost estimation. In analogy concept, there is deficiency in handling the datasets containing categorical variables though there are innumerable methods to estimate the cost. Due to the nature of software engineering domain, generally project attributes are often measured in terms of linguistic values such as very low, low, high and very high. The imprecise nature of such value represents the uncertainty and vagueness in their elucidation. However, there is no efficient method that can directly deal with the categorical variables and tolerate such imprecision and uncertainty without taking the classical intervals and numeric value approaches. In this paper, a new approach for optimization based on fuzzy logic, linguistic quantifiers and analogy based reasoning is proposed to improve the performance of the effort in software project when they are described in either numerical or categorical data. The performance of this proposed method exemplifies a pragmatic validation based on the historical NASA dataset. The results were analyzed using the prediction criterion and indicates that the proposed method can produce more explainable results than other machine learning methods.Comment: 14 pages, 8 figures; Journal of Systems and Software, 2011. arXiv admin note: text overlap with arXiv:1112.3877 by other author

An Investigation into Software Estimation Methods

There are currently no fully validated estimation approaches that can accurately predict the effort needed for developing a software system (Kitchenham, et al, 1995). Information gathered at the early stages of system development is not enough to provide precise effort estimates, even though similar software systems may have been developed in the past. Where similar systems have been developed, there are often inherent differences in the features of these systems and in the development process used. These differences are often sufficient to significantly reduce estimation accuracy. Historically, cost estimation focuses on project effort and duration. There are many estimation techniques, but none is consistently ‘best’ (Shepperd, 2003). Software project management has become a crucial field of research due to the increasing role of software in today’s world. Improving the functions of project management is a main concern in software development organisation. The purpose of this thesis is to develop a new model which incorporates cultural and leadership factors in the cost estimation model, and is based on Case-Based Reasoning. The thesis defines a new knowledge representation “ontology” to provide a common understanding of project parameters. The associated system uses a statistically simulated bootstrap method, which helps in tuning the analogy approach before application to real projects. This research also introduces a new application of Profile Theory, which takes a formal approach to the measurement of leadership capabilities. A pilot study was performed in order to understand the approaches used for cost estimation in the Gulf region. Based on this initial study, a questionnaire was further refined and tested. Consequently, further surveys were conducted in the United Arab Emirates. It was noticed that most of the software development projects failed in terms of cost estimate. This was due to the lack of a precise software estimation model. These studies also highlighted the importance of leadership and culture in software cost estimation. Effort was estimated using regression and analogy. The Bootstrap method was used to refine the estimate of effort based on analogy, with correction for bias. Due to the very different nature of the core and support systems, a separate model was developed for each of them. As a result of the study, a new model for identifying and analysing was developed. The model was then evaluated, and conclusions were drawn. These show the importance of the model and the factors of organisational culture and leadership in software project development and in cost estimation. Potential areas for future research were identified