Construction and evaluation of a tool for quantifying uncertainty of software cost estimates

Software development effort estimation is a continuous challenge in the software industry. The inherent uncertainty of effort estimates, which is due to factors such as evolving technology and significant elements of creativity in software development, is an important challenge for software project management. The specific management challenge addressed in this thesis is to assess the uncertainty of effort required for a new software release in the context of incremental software development. The evaluated approach combines task-level estimates with historical data on the estimation accuracy of past tasks for this assessment, by creating effort prediction intervals. The approach was implemented in a web-based tool, and evaluated in the context of a large Norwegian software project with estimation data from three contracted software development companies. In the evaluation we compared the approach to a simpler baseline method, and we found that our suggested approach more consistently produced reasonably accurate prediction intervals. Several variants of the basic approach were investigated. Fitting the historical data to a parametric distribution consistently improved the efficiency of the produced prediction intervals, but the accuracy suffered in cases where the parametric distribution could not reflect the historical distribution of estimation accuracy. Clustering tasks based on size had a positive effect on the produced effort intervals, both in terms of accuracy and efficiency. We believe the suggested approach and tool can be useful in software development project planning and estimation processes providing useful information to support planning, budgeting and resource allocation

Data analysis in software engineering: an approach to incremental data-driven effort estimation.

Cost and effort estimation in software projects have been investigated for several years. Nonetheless, compared to other engineering fields, there is still a large number of projects that fail into different phases due to prediction errors. On average, large IT projects run 45 percent over budget and seven percent over time, while delivering 56 percent less value than predicted. Several effort estimation models have been defined in the past, mainly based on user experience or on data collected in previous projects, but no studies support an incremental effort estimation and tracking. Iterative development techniques, and in particular Agile techniques, partially support the incremental effort estimation, but due to the complexity of the estimation, the total effort always tend to be higher than expected. Therefore, this work focuses on defining an adequate incremental and data driven estimation model so as to support developers and project managers to keep track of the remaining effort incrementally. The result of this work is a set of estimation models for effort estimation, based on a set of context factors, such as the domain of application developed, size of the project team and other characteristics. Moreover, in this work we do not aim at defining a model with generic parameters to be applied in similar context, but we define a mathematical approach so as to customize the model for each development team. The first step of this work focused on analysis of the existing estimation models and collection of evidence on the accuracy of each model. We then defined our approach based on Ordinary Least Squares regression analysis (OLS)so as to investigate the existence of a correlation between the actual effort and other characteristics. While building the OLS models we analyzed the data set and removed the outliers to prevent them from unduly influencing the OLS regression lines obtained. In order to validate the result we apply a 10-fold cross-validation assessing the accuracy of the results in terms of R2, MRE and MdMRE. The model has been applied to two different case studies. First, we analyzed a large number of projects developed by means of the waterfall process. Then, we analyzed an Agile process, so as to understand if the developed model is also applicable to agile methodologies. In the first case study we want to understand if we can define an effort estimation model to predict the effort of the next development phase based on the effort already spent. For this reason, we investigated if it is possible to use: \u2022 the effort of one phase for estimating the effort of the next development phase \u2022 the effort of one phase for estimating the remaining project effort \u2022 the effort spent up to a development phase to estimate its effort \u2022 the effort spent up to a development phase to estimate the remaining project effort Then, we investigated if the prediction accuracy can be improved considering other common context factors such as project domain, development language, development platform, development process, programming language and number of Function Points. We analyzed projects collected in the ISBSG dataset and, considering the different context factors available, we run a total of 4500 analysis, to understand which are the more suitable factors to be applied in a specific context. The results of this first case study show a set of statistically significant correlations between: (1) the effort spent in one phase and the effort spent in the following one; (2) the effort spent in a phase and the remaining effort; (3) the cumulative effort up to the current phase and the remaining effort. However, the results also show that these estimation models come with different degrees of goodness of fit. Finally, including further information, such as the functional size, does not significantly improve estimation quality. In the second case study, a project developed with an agile methodology (SCRUM) has been analyzed. In this case, we want to understand if is possible to use our estimation approach, so as to help developers to increase the accuracy of the expert based estimation. SCRUM, effort estimations are carried out at the beginning of each sprint, usually based on story points. The usage of functional size measures, specifically selected for the type of application and development conditions, is expected to allow for more accurate effort estimates. The goal of the work presented here is to verify this hypothesis, based on experimental data. The association of story measures to actual effort and the accuracy of the resulting effort model is evaluated. The study shows that developers\u2019 estimation is more accurate than those based on functional measurement. In conclusion, our study shows that, easy to collect functional measures do not help developers in improving the accuracy of the effort estimation in Moonlight SCRUM. These models derived in our work can be used by project managers and developers that need to estimate or control the project effort in a development process. These models can also be used by the developers to track their performances and understand the reasons of effort estimation errors. Finally the model help project managers to react as soon as possible and reduce project failures due to estimation errors. The detailed results are reported in the next sections as follows: \u2022 Chapter 1 reports the introduction to this work \u2022 Chapter 2 reports the related literature review on effort estimation techniques \u2022 Chapter 3 reports the proposed effort estimation approach \u2022 Chapter 4 describe the application of our approach to Waterfall process \u2022 Chapter 5 describe the application of our approach to SCRUM \u2022 Chapter 6 reports the conclusion and the future work

Complementing Measurements and Real Options Concepts to Support Inter-iteration Decision-Making in Agile Projects

Agile software projects are characterized by iterative and incremental development, accommodation of changes and active customer participation. The process is driven by creating business value for the client, assuming that the client (i) is aware of it, and (ii) is capable to estimate the business value, associated with the separate features of the system to be implemented. This paper is focused on the complementary use of measurement techniques and concepts of real-option-analysis to assist clients in assessing and comparing alternative sets of requirements. Our overall objective is to provide systematic support to clients for the decision-making process on what to implement in each iteration. The design of our approach is justified by using empirical data, published earlier by other authors

COSTMODL: An automated software development cost estimation tool

The cost of developing computer software continues to consume an increasing portion of many organizations' total budgets, both in the public and private sector. As this trend develops, the capability to produce reliable estimates of the effort and schedule required to develop a candidate software product takes on increasing importance. The COSTMODL program was developed to provide an in-house capability to perform development cost estimates for NASA software projects. COSTMODL is an automated software development cost estimation tool which incorporates five cost estimation algorithms including the latest models for the Ada language and incrementally developed products. The principal characteristic which sets COSTMODL apart from other software cost estimation programs is its capacity to be completely customized to a particular environment. The estimation equations can be recalibrated to reflect the programmer productivity characteristics demonstrated by the user's organization, and the set of significant factors which effect software development costs can be customized to reflect any unique properties of the user's development environment. Careful use of a capability such as COSTMODL can significantly reduce the risk of cost overruns and failed projects

Deriving Models for Software Project Effort Estimation By Means of Genetic Programming

Software engineering, effort estimation, genetic programming, symbolic regression. This paper presents the application of a computational intelligence methodology in effort estimation for software projects. Namely, we apply a genetic programming model for symbolic regression; aiming to produce mathematical expressions that (1) are highly accurate and (2) can be used for estimating the development effort by revealing relationships between the project’s features and the required work. We selected to investigate the effectiveness of this methodology into two software engineering domains. The system was proved able to generate models in the form of handy mathematical expressions that are more accurate than those found in literature.

Rancang Bangun Aplikasi Use Case Point Untuk Estimasi Harga Perangkat Lunak (Studi Kasus: Perusahaan DTS)

Perusahaan DTS merupakan perusahaan yang bergerak dalam bidang pengembangan perangkat lunak berbasis proyek. Sebelum proyek pengembangan perangkat lunak dikerjakan, tim pengembang di DTS melakukan estimasi nilai harga proyek perangkat lunak terlebih dahulu untuk kemudian dibuatkan dokumen penawaran. Namun, sebelum dokumen penawaran dibuat, tim pengembang harus melaporkan dulu kepada direktur untuk dilakukan validasi. Setelah itu, dokumen penawaran dicetak oleh sekretaris. Selama proses estimasi, tim pengembang sering menemui beberapa permasalahan. Pertama, dalam proses estimasi harga perangkat lunak, tim pengembang berbeda pendapat antara anggota satu dengan anggota lainnya. Hal ini karena dalam menentukan harga, masih menggunakan pendapat masingmasing individu tanpa ada dasar yang jelas. Kedua, dalam melakukan validasi proyek, direktur kesulitan dalam memberikan keputusan apakah nilai proyek yang diajukan oleh tim pengembang terlalu murah atau terlalu mahal. Ketiga, terkadang dalam melakukan estimasi harga, direktur dan tim pengembang sering salah paham karena tim tidak melaporkan hasil estimasi untuk dilakukan validasi. vi Oleh karena itu, aplikasi Use Case Point sangat dibutuhkan untuk estimasi harga perangkat lunak berdasarkan metode Use Case Point (UCP) dan proses bisnis perusahaan DTS. UCP merupakan metode untuk menentukan usaha (Effort) berdasarkan Use Case. Nilai Effort kemudian diubah kedalam bentuk nilai biaya. Sebenarnya, terdapat beberapa metode dalam estimasi usaha(Effort), salah satunya yaitu Metode Use Case Point(UCP). Metode estimasi menggunakan Use Case Point dipilih karena metode ini memiliki keakuratan lebih tinggi dibanding dengan metode lainnya. Metode pengembangan perangkat lunak untuk membuat aplikasi Use Case Point adalah metode incremental model. Metode incremental model dipilih karena dapat meminimalisir ketidaksesuaian antara kebutuhan pengguna dengan aplikasi yang sudah dibuat. Hasil dari pengerjaan tugas akhir peneliti ini berupa Aplikasi Use Case Point melalui 3 kali increment. Increment pertama ditambahkan fitur estimasi usaha. Increment kedua ditambahkan fitur biaya, increment ketiga ditambahkan fitur-fitur untuk melakukan kalibrasi perhitungan estimasi. Selain itu, dihasilkan dokumen Spesifikasi Kebutuhan Perangkat Lunak (SKPL), Deskripsi Perancangan Perangkat Lunak (DPPL), dokumen pengujian, dan dokumen panduan. ======================================================================================================= DTS Corporation is a company that focused in developing application based project. Before start to be developed, DTS’s developer team estimate about the price of the project. After that, they submit bidding document to client for approving. And then, printed out by secretary. During estimation process, developer team often faced some problems. First, Developer Team have difficulties to estimate about the price of the project dan sometimes having different argument with team member. It’s happened because they have no standard or rigid procedure to estimate the price of the software, So they estimate the software price by feeling of each other. Secound, director feel difficult to decide or validate the result of estimation reported by team. Director feel afraid wheter the estimation result done by developer team is higer or lower. Third, sometimes director have miss communication with developer team because developer team forget to report result of the estimation. viii Because of that, researcher develop Use Case Point Cost Estimate Software based Use Case Point (UCP) method and business process of the cost estimation in DTS Corporation. UCP is method that used for estimate Effort. The Effort and then will be converted to form of cost value. Actually, there are some Effort estimate methods, Use Case Point (UCP) is one of them. Use Case Point method is chosen because the accuration of the estimate result is higher than another method. The Development methodology that chosen for develop Use Case Point Aplication is incremental model. The reason for chosing this methodology is because incremental model can avoid incompacitibility between application and user needs. The result of this final project is Use Case Point Cost Estmate Software through three times of increment. The first increment added feature Effort estimate, second increment added feature cost estimate, third increment added feature that peform calibrate estimation result. Additionally, generated documents Software Requirment Spesification (SRS), Software Design Description (SDD), Testing Documentation, and User Guid

Evolution of Ada technology in the flight dynamics area: Implementation/testing phase analysis

An analysis is presented of the software engineering issues related to the use of Ada for the implementation and system testing phases of four Ada projects developed in the flight dynamics area. These projects reflect an evolving understanding of more effective use of Ada features. In addition, the testing methodology used on these projects has changed substantially from that used on previous FORTRAN projects

Estimating, planning and managing Agile Web development projects under a value-based perspective

Context: The processes of estimating, planning and managing are crucial for software development projects, since the results must be related to several business strategies. The broad expansion of the Internet and the global and interconnected economy make Web development projects be often characterized by expressions like delivering as soon as possible, reducing time to market and adapting to undefined requirements. In this kind of environment, traditional methodologies based on predictive techniques sometimes do not offer very satisfactory results. The rise of Agile methodologies and practices has provided some useful tools that, combined with Web Engineering techniques, can help to establish a framework to estimate, manage and plan Web development projects. Objective: This paper presents a proposal for estimating, planning and managing Web projects, by combining some existing Agile techniques with Web Engineering principles, presenting them as an unified framework which uses the business value to guide the delivery of features. Method: The proposal is analyzed by means of a case study, including a real-life project, in order to obtain relevant conclusions. Results: The results achieved after using the framework in a development project are presented, including interesting results on project planning and estimation, as well as on team productivity throughout the project. Conclusion: It is concluded that the framework can be useful in order to better manage Web-based projects, through a continuous value-based estimation and management process.Ministerio de Economía y Competitividad TIN2013-46928-C3-3-