The consistency of empirical comparisons of regression and analogy-based software project cost prediction

OBJECTIVE - to determine the consistency within and between results in empirical studies of software engineering cost estimation. We focus on regression and analogy techniques as these are commonly used. METHOD – we conducted an exhaustive search using predefined inclusion and exclusion criteria and identified 67 journal papers and 104 conference papers. From this sample we identified 11 journal papers and 9 conference papers that used both methods. RESULTS – our analysis found that about 25% of studies were internally inconclusive. We also found that there is approximately equal evidence in favour of, and against analogy-based methods. CONCLUSIONS – we confirm the lack of consistency in the findings and argue that this inconsistent pattern from 20 different studies comparing regression and analogy is somewhat disturbing. It suggests that we need to ask more detailed questions than just: “What is the best prediction system?

Web Engineering: An Assessment of Empirical Research

Web engineering is the process used to create high-quality Web-based systems and applications that deliver a complex array of content and functionality to a broad population of end-users. As Web Engineering continues to grow in popularity with practitioners and academics alike, so far, there hasn\u27t been any assessment of its accumulated body of knowledge in terms of academic research. Because Web engineering was established as a new discipline some five years ago, it is perhaps time to take stock of the efforts made in this field. Using the Web Engineering Process Model developed by Pressman, this paper organizes and map progress made so far. The results suggest a significant need for theory-based research in Web Engineering. The paper discusses some of the managerial and research implications of the findings

A systematic review of software development cost estimation studies

This paper aims to provide a basis for the improvement of software estimation research through a systematic review of previous work. The review identifies 304 software cost estimation papers in 76 journals and classifies the papers according to research topic, estimation approach, research approach, study context and data set. A web-based library of these cost estimation papers is provided to ease the identification of relevant estimation research results. The review results combined with other knowledge provide support for recommendations for future software cost estimation research, including: 1) Increase the breadth of the search for relevant studies, 2) Search manually for relevant papers within a carefully selected set of journals when completeness is essential, 3) Conduct more studies on estimation methods commonly used by the software industry, and, 4) Increase the awareness of how properties of the data sets impact the results when evaluating estimation methods

Investigating effort prediction of web-based applications using CBR on the ISBSG dataset

As web-based applications become more popular and more sophisticated, so does the requirement for early accurate estimates of the effort required to build such systems. Case-based reasoning (CBR) has been shown to be a reasonably effective estimation strategy, although it has not been widely explored in the context of web applications. This paper reports on a study carried out on a subset of the ISBSG dataset to examine the optimal number of analogies that should be used in making a prediction. The results show that it is not possible to select such a value with confidence, and that, in common with other findings in different domains, the effectiveness of CBR is hampered by other factors including the characteristics of the underlying dataset (such as the spread of data and presence of outliers) and the calculation employed to evaluate the distance function (in particular, the treatment of numeric and categorical data)

An Experimental Comparison of Three Machine Learning Techniques for Web Cost Estimation

Many comparative studies on the performance of machine learning (ML) techniques for web cost estimation (WCE) have been reported in the literature. However, not much attention have been given to understanding the conceptual differences and similarities that exist in the application of these ML techniques for WCE, which could provide credible guide for upcoming practitioners and researchers in predicting the cost of new web projects. This paper presents a comparative analysis of three prominent machine learning techniques – Case-Based Reasoning (CBR), Support Vector Regression (SVR) and Artificial Neural Network (ANN) – in terms of performance, applicability, and their conceptual differences and similarities for WCE by using data obtained from a public dataset (www.tukutuku.com). Results from experiments show that SVR and ANN provides more accurate predictions of effort, although SVR require fewer parameters to generate good predictions than ANN. CBR was not as accurate, but its good explanation attribute gives it a higher descriptive value. The study also outlined specific characteristics of the 3 ML techniques that could foster or inhibit their adoption for WCE

Effort Estimation for Service-Oriented Computing Environments

The concept of service in Service-Oriented Architecture (SOA) makes possible to introduce other ideas like service composition, governance and virtualization. Each of these ideas, when exercised to an enterprise level, provides benefits in terms of cost and performance. These ideas bring many new opportunities for the project managers in making the estimates of effort required to produce SOA systems. This is because the SOA systems are different from traditional software projects and there is a lack of efficient metrics and models for providing a high level of confidence in effort estimation. Thus, in this paper, an efficient estimation methodology has been presented based on analyzing the development phases of past SOA based software systems. The objective of this paper is twofold: first, to study and analyze the development phases of some past SOA based systems; second, to propose estimation metrics based on these analyzed parameters. The proposed methodology is facilitated from the use of four regression(s) based estimation models. The validation of the proposed methodology is cross checked by comparing the predictive accuracy, using some commonly used performance measurement indicators and box-plots evaluation. The evaluation results of the study (using industrial data collected from 10 SOA based software systems) show that the effort estimates obtained using the multiple linear regression model are more accurate and indicate an improvement in performance than the other used regression models

A comparative analysis of maintainability approaches for web applications

Web applications incorporate important business assets and offer a convenient way for businesses to promote their services through the internet. Many of these web applications have evolved from simple HTML pages to complex applications that have high maintenance cost. The high maintenance cost of web applications is due to the inherent characteristics of web applications, to the fast internet evolution and to the pressing market which imposes short development cycles and frequent modifications. In order to control the maintenance cost, quantitative metrics and models for predicting web applications' maintainability must be used. Since, web applications are different from traditional software systems, models and metrics for traditional systems can not be applied to web applications. The reason for that is that web applications have special features such as hypertext structure, dynamic code generation and heterogenousity that can not be captured by traditional and object-oriented metrics. In this paper, we will provide a comparative analysis of the different approaches for predicting web applications

Design metrics for web application maintainability measurement

Many web applications have evolved from simple HTML pages to complex applications that have a high maintenance cost. This high maintenance cost is due to the heterogeneity of web applications, to fast Internet evolution and the fast- moving market which imposes short development cycles and frequent modifications. In order to control the maintenance cost, quantitative metrics for predicting web applications maintainability must be used. This paper provides an exploratory study for new design metrics used for measuring the maintainability of web applications from class diagrams. The metrics are based on Web Application Extension (WAE)for UML and will measure the following design attributes: size, complexity, coupling and reusability. In this study the metrics are applied to two web applications from the telecommunications domain