Effort Estimation Methods in Software Development Using Machine Learning Algorithms

Estimation of effort for the proposed software is a standout amongst the most essential activities in project management. Proper estimation of effort is often desirable in order to avoid any sort of failures in a project and is the practice to adopted by developers at the very beginning stage of the software development life cycle. Estimating the effort and schedule with a higher accuracy is a challenge that attracts attention of researchers as well as practitioners. Predicting the effort required to develop a software to a certain level of accuracy is definitely a difficult assignment for a manager or system analyst, when the requirements are not very clearly identified. Effort estimation helps project managers to determine time and effort required for the successful completion of the project. In order to help the organization in developing qualitative products within a planned time frame, the job of appropriate software effort estimation is of primary requirement. For measuring the cost and effort of software development, traditional software estimation techniques like Constructive Cost Estimation (COCOMO) model and Function Point Analysis (FPA) have not been proved very much satisfactory, because of uncertainties associated with parameters such as Line Of Code (LOC) and Function Point (FP) respectively, used for procedural programming concept. The procedural oriented design splits the data and procedure, whereas accepted practice of present day i.e., the object-oriented design combines both of them Since class and use case are the basic logical units of an object-oriented system, the use of Class Point (CP) and Use Case Point (UCP) approach to estimate the project effort helps to get more accurate result. For projects based on the aspect of Web Engineering, effort estimation practice is identified as a critical issue Considering these facts, there is a strong need for formal estimation of web-based projects, which can be accomplished by the help of International Software Benchmarking Standards Group (ISBSG) dataset. Similarly, in case of agile projects, Story Point Approach (SPA) is used to measure the effort required to implement a user story. By adding up the estimates of user stories which were nished during an iteration (story point iteration), the project velocity is obtained. The dataset related to CP, UCP and SPA are collected from previous projects mentioned in few research articles or from industries in order to assess the results. In order to create results of estimation with more accuracy, when managing issues of complex connections in the middle of inputs as well as yields, and where, there is a distortion in the inputs by high noise levels, the application of machine learning (ML) techniques helps to bring out results with more accuracy. A number of past research studies indicate that no single technique turns out to be the best for all cases. This is because of the dependency of system's execution altogether on the predicted function types, variations in properties of collected data, number of tests, noise ratio and so on. Hence the use of ML techniques in order to cope with issues arises in real-life situation is considered to be worthwhile. The research work carried out here presents the use of various ML techniques for software effort estimation using CP, UCP, Web-based and SPA approaches. The ML techniques are implemented taking into consideration of related dataset to predict the required effort

Effort Estimation of Agile and Web-Based Software Using Artificial Neural Networks

The agile methodology of software development is accepted as a superior alternative to conventional methods of software development, because of its inherent benefits like iterative development, rapid delivery and reduced risk. Hence, software developers are required to estimate the effort necessary to develop projects by agile methodology in an efficient manner because the requirements keep on changing. Web has become a part and parcel of our lives. People depend on Internet for almost everything these days. Many business units depend on Internet for communication with clients and for outsourcing load to other branches. In such a scenario, there is a necessity of efficient development of web-based software. For improving the efficiency of software development, resource utilization must be optimum. For achieving this, we need to be able to ascertain effectively, what kind of people/materials are required in what quantity, for development. This research aims at developing efficient effort estimation models for agile and web-based software by using various neural networks such as Feed-Forward Neural Network (FFNN), Radial Basis Function Neural Network (RBFN), Functional Link Artificial Neural Network (FLANN) and Probabilistic Neural Network (PNN) and provide a comparative assessment of their performance. The approach used for agile software effort estimation is the Story Point Approach and that for web-based software effort estimation is the IFPUG Function Point Approach

Software Effort Estimation Accuracy Prediction of Machine Learning Techniques: A Systematic Performance Evaluation

Software effort estimation accuracy is a key factor in effective planning, controlling and to deliver a successful software project within budget and schedule. The overestimation and underestimation both are the key challenges for future software development, henceforth there is a continuous need for accuracy in software effort estimation (SEE). The researchers and practitioners are striving to identify which machine learning estimation technique gives more accurate results based on evaluation measures, datasets and the other relevant attributes. The authors of related research are generally not aware of previously published results of machine learning effort estimation techniques. The main aim of this study is to assist the researchers to know which machine learning technique yields the promising effort estimation accuracy prediction in the software development. In this paper, the performance of the machine learning ensemble technique is investigated with the solo technique based on two most commonly used accuracy evaluation metrics. We used the systematic literature review methodology proposed by Kitchenham and Charters. This includes searching for the most relevant papers, applying quality assessment criteria, extracting data and drawing results. We have evaluated a state-of-the-art accuracy performance of 28 selected studies (14 ensemble, 14 solo) using Mean Magnitude of Relative Error (MMRE) and PRED (25) as a set of reliable accuracy metrics for performance evaluation of accuracy among two techniques to report the research questions stated in this study. We found that machine learning techniques are the most frequently implemented in the construction of ensemble effort estimation (EEE) techniques. The results of this study revealed that the EEE techniques usually yield a promising estimation accuracy than the solo techniques.Comment: Pages: 27 Figures: 15 Tables:

Challenges of using software size in agile software development: A systematic literature review

Academic Papers at IWSM Mensura, IWSM-Mensura 2018; Beijing; China; 19 September 2018 through 20 September 2018Software size is a fundamental measure for software management. Size is used for a variety of purposes, such as benchmarking, normalization, and portfolio measurement, and it is frequently considered as the sole input of estimation. Estimations can be produced for various reasons; e.g., to predict effort, cost and duration of software development projects. There are different types of software size measures. Particularly in projects where agile methodologies are adopted, measurement becomes a significant challenge as it is perceived as a non-value-added task and records of tasks such as requirements identification are not always consistent. The difficulties of applying traditional size measurement techniques in agile contexts, however, do not diminish the need, and new methods and techniques are introduced to improve the manageability of the agile projects. In this paper, we discuss estimation and measurement approaches in relation with ―software size in agile contexts. Based on this review, we present the perceptions of software size and related challenges, such as misinterpretation of size, difficulties in implementation, and acceptability of the measurement processes. We anticipate that providing a baseline for the state of software size measures in agile contexts and presenting related challenges, particularly in terms of its acceptability by practitioners can shed light on the development of new techniques

Lean Forecasting In Software Projects

Quando se desenvolve um projeto de software, é reconhecível que estimativas precisas do esforço envolvido no desenvolvimento são uma parte importante na gestão bem-sucedida do projeto. Embora este processo seja tão importante, desenvolvedores e especialistas não conseguem normalmente estimar precisamente o esforço, tempo e custo que o projeto a ser desenvolvido terá. Isto é inerente à incerteza subjacente à sua atividade. Depois da primeira estimativa do esforço ser feita, o projeto pode, com alguma probabilidade, necessitar de se adaptar a circunstâncias em evolução, o que pode levar a mudanças nas características do projeto, e subsequentemente levar a que os gestores ponham mais pressão nos desenvolvedores para que sejam respeitados os prazos de entrega. No fim, o desenvolvimento do projeto irá, provavelmente, atrasar-se e estes atrasos não só afetam a equipa de desenvolvimento, mas também outras partes da empresa, como os departamentos responsaveis pelos funcionários e pelo marketing. Isto pode, em algumas situações, levar a que a empresa perca tempo e muitas vezes a confiança do cliente interessado no projeto. Mesmo que a estimativa seja precisa o suficiente para que as datas de entrega sejam respeitadas, métodos que dependem das estimativas de humanos consomem, normalmente, muito tempo, o que pode representar um problema quando equipas gastam tempo precioso a fazer estimativas. De maneira a mitigar estes problemas, iremos procurar identificar as motivações e forças em jogo no processo de fazer estimativas precisas e determinar que métodos de previsão alcançam os resultados mais precisos com alguma generalização, de modo a satisfazer a variedade de projetos de software existente. Vamos nos focar nos métodos de previsão devido à sua automaticidade, que irá ajudar a reduzir o tempo que as equipas gastam em estimações, mantendo a precisão dos resultados. Este método deve, também, ser fácil de perceber, implementar e usar, logo o número de dados que deve receber e a sua dificuldade de obter deve ser reduzida. As previsões do método devem conter um certo nível de ambiguidade, de modo a representar melhor o problema. Para a fase de validação do método, uma ferramenta baseada no método irá ser desenvolvida, testada em termos de eficácia e precisão contra outros métodos existentes, e irá ser integrada com ferramentas de gest�\xA3o de desenvolvimento de software, de modo a validar a sua usabilidade em projetos reais durante a fase de desenvolvimento destes. Assim, o objetivo principal desta dissertação é o de ajudar a reduzir o tempo perdido em estimações, mantendo ou até melhorando a precisão das previsões feitas e mantendo a facilidade de percepção e de uso para os desenvolvedores e equipas que utilizem este método.When developing a software project, it's recognisable that accurate estimations of development effort play an important part in the successful management of the project. Although this process is so important, developers and experts can't usually estimate accurately the effort, time and cost of a project to be developed. This is inherit to the uncertainty that underlies their activity. After the first estimation of the effort, the project may, with some likelihood, need to adapt to evolving circumstances, which may lead to changes in its scope, and consequently lead to managers putting pressure in the developers to respect delivery dates. In the end, the project's development will, probably, get delayed and this delays not only affect the development team but also other parts of the company, such as staffing or marketing. This could, in some situations, lead to the company losing time and in many times the trust of the stakeholder. Even if the estimate is accurate enough so that delivery dates are respected, methods that relay on Human estimation are, often, time consuming, what can represent a problem when teams waste precious time in making estimations. In order to mitigate this problems, we will seek to identify the motivations and forces playing in a accurate estimate and determine which forecast method could provide the bet- ter accuracy with some generalization, in order to satisfy the existing variety of software projects. We will focus on forecast methods because of their automatability, that will help reduce the time teams waste on estimations, still delivering accurate results. This method must also be easy to understand, implement and use, so the number of inputs required and the difficulty to collect this inputs should be low. The output of the method should contain a certain level of uncertainty, in order to better represent the problem. In order to validate this method, a tool based on it will be developed, tested in terms of effective- ness and accuracy against other existing methods, and it will be integrated with software development management tools to validate it's ability to be used in real projects during their development phase. Following this lines, the main goal of this dissertation is to help reduce the time wasted in estimations, while maintaining or even increase the accuracy of the prediction made and maintaining the understandability and usability easy for the teams and developers using it

Evaluating an automated procedure of machine learning parameter tuning for software effort estimation

Software effort estimation requires accurate prediction models. Machine learning algorithms have been used to create more accurate estimation models. However, these algorithms are sensitive to factors such as the choice of hyper-parameters. To reduce this sensitivity, automated approaches for hyper-parameter tuning have been recently investigated. There is a need for further research on the effectiveness of such approaches in the context of software effort estimation. These evaluations could help understand which hyper-parameter settings can be adjusted to improve model accuracy, and in which specific contexts tuning can benefit model performance. The goal of this work is to develop an automated procedure for machine learning hyper-parameter tuning in the context of software effort estimation. The automated procedure builds and evaluates software effort estimation models to determine the most accurate evaluation schemes. The methodology followed in this work consists of first performing a systematic mapping study to characterize existing hyper-parameter tuning approaches in software effort estimation, developing the procedure to automate the evaluation of hyper-parameter tuning, and conducting controlled quasi experiments to evaluate the automated procedure. From the systematic literature mapping we discovered that effort estimation literature has favored the use of grid search. The results we obtained in our quasi experiments demonstrated that fast, less exhaustive tuners were viable in place of grid search. These results indicate that randomly evaluating 60 hyper-parameters can be as good as grid search, and that multiple state-of-the-art tuners were only more effective than this random search in 6% of the evaluated dataset-model combinations. We endorse random search, genetic algorithms, flash, differential evolution, and tabu and harmony search as effective tuners.Los algoritmos de aprendizaje automático han sido utilizados para crear modelos con mayor precisión para la estimación del esfuerzo del desarrollo de software. Sin embargo, estos algoritmos son sensibles a factores, incluyendo la selección de hiper parámetros. Para reducir esto, se han investigado recientemente algoritmos de ajuste automático de hiper parámetros. Es necesario evaluar la efectividad de estos algoritmos en el contexto de estimación de esfuerzo. Estas evaluaciones podrían ayudar a entender qué hiper parámetros se pueden ajustar para mejorar los modelos, y en qué contextos esto ayuda el rendimiento de los modelos. El objetivo de este trabajo es desarrollar un procedimiento automatizado para el ajuste de hiper parámetros para algoritmos de aprendizaje automático aplicados a la estimación de esfuerzo del desarrollo de software. La metodología seguida en este trabajo consta de realizar un estudio de mapeo sistemático para caracterizar los algoritmos de ajuste existentes, desarrollar el procedimiento automatizado, y conducir cuasi experimentos controlados para evaluar este procedimiento. Mediante el mapeo sistemático descubrimos que la literatura en estimación de esfuerzo ha favorecido el uso de la búsqueda en cuadrícula. Los resultados obtenidos en nuestros cuasi experimentos demostraron que algoritmos de estimación no-exhaustivos son viables para la estimación de esfuerzo. Estos resultados indican que evaluar aleatoriamente 60 hiper parámetros puede ser tan efectivo como la búsqueda en cuadrícula, y que muchos de los métodos usados en el estado del arte son solo más efectivos que esta búsqueda aleatoria en 6% de los escenarios. Recomendamos el uso de la búsqueda aleatoria, algoritmos genéticos y similares, y la búsqueda tabú y harmónica.Escuela de Ciencias de la Computación e InformáticaCentro de Investigaciones en Tecnologías de la Información y ComunicaciónUCR::Vicerrectoría de Investigación::Sistema de Estudios de Posgrado::Ingeniería::Maestría Académica en Computación e Informátic

Modelo de Custo Híbrido para o Desenvolvimento de Software em Ambientes Ágeis

A estimativa de esforço é uma das principais tarefas no planeamento e gestão de qualquer projeto de desenvolvimento de software. Desde a sua proposta inicial até ao seu desenvolvimento e manutenção é crucial ter uma previsão precisa do esforço necessário em cada etapa. Estimativas muito elevadas irão certamente levar à perda de competitividade no mercado, por outro lado estimativas muito baixas poderão levar à falha de compromissos, datas de entrega e consequentemente à perda de dinheiro. Embora exista uma grande quantidade de técnicas e modelos de estimativa de esforço de software, a grande maioria foca-se no desenvolvimento de software tradicional. O surgimento de novas metodologias de trabalho, como as metodologias ágeis, levou a que a aplicabilidade dos modelos existentes seja reduzida, pois estas novas metodologias baseiam-se num conceito totalmente diferente do desenvolvimento de software tradicional. Apesar de nos últimos anos a utilização de metodologias ágeis, nas mais diversas áreas, ter crescido, continuam a ser escassos os métodos de estimativa criados especificamente para este tipo de ambientes. Nesta dissertação foi proposto um modelo de custo híbrido para o desenvolvimento de software em ambientes ágeis. O modelo proposto combina vertentes tanto de modelos de estimativas ágeis, bem como de modelos de estimativas de esforço tradicionais com técnicas de Machine Learning de modo a aumentar a precisão das estimativas produzidas. Os resultados dos modelos referentes às várias técnicas de Machine Learning utilizadas no modelo proposto foram comparados entre si e também com outros modelos existentes na literatura.ABSTRACT: Effort estimation is one of the major tasks in planning and managing any software development project. From the initial proposal to its development and maintenance it is crucial to have an accurate forecast of the effort required at each stage. Very high estimates will certainly lead to loss of competitiveness in the market, on the other hand very low estimates may lead to failure to meet commitments, delivery dates and consequently loss of money. Although there are a lot of techniques and models for software effort estimation, the vast majority focuses on traditional software development. The emergence of new methodologies, such as agile methodologies, has led to the reduced applicability of existing models, as these new methodologies are based on a totally different concept from the development of traditional software. Although in recent years the use of agile methodologies in the most diverse areas has grown, estimation methods created specifically for agile environments are still scarce. In this dissertation a hybrid cost model was proposed for software development in agile environments. The proposed model combines aspects of both agile estimation models as well as traditional effort estimation models with Machine Learning techniques in order to increase the precision of the estimates produced. The results of the models related to the various Machine Learning techniques used in the proposed model were compared with each other and also with other models present in the literature

Incorporating statistical and machine learning techniques into the optimization of correction factors for software development effort estimation

Accurate effort estimation is necessary for efficient management of software development projects, as it relates to human resource management. Ensemble methods, which employ multiple statistical and machine learning techniques, are more robust, reliable, and accurate effort estimation techniques. This study develops a stacking ensemble model based on optimization correction factors by integrating seven statistical and machine learning techniques (K-nearest neighbor, random forest, support vector regression, multilayer perception, gradient boosting, linear regression, and decision tree). The grid search optimization method is used to obtain valid search ranges and optimal configuration values, allowing more accurate estimation. We conducted experiments to compare the proposed method with related methods, such as use case points-based single methods, optimization correction factors-based single methods, and ensemble methods. The estimation accuracies of the methods were evaluated using statistical tests and unbiased performance measures on a total of four datasets, thus demonstrating the effectiveness of the proposed method more clearly. The proposed method successfully maintained its estimation accuracy across the four experimental datasets and gave the best results in terms of the sum of squares errors, mean absolute error, root mean square error, mean balance relative error, mean inverted balance relative error, median of magnitude of relative error, and percentage of prediction (0.25). The p-value for the t-test showed that the proposed method is statistically superior to other methods in terms of estimation accuracy. The results show that the proposed method is a comprehensive approach for improving estimation accuracy and minimizing project risks in the early stages of software development.Faculty of Applied Informatics, Tomas Bata University, (IGA/CebiaTech/2022/001, RVO/FAI/2021/002)Tomas Bata University in Zlin [RVO/FAI/2021/002, IGA/CebiaTech/2022/001

An Update on Effort Estimation in Agile Software Development: A Systematic Literature Review

[EN] Software developers require effective effort estimation models to facilitate project planning. Although Usman et al. systematically reviewed and synthesized the effort estimation models and practices for Agile Software Development (ASD) in 2014, new evidence may provide new perspectives for researchers and practitioners. This article presents a systematic literature review that updates the Usman et al. study from 2014 to 2020 by analyzing the data extracted from 73 new papers. This analysis allowed us to identify six agile methods: Scrum, Xtreme Programming and four others, in all of which expert-based estimation methods continue to play an important role. This is particularly the case of Planning Poker, which is very closely related to the most frequently used size metric (story points) and the way in which software requirements are specified in ASD. There is also a remarkable trend toward studying techniques based on the intensive use of data. In this respect, although most of the data originate from single-company datasets, there is a significant increase in the use of cross-company data. With regard to cost factors, we applied the thematic analysis method. The use of team and project factors appears to be more frequent than the consideration of more technical factors, in accordance with agile principles. Finally, although accuracy is still a challenge, we identified that improvements have been made. On the one hand, an increasing number of papers showed acceptable accuracy values, although many continued to report inadequate results. On the other, almost 29% of the papers that reported the accuracy metric used reflected aspects concerning the validation of the models and 18% reported the effect size when comparing models.This work was supported by the Spanish Ministry of Science, Innovation and Universities through the Adapt@Cloud Project under Grant TIN2017-84550-R.Fernández-Diego, M.; Méndez, ER.; González-Ladrón-De-Guevara, F.; Abrahao Gonzales, SM.; Insfran, E. (2020). An Update on Effort Estimation in Agile Software Development: A Systematic Literature Review. IEEE Access. 8:166768-166800. https://doi.org/10.1109/ACCESS.2020.3021664S166768166800

Estimación de esfuerzo en proyectos de desarrollo de software con metodologías ágiles

[ES] La estimación de esfuerzo es una actividad fundamental en la industria de desarrollo de software. Ya sea mediante el empleo de modelos simples basados en el juicio u opinión de uno o varios miembros del equipo de desarrollo (expertos en el área) o modelos más complejos que utilizan datos históricos, algoritmos y/o herramientas sistematizadas, la finalidad de este proceso de estimación es poder contar con un valor representativo del esfuerzo que requiere llevar a cabo el proyecto, sirviendo de apoyo a las tareas de planificación y gestión del mismo. A su vez, constituye el principal soporte para la toma de decisiones respecto a factores como coste, recursos, duración y entregables del proyecto. En el presente trabajo de investigación se estudia el concepto de estimación de esfuerzo dentro del contexto de proyectos de desarrollo de software con metodologías ágiles. Para ello, primero se presentan las características particulares del movimiento ágil desde una perspectiva general, y luego se destacan los aspectos principales de las metodologías ágiles más conocidas en la actualidad. Asimismo, se introduce el concepto de estimación de esfuerzo, abarcando desde los aspectos más generales hasta su aplicación específica en proyectos de desarrollo ágil. Para analizar el estado del arte del uso de modelos de estimación de esfuerzo en este contexto, se ha desarrollado una revisión sistemática de literatura, en la que se han utilizado cinco motores de búsqueda o librerías digitales (ACM Digital Library, IEEE Xplore, Science Direct, Scopus, Web of Science) para la identificación de artículos de revistas científicas o conferencias publicados entre Diciembre 2013 y Diciembre 2017, respecto a modelos de estimación de esfuerzo utilizados en proyectos de desarrollo ágil. Un total de 46 estudios primarios han sido obtenidos después de la depuración de los resultados de la búsqueda inicial, los cuales han sido sometidos a una serie de preguntas de investigación para la extracción de datos relevantes para este estudio y el posterior análisis comparativo de los mismos. Esta revisión de literatura está basada en un estudio realizado en 2014, por lo que se ha intentado mantener, en la medida de lo posible, una coherencia con el proceso de análisis utilizado en dicho estudio. Esto permite obtener resultados comparables y complementarios a los anteriores, revelando una nueva tendencia en el uso de métodos algorítmicos y técnicas de Machine Learning para la estimación de esfuerzo en proyectos desarrollados con metodologías ágiles. Además, se ha realizado un análisis exploratorio del repositorio ISBSG en sus versiones R12-2012 y R1-2017. Esta evaluación, que sirve como valor añadido al estudio, se enfoca en aquellos proyectos desarrollados con metodologías ágiles y en la identificación de las variables relacionadas con el esfuerzo. Ambos resultados (la revisión de literatura y el análisis exploratorio) son comparados y combinados para obtener una visión conjunta del progreso de la estimación de esfuerzo en entornos de desarrollo ágil.[EN] Effort estimation is one of the fundamental practices in the software development industry. Whether it is through simple, expert judgment based models or more complex models that use historical data, algorithms and computational tools, the purpose of this estimation process is to obtain a representative value of the required effort to carry out a software project, supporting the planning and management activities thereof. Also, this value is the main input for decision making regarding cost, resources, duration, and deliverables of the project. In the present work, the concept of effort estimation is studied in the context of agile software development projects. First, the characteristics of the agile movement are presented from a general perspective and then the main agile methodologies are highlighted. Likewise, effort estimation is first overviewed in a general context, to later develop the specifics of its application in agile software development. To analyze the state of the art of effort estimation models in this context, a systematic literature review is developed, using five search engines or digital libraries (ACM Digital Library, IEEE Xplore, Science Direct, Scopus, Web of Science) to identify journal articles and conference papers published between December 2013 and December 2017, regarding effort estimation models in agile software development. 46 primary studies resulted after the search results depuration, which have been analyzed through a series of research questions for the extraction of relevant data and the comparative analysis thereof. This literature review is based on a previous study in 2014, so a coherence with the process of analysis use in that study has been kept as far as possible. This allows obtaining comparable and complementary results in relation to the previous study, revealing also a new trend on using algorithm methods and Machine Learning techniques for effort estimation in Agile projects. Also, an exploratory analysis of ISBSG repository has been carried out for 2012 and 2017 releases. This evaluation, as added value to the study, is focused on projects developed with agile methodologies and the identification of relevant variables for effort estimation. Both, the SRL and the exploratory analysis, are compared and combined to get a general overview of the progress of effort estimation in Agile.Méndez Solimán, ER. (2018). Estimación de esfuerzo en proyectos de desarrollo de software con metodologías ágiles. http://hdl.handle.net/10251/109927TFG