Combining Data Analytics with Team Feedback to Improve the Estimation Process in Agile Software Development

We apply a mixed research method to improve the user stories estimation process in a German company following agile software development. We combine software project data analytics with elicitation of teams' feedback, identify root causes for wrong estimates and propose an improved version of the estimation process. Three major changes are adopted in the new process: a shorter non numerical scale for story points, an analogy-based estimation process, and retrospectives analyses on the accuracy of previous sprints estimates. The new estimation process is applied on a new project, and an improvement of estimates accuracy from 10% to 45% is observed

Effort Estimation in Agile Software Development: A Systematic Map Study

Introduction − Making effort estimation as accurate and suitable for software development projects becomes a fundamental stage to favor its success, which is a difficult task, since the application of these techniques in constant changing agile development projects raises the need to evaluate different methods frequently.  Objectives− The objective of this study is to provide a state of the art on techniques of effort estimation in agile software development (ASD), performance evaluation and the drawbacks that arise in its application.  Method− A systematic mapping was developed involving the creation of research questions to provide a layout of this study, analysis of related words for the implementation of a search query to obtain related studies, application of exclusion, inclusion, and quality criteria to filter nonrelated studies and finally the organization and extraction of the necessary information from each study.   Results− 25 studies were selected; the main findings are: the most applied estimation techniques in agile contexts are: Estimation of Story Points (SP) followed by Planning Poker (PP) and Expert Judgment (EJ). The most frequent solutions supported in computational techniques such as: Naive Bayes, Regression Algorithms and Hybrid System; also, the performance evaluation measures Mean Magnitude of Relative Error (MMRE), Prediction Assessment (PRED) and Mean Absolute Error (MAE) have been found to be the most commonly used. Additionally, parameters such as feasibility, experience, and the delivery of expert knowledge, as well as the constant particularity and lack of data in the process of creating models to be applied to a limited number of environments are the challenges that arise the most when estimating software in agile software development (ASD)    Conclusions− It has been found there is an increase in the number of articles that address effort estimation in agile development, however, it becomes evident the need to improve the accuracy of the estimation by using estimation  techniques supported in machine learning  that have been shown to facilitate and improve the performance of this.  Key Words − Effort Estimation; Agile Software Development; Issues and Challenges; Automatic Learning; Performance Metrics  Introducción − Realizar una estimación de esfuerzo lo más precisa y adecuada para proyectos de desarrollo de software, se ha convertido en pieza fundamental para favorecer el éxito y desarrollo de los mismos, sin embargo, aplicar este tipo de estimación en proyectos de desarrollo ágil, en donde los cambios son constantes, la convierte en una tarea muy compleja de implementar.    Objetivo− El objetivo de este estudio es proveer un estado del arte sobre técnicas de estimación de esfuerzo en desarrollo de software ágil, la evaluación de su desempeño y los inconvenientes que se presentan en su aplicación.    Metodología− Se desarrolló un mapeo sistemático que involucró la creación de preguntas de investigación con el fin de proveer una estructura a seguir, análisis de palabras relacionadas con el tema de investigación para la creación e implementación de una cadena de búsqueda para la identificación de estudios relacionados con el tema, aplicación de criterios de exclusión, inclusión y calidad a los artículos encontrados para poder descartar estudios no relevantes y finalmente la organización y extracción de la información necesaria de cada artículo.     Resultados− De los 25 estudios seleccionados; los principales hallazgos son: las técnicas de estimación más aplicadas en contextos ágiles son: Estimación por medio de Puntos de Historia (SP) seguidos de Planning Poker (PP) y Juicio de Expertos (EJ). Soluciones soportadas en técnicas computacionales como: Naive Bayes, Algoritmos de Regresión y Sistema Híbridos; también se ha encontrado que la Magnitud Media del Error Relativo (MMRE), la Evaluación de la Predicción (PRED) y Error Absoluto Medio (MAE) son las medidas de evaluación de desempeño más usadas. Adicionalmente, se ha encontrado que parámetros como la viabilidad, la experiencia y la entrega de conocimiento de expertos, así como la constante particularidad y falta de datos en el proceso de creación de modelos para aplicarse a un limitado número de entornos son los desafíos que más se presentan al momento de realizar estimación de software en el desarrollo de software ágil (ASD)    Conclusiones− Se ha encontrado que existe un aumento en la cantidad de artículos que abordan la estimación de esfuerzo en el desarrollo ágil, sin embargo, se hace evidente la necesidad de mejorar la precisión de la estimación mediante el uso de técnicas de estimación soportadas en el aprendizaje de máquina que han demostrado que facilita y mejora el desempeño de este.    

A Systematic Mapping of Factors Affecting Accuracy of Software Development Effort Estimation

Software projects often do not meet their scheduling and budgeting targets. Inaccurate estimates are often responsible for this mismatch. This study investigates extant research on factors that affect accuracy of software development effort estimation. The purpose is to synthesize existing knowledge, propose directions for future research, and improve estimation accuracy in practice. A systematic mapping study (a comprehensive review of existing research) is conducted to identify such factors and their impact on estimation accuracy. Thirty-two factors assigned to four categories (estimation process, estimator’s characteristics, project to be estimated, and external context) are identified in a variety of research studies. Although the significant impact of several factors has been shown, results are limited by the lack of insight into the extent of these impacts. Our results imply a shift in research focus and design to gather more in-depth insights. Moreover, our results emphasize the need to argue for specific design decisions to enable a better understanding of possible influences of the study design on the credibility of the results. For software developers, our results provide a useful map to check the assumptions that undergird their estimates, to build comprehensive experience databases, and to adequately staff design projects

Continual learning with a Bayesian approach for evolving the baselines of a leagile project portfolio

This article introduces a Bayesian learning approach for planning continuously evolving leagile project and portfolio baselines. Unlike the traditional project management approach, which uses static project baselines, the approach proposed in this study suggests learning from immediately prior experience to establish an evolving baseline for performance estimation. The principle of Pasteur’s quadrant is used to realize a highly practical solution, which extends the existing wisdom on leagile continuous planning. This study compares the accuracy of the proposed Bayesian approach with the traditional approach using real data. The results suggest that the evolving Bayesian baselines can generate a more realistic measure of performance than traditional baselines, enabling leagile projects and portfolios to be better managed in the continuously changing environments of today

A Bayesian Approach for Software Release Planning under Uncertainty

Release planning — deciding what features to implement in upcoming releases of a software system— is a critical activity in iterative software development.Many release planning methods exist but most ignore the inevitable uncertainty of future development effort and business value. The thesis investigates how to analyse uncertainty during release planning and whether analysing uncertainty leads to better decisions than if uncertainty is ignored. The thesis’s first contribution is a novel release planning method designed to analyse uncertainty in the context of the Incremental Funding Method, an incremental cost-value based approach to software development. Our method uses triangular distributions, Monte-Carlo simulation and multi-objective optimisation to shortlist release plans that maximise expected net present value and minimise investment cost and risk. The second contribution is a new release planning method, called BEARS, designed to analyse uncertainty in the context of fixed-date release processes.Fixed-date release processes are more common in industry than fixed-scope release processes. BEARS models uncertainty about feature development time and economic value using lognormal distributions. It then uses Monte-Carlo simulation and search-based multi-objective optimisation to shortlist release plans that maximise expected net present value and expected punctuality. The method helps release planners explore possible tradeoffs between these two objectives. The thesis’ third contribution is an experiment to study whether analysing uncertainty using BEARS leads to shortlisting better release plans than if uncertainty is ignored, or if uncertainty is analysed assuming fixed-scope releases. The experiment compares 5 different release planning models on 32 release planning problems.The results show that analysing uncertainty using BEARS leads to shortlisting release plans with higher expected net present value and higher expected punctuality than methods that ignore uncertainty or that assume fixed-scope releases.Our experiment therefore shows that analysing uncertainty can lead to better release planning decisions than if uncertainty is ignored

On top management support for software cost estimation

Inaccurate software cost estimates continue causing project overruns and hurting firms’ economy. This thesis addresses the problem by focusing on top management role in applying estimation methodologies successfully in organisations. The research questions are 1) How does top management support software cost estimation, and 2) What are the impacts of top management support for creating a good cost estimate for a software project? Three empirical studies, one quantitative and two qualitative, were conducted to address the research questions. The studies identified practices, through which top management is involved in cost estimation, and collected evidence on the impact of practices on estimation success. The quantitative study is based on views of 114 Finnish software professionals, and the quantitative studies are based on in-depth findings from three Finnish software producing companies and projects. The results show that top management support for estimation is mostly indirect. Management focuses on creating a successful environment for estimation instead of hands-on participation.The key factors of top management support include adequate resources, demonstrating the importance of estimation and seeking realism. This indirect role is enough for successful estimation.On the other hand, the results provide evidence that top management may negatively impact estimation. For example, unclear expectations may cause the project team to aim for the wrong outcome, expressed expectations may bias estimation and interpreting estimates as commitments may decrease estimators’ motivation and cause them to give high estimates. The practical implication is that top management should avoid direct participation in software estimation and focus on sustaining a supportive and unbiased environment. By doing this, many projects should be able to avoid failures hurting firms’ competitiveness. From the research perspective, the results provide evidence that people-related perspectives are an important factor in software estimation, implying that a shifting focus from methodologies toward managerial topics is justified.Epätarkat ohjelmistoprojektien kustannusarviot johtavat suunnitelmien ylittymiseen ja rasittavat yritysten taloutta. Tämä väitöskirja keskittyy ylimmän johdon rooliin arviointimenetelmien menestyksekkäässä soveltamisessa organisaatioissa. Väitöskirjan tutkimuskysymykset ovat 1) kuinka ylin johto tukee ohjelmistojen kustannusarviointia ja 2) mitä vaikutuksia johdon tuella hyvän kustannusarvion laatimiseksi on ohjelmistoprojektille? Väitöskirjan tulokset perustuvat yhteen määrälliseen ja kahteen laadulliseen tutkimukseen. Tutkimukset tunnistivat tapoja, joilla johto osallistui kustannusarviointiin sekä keräsi näyttöä osallistumiskäytänteiden vaikutuksista arvioinnin onnistumiseen. Määrällinen tutkimus pohjautuu 114 suomalaisen ohjelmistoammattilaisen näkemyksiin, kun taas laadulliset tutkimukset pohjautuvat löydöksiin kolmen suomalaisen ohjelmistoyrityksen toteuttamista kolmesta ohjelmistoprojektista. Tulokset osoittavat, että ylimmän johdon tuki arvioinnille on pääasiallisesti epäsuoraa. Johto keskittyy hyvien edellytysten luomiseen sen sijaan, että osallistuisi arviointiin henkilökohtaisesti. Tärkeimpiin tapoihin tukea arviointia kuuluvat mm. riittävien resurssien varmistaminen ja realististen arvioiden tavoittelu. Yllä kuvattu epäsuora osallistuminen on riittävää arvioinnin onnistumiseksi. Toisaalta johdon toimet voivat myös vaikuttaa arviointiin negatiivisesti. Esimerkiksi epäselvät tavoitteet saattavat johtaa väärien asioiden tavoitteluun, johdon esittämät odotukset voivat vääristää arvioinnin tuloksia ja arvioiden tulkitseminen lupauksiksi voi laskea arvioitsijoiden motivaatiota ja johtaa perusteettoman korkeiden arvioiden antamiseen. Esitettyjen tulosten perusteella johdon pitäisi välttää suoraa osallistumista arviointiin ja keskittyä arviointia tukevan ilmapiirin luomiseen. Näillä toimilla useat projektit voisivat todennäköisesti välttää yrityksille vahingolliset epäonnistumiset. Tutkimusnäkökulmasta tulokset osoittavat, että inhimilliset tekijät ovat merkittävässä roolissa kustannusarvioinnissa, ja lisäpanostukset johtamisnäkökulmien tutkimiseen ovat perusteltuja