Are Delayed Issues Harder to Resolve? Revisiting Cost-to-Fix of Defects throughout the Lifecycle

Many practitioners and academics believe in a delayed issue effect (DIE); i.e. the longer an issue lingers in the system, the more effort it requires to resolve. This belief is often used to justify major investments in new development processes that promise to retire more issues sooner. This paper tests for the delayed issue effect in 171 software projects conducted around the world in the period from 2006--2014. To the best of our knowledge, this is the largest study yet published on this effect. We found no evidence for the delayed issue effect; i.e. the effort to resolve issues in a later phase was not consistently or substantially greater than when issues were resolved soon after their introduction. This paper documents the above study and explores reasons for this mismatch between this common rule of thumb and empirical data. In summary, DIE is not some constant across all projects. Rather, DIE might be an historical relic that occurs intermittently only in certain kinds of projects. This is a significant result since it predicts that new development processes that promise to faster retire more issues will not have a guaranteed return on investment (depending on the context where applied), and that a long-held truth in software engineering should not be considered a global truism.Comment: 31 pages. Accepted with minor revisions to Journal of Empirical Software Engineering. Keywords: software economics, phase delay, cost to fi

An Efficient Approach for Reviewing Security-Related Aspects in Agile Requirements Specifications of Web Applications

Defects in requirements specifications can have severe consequences during the software development lifecycle. Some of them may result in poor product quality and/or time and budget overruns due to incorrect or missing quality characteristics, such as security. This characteristic requires special attention in web applications because they have become a target for manipulating sensible data. Several concerns make security difficult to deal with. For instance, security requirements are often misunderstood and improperly specified due to lack of security expertise and emphasis on security during early stages of software development. This often leads to unspecified or ill-defined security-related aspects. These concerns become even more challenging in agile contexts, where lightweight documentation is typically produced. To tackle this problem, we designed an approach for reviewing security-related aspects in agile requirements specifications of web applications. Our proposal considers user stories and security specifications as inputs and relates those user stories to security properties via Natural Language Processing. Based on the related security properties, our approach identifies high-level security requirements from the Open Web Application Security Project (OWASP) to be verified, and generates a reading technique to support reviewers in detecting defects. We evaluate our approach via three experiment trials conducted with 56 novice software engineers, measuring effectiveness, efficiency, usefulness, and ease of use. We compare our approach against using: (1) the OWASP high-level security requirements, and (2) a perspective-based approach as proposed in contemporary state of the art. The results strengthen our confidence that using our approach has a positive impact (with large effect size) on the performance of inspectors in terms of effectiveness and efficiency.Comment: Preprint accepted for publication at the Requirements Engineering journal. arXiv admin note: text overlap with arXiv:1906.1143

An Empirical Study on the Role of Requirement Engineering in Agile Method and Its Impact on Quality

Agile Methods are characterized as flexible and easily adaptable. The need to keep up with multiple high-priority projects and shorter time-to-market demands could explain their increasing popularity. It also raises concerns of whether or not use of these methods jeopardizes quality. Since Agile methods allow for changes throughout the process, they also create probabilities to impact software quality at any time. This thesis examines the process of requirement engineering as performed with Agile method in terms of its similarities and differences to requirement engineering as performed with the more traditional Waterfall method. It compares both approaches from a software quality perspective using a case study of 16 software projects. The main contribution of this work is to bring empirical evidence from real life cases that illustrate how Agile methods significantly impacts software quality, including the potential for a larger number of defects due to poor non-functional requirements elicitation

Software Test Management Tool Evaluation Framework

Tarkvara testimine on korduvalt tõestanud oma olulisust tarkvara arenduse juures viimase kümnendi jooksul. Tarkvara testimise tunnustuse kasvuga on esile kerkinud paljud elektroonilised testide haldamissüsteemid (THS). Kuigi nende hindamiseks on mitmeid võimalusi, pole me siiski leidnud selleks ühtselt aktsepteeritud meetodit. Me usume, et see on probleem, mida tuleks uurida, sest THS hindamine on sageli subjektiivne, sõltudes pigem hindaja arvamusest kui objektiivsest lähenemisest. Sama mure on ka kvaliteedikontrolli meeskondade juhtidel, kui neil palutakse hinnata, kas THS, mida neil kasutatakse, vastab ettevõtte vajadustele. Mõistmaks THS hindamise olulisust, uurisime me testimisprotsesside alast kirjandust ning analüüsisime hetkel olemasolevaid rakendusi. Seejärel kaardistasime tuvastatud testimisprotsessid ning nende väljundid. Läbi viidud analüüsi tulemusena saadud andmete põhjal koostasime veebiküsitluse ning saatsime Eesti IT-firmadele. Uuringu tulemuste põhjal koostasime me THS hindamisraamistiku, mis aitab ettevõtetel mõõta, kas ostetav THS on joondatud firma eesmärkidega, ning vähendab hinnangu andmisel subjektiivsust. Meie raamistik võimaldab testimis- ning projektijuhtidel mõista, kas nende ettevõttes kasutusel olev rakendus vastab firma ootustele. Veendumaks loodud hindamisprotsessi kasutatavuses, viisime kvaliteedikontrolli spetsialistide seas läbi täiendava uuringu, mis kinnitas meie ootusi. Meie lõputöö edasi arendamiseks on mitmeid võimalusi. Raamistikust võib luua veebirakenduse, et seda oleks kergem kasutada või laiemalt levitada. Samuti tuleks uurimust laiendada, kaasates ning analüüsides teiste euroopa riikide IT-firmade THS nõudeid. Kindlasti ei saa mainimata jätta, et THS nõudeid tuleks aja möödudes täiendada vastavalt uutele trendidele kvaliteedikontrollis. Lõpetuseks me usume, et käesoleva lõputöö tulemus, THS hindamisraamistik, on praktiline ning vajalik panus tarkvara kvaliteedikontrolli kogukonnale.Software testing has proven its value for software development increasingly over the last decade. With the recognition of the benefits of software testing, several software test management tools (TMT) have emerged on the market. Although there exist different approaches, there is no method for a systematic TMT assessment. This is a problem because to our knowledge, evaluating TMT is rather a subjective task, heavily depending on the evaluators’ opinions rather than based on the objective approach. The same problem applies when test managers are asked to evaluate whether their currently used TMT meets the company’s expectations. In order to understand the importance and neccessity of TMT evaluation we perform a literature study on software testing processes and existing TMT market studies. Then we map together the identified test activities and test artifacts. The results help us formulate and design an online questionnaire and perform a TMT survey within the Estonian IT companies. Based on the survey results, a framework for evaluating TMT software is created. Such a framework could potentially help companies to measure the TMT suitability to company’s goals and to decrease subjectivity of the TMT assessment. The framework also provides test and project managers the understanding whether their current TMTs meet the company’s expectations. We validate the framework with a case study performed among Quality Assurance specialists to collect information on the framework usability. Possibilities for future work based on this thesis are numerous. The framework can be made into an application for ease of use and wider distribution. Expanding the research onto other European countries is another viable choice. Also expanding the TMT requirements based on new trends in testing can be taken into consideration. In conclusion, we believe this thesis contributes to the testing community with a practical TMT evaluation method

Guidelines for creating a testing process for a software - case study of comparing testing of two different size of slot game projects

Nowadays an important part of software development life cycle is software testing. As software and systems become more complex and integrated with other software and systems the importance of software testing becomes critical part of a successful software development process. Testing itself has a little value but for the software the testing is necessary. Well planned software testing can make mediocre software a great one whereas poorly executed testing can even harm the final product. Thus, it is important that there is well designed testing process for a software project. As every software project differ from each other the testing process can also vary from project to project. There are multiple levels, types and techniques of testing that can implemented to a testing process made for a certain software project. Creating an efficient and functional testing process can be a difficult task. In the case study of the paper it is concluded that two similar software projects of different size have multiple differences but do not force of creating two different testing process as both use the same kind development life cycle and both software projects are the same type, slot games. The major difference for testing is that as a major software project, compared to a minor one, has more requirements it is important to consider them in testing strategy and planning but not in the testing process

We Don't Need Another Hero? The Impact of "Heroes" on Software Development

A software project has "Hero Developers" when 80% of contributions are delivered by 20% of the developers. Are such heroes a good idea? Are too many heroes bad for software quality? Is it better to have more/less heroes for different kinds of projects? To answer these questions, we studied 661 open source projects from Public open source software (OSS) Github and 171 projects from an Enterprise Github. We find that hero projects are very common. In fact, as projects grow in size, nearly all project become hero projects. These findings motivated us to look more closely at the effects of heroes on software development. Analysis shows that the frequency to close issues and bugs are not significantly affected by the presence of project type (Public or Enterprise). Similarly, the time needed to resolve an issue/bug/enhancement is not affected by heroes or project type. This is a surprising result since, before looking at the data, we expected that increasing heroes on a project will slow down howfast that project reacts to change. However, we do find a statistically significant association between heroes, project types, and enhancement resolution rates. Heroes do not affect enhancement resolution rates in Public projects. However, in Enterprise projects, the more heroes increase the rate at which project complete enhancements. In summary, our empirical results call for a revision of a long-held truism in software engineering. Software heroes are far more common and valuable than suggested by the literature, particularly for medium to large Enterprise developments. Organizations should reflect on better ways to find and retain more of these heroesComment: 8 pages + 1 references, Accepted to International conference on Software Engineering - Software Engineering in Practice, 201

How Scrum Adds Value to Achieving Software Quality?

Scrum remains the most popular agile software development method implementation for a variety of reasons; one important motive is to improve software quality. Yet many organizations fail to achieve quality improvements through the use of Scrum, and existing research sheds little light on the value-add of Scrum for software quality. More specifically, (1) how notions of software quality among Scrum practitioners relate to established quality perspectives, (2) how Scrum helps teams to achieve higher software quality and (3) why some teams fail to meet the objective of higher quality. We addressed these gaps through a two-phased qualitative study based on 39 interviews and two in-depth case studies. We find that Scrum practitioners emphasize established notions of external quality comprising of conformity to business needs and absence of defects, while they also value internal quality, especially sustainable software design. Our results show that Scrum helps teams achieve both dimensions of quality by promoting some social antecedents (collaboration, psychological safety, accountability, transparency) and process-induced advantages (iterative development, formal inspection, and adaptation). Our findings unveil how these factors contribute to achieving software quality and under what conditions their effects can fail to materialize. These conditions include inconsistent Scrum implementations, cultural constraints, team tensions, and inaccessibility of end-users. In addition, the complexity of the project aggravates the impact of these conditions. Taken together, these findings show that Scrum can complement established quality assurance and software engineering practices by promoting a social environment that is conducive to creating high-quality software. Based on our findings, we provide specific recommendations for how practitioners can create such an environment

Agile Testing: Improving the Process : Case Descom

The thesis was assigned by Descom, a marketing and technology company based in Jyväskylä. The aim of the thesis was to research the current state of testing inside the organization, and to improve on the existing processes and practices. The thesis was carried out as a design research (applied action research), because the focus was improving already existing processes inside a company. The theory base contains a wide range of subjects from agile development models, the testing process, and process improvement models to agile testing. Without a solid base of multiple aspects it would have been impossible to understand how the testing works as a process and how it could have been improved. As Descom uses agile development it was necessary to follow the same principles throughout the writing of the thesis and on results. As a result information was provided for the company about the current state of testing procedures at Descom and how to improve the testing and processes in the future. The documentation already existing for testing such as the test plan and test report were updated. New documents such as a process improvement plan based on Critical Testing Processes, test strategy and testing policy were also created. Figures of the testing process, and the processes for all test types in use were created to be used as a visual aid for understanding the testing as whole at Descom.Opinnäytetyön toimeksianto tuli Descomilta, joka on Jyväskylästä lähtöisin oleva markkinointi ja teknologia yritys. Työn tavoitteena oli tutkia testauksen tilaa organisaatiossa ja kehittää olemassa olevia prosesseja ja käytäntöjä. Tutkimusmenetelmäksi valikoitui kehittämistutkimus, koska painotus oli olemassa olevien prosessien kehityksessä yrityksen sisällä. Teoriapohjassa käsiteltiin monia aiheita ketterästä sovelluskehityksestä, testausprosessista ja prosessi kehityksestä aina ketterään testaukseen asti. Ilman kattavaa pohjaa monille osa-alueille, olisi ollut mahdotonta ymmärtää miten testaus toimii prosessina ja miten sitä pystyy kehittämään. Descom toimii ketterän sovelluskehityksen mukaisesti projekteissaan, joten oli tärkeää seurata samoja ketteriä periaatteita läpi opinnäytetyön kirjoittamisen ja tuloksissa. Tuloksena saatiin tietoa yritykselle, siitä miten testaus on toiminut Descomilla ja kuinka testausta ja prosesseja tulisi kehittää tulevaisuudessa. Myös aiemmin olemassa olleet testausdokumentit päivitettiin. Uusina dokumentteina laadittiin suunnitelma prosessikehitykseen, joka perustui Critical Testing Processes –malliin, testausstrategia ja testauspolitiikka. Prosessikuvaus tehtiin kaavioita käyttäen, joilla kuvattiin prosessi kokonaisuutena sekä käytettävät testaustasot