136 research outputs found
On the real world practice of Behaviour Driven Development
Surveys of industry practice over the last decade suggest that Behaviour Driven Development is a popular Agile practice. For example, 19% of respondents to the 14th State of Agile annual survey reported using BDD, placing it in the top 13 practices reported. As well as potential benefits, the adoption of BDD necessarily involves an additional cost of writing and maintaining Gherkin features and scenarios, and (if used for acceptance testing,) the associated step functions. Yet there is a lack of published literature exploring how BDD is used in practice and the challenges experienced by real world software development efforts. This gap is significant because without understanding current real world practice, it is hard to identify opportunities to address and mitigate challenges. In order to address this research gap concerning the challenges of using BDD, this thesis reports on a research project which explored: (a) the challenges of applying agile and undertaking requirements engineering in a real world context; (b) the challenges of applying BDD specifically and (c) the application of BDD in open-source projects to understand challenges in this different context.
For this purpose, we progressively conducted two case studies, two series of interviews, four iterations of action research, and an empirical study. The first case study was conducted in an avionics company to discover the challenges of using an agile process in a large scale safety critical project environment. Since requirements management was found to be one of the biggest challenges during the case study, we decided to investigate BDD because of its reputation for requirements management. The second case study was conducted in the company with an aim to discover the challenges of using BDD in real life. The case study was complemented with an empirical study of the practice of BDD in open source projects, taking a study sample from the GitHub open source collaboration site.
As a result of this Ph.D research, we were able to discover: (i) challenges of using an agile process in a large scale safety-critical organisation, (ii) current state of BDD in practice, (iii) technical limitations of Gherkin (i.e., the language for writing requirements in BDD), (iv) challenges of using BDD in a real project, (v) bad smells in the Gherkin specifications of open source projects on GitHub. We also presented a brief comparison between the theoretical description of BDD and BDD in practice. This research, therefore, presents the results of lessons learned from BDD in practice, and serves as a guide for software practitioners planning on using BDD in their projects
Chatbots for Modelling, Modelling of Chatbots
Tesis Doctoral inédita leÃda en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de IngenierÃa Informática. Fecha de Lectura: 28-03-202
An Empirical Study of Using Large Language Models for Unit Test Generation
A code generation model generates code by taking a prompt from a code
comment, existing code, or a combination of both. Although code generation
models (e.g. GitHub Copilot) are increasingly being adopted in practice, it is
unclear whether they can successfully be used for unit test generation without
fine-tuning. We investigated how well three generative models (Codex,
GPT-3.5-Turbo, and StarCoder) can generate test cases to fill this gap. We used
two benchmarks (HumanEval and Evosuite SF110) to investigate the context
generation's effect in the unit test generation process. We evaluated the
models based on compilation rates, test correctness, coverage, and test smells.
We found that the Codex model achieved above 80% coverage for the HumanEval
dataset, but no model had more than 2% coverage for the EvoSuite SF110
benchmark. The generated tests also suffered from test smells, such as
Duplicated Asserts and Empty Tests.Comment: Preprint submitted to Journal of Systems and Software; 36 pages, 4
figures, 7 table
Understanding the Issues, Their Causes and Solutions in Microservices Systems: An Empirical Study
Many small to large organizations have adopted the Microservices Architecture
(MSA) style to develop and deliver their core businesses. Despite the
popularity of MSA in the software industry, there is a limited evidence-based
and thorough understanding of the types of issues (e.g., errors, faults,
failures, and bugs) that microservices system developers experience, the causes
of the issues, and the solutions as potential fixing strategies to address the
issues. To ameliorate this gap, we conducted a mixed-methods empirical study
that collected data from 2,641 issues from the issue tracking systems of 15
open-source microservices systems on GitHub, 15 interviews, and an online
survey completed by 150 practitioners from 42 countries across 6 continents.
Our analysis led to comprehensive taxonomies for the issues, causes, and
solutions. The findings of this study inform that Technical Debt, Continuous
Integration and Delivery, Exception Handling, Service Execution and
Communication, and Security are the most dominant issues in microservices
systems. Furthermore, General Programming Errors, Missing Features and
Artifacts, and Invalid Configuration and Communication are the main causes
behind the issues. Finally, we found 177 types of solutions that can be applied
to fix the identified issues. Based on our study results, we formulated future
research directions that could help researchers and practitioners to engineer
emergent and next-generation microservices systems.Comment: 35 pages, 5 images, 7 tables, Manuscript submitted to a Journal
(2023
Recommended from our members
Improving Application Quality using Mobile Analytics
The purpose of this research is to investigate and report on how mobile analytics can help real-world developers improve the quality of their apps efficiently and effectively. The research also considers the effects of mobile analytics in terms of the artefacts developed and maintained by the development team and also researches key characteristics of a range of mobile analytics tools and services.
Research Design: the research takes a developer-oriented perspective of using three complementary sources of data: 1) platform-level analytics, using Android Vitals as the primary analytics tool, 2) in-app analytics with a focus on runtime failures caused by crashes and freezes (known as Application Not Responding (ANR) in Android), and 3) interviews with developers. Action research techniques included roles of embedded developer, guide, and observer across different mobile app projects I was involved in. Hackathons were used to experiment with the speed and ability to find and address issues reported by the analytics tools used by the app developers. Their apps have a combined active user base of over 3,000,000 users. Many of these apps use a mainstream crash analytics library which was used to complement and contrast the results provided in the primary analytics tool. The research is intended to facilitate ease of future research and reproducibility, e.g. by using open-source projects as the code, bug reports, etc. are all published and available. This research was complemented by a) collaborating with professional developers who provided additional examples and results, and b) investigating grey material including grey literature and grey data.
The findings of this research highlights that using mobile analytics helped to reduce failure rates markedly, quickly, and effectively by applying techniques described here. Various limitations and flaws were found in the analytics tools; these provide cause for concern as they may affect the app’s placement in the app store and revenues. These limitations and flaws also make some issues in the apps harder to identify, prioritise, and fix.We identified ways to compensate for many of these and developed open-source software to facilitate additional analysis. Flaws and bugs were reported to the Android Vitals team at Google who acknowledged they would fix several of them. Several bugs were hard to reproduce, partly as Google deliberately hid pertinent details from the data they gather. Nonetheless app developers were able to ameliorate or fix the bugs for some issues even when they were not able to reproduce them.
Android Vitals shows the potential of how the combination of an app store and platform could be used to improve the quality of apps without users needing to actively participate. Some crashes were hard to reproduce and may be impractical to find before the app is released to end users. Developers can determine comparative improvements in their releases, such as whether they fixed a bug, by using Android Vitals and similar analytics tools; i.e. mobile analytics may help teams to determine whether they have improved the quality of their app even with flaws and limitations in the mobile analytics.</i
Testes End to End em Microserviços
The microservices architecture is a recent trend in the software engineering community, with the number of research articles in the field increasing, and more companies adopting the architectural style every year. However, the testing aspect of this architecture can sometimes be overlooked, with a lack of guidelines for its execution. Also, microservices testing introduce a lot of different challenges that are not faced when following a monolithic architecture. This work aims to fill some gaps in current microservices testing research by providing a study of five existing service virtualization tools, implementing them in a company system and extracting the results of the tool properties identified through a survey delivered to a convenience sample of software development experts. Finally, the mentioned experts in the microservices field validated the results of the research and the tool identified as the best and provided insights regarding the value of this work.A arquitetura de micros serviços é uma tendência recente na comunidade de engenharia de software, com o número de artigos na área a aumentar e mais empresas a adotar o estilo arquitetónico a cada ano. No entanto, a área dos testes end-to-end desta arquitetura pode à s vezes ser esquecido. Além disso, os testes de micros serviços apresentam muitos desafios diferentes que não são enfrentados ao seguir uma arquitetura monolÃtica. Este trabalho visa preencher algumas lacunas na pesquisa de teste de micros serviços atuais, fornecendo um estudo de cinco ferramentas de virtualização de serviço existentes, implementando-as num sistema empresarial e extraindo os resultados das propriedades da ferramenta identificadas por meio de um questionário entregue a uma amostra de conveniência de especialistas em desenvolvimento de software. Por fim, os citados especialistas na área de micros serviços validaram os resultados da pesquisa e a ferramenta identificada como a melhor e forneceram a sua visão geral sobre o valor deste trabalho
- …