Safety-Critical Systems and Agile Development: A Mapping Study

In the last decades, agile methods had a huge impact on how software is developed. In many cases, this has led to significant benefits, such as quality and speed of software deliveries to customers. However, safety-critical systems have widely been dismissed from benefiting from agile methods. Products that include safety critical aspects are therefore faced with a situation in which the development of safety-critical parts can significantly limit the potential speed-up through agile methods, for the full product, but also in the non-safety critical parts. For such products, the ability to develop safety-critical software in an agile way will generate a competitive advantage. In order to enable future research in this important area, we present in this paper a mapping of the current state of practice based on {a mixed method approach}. Starting from a workshop with experts from six large Swedish product development companies we develop a lens for our analysis. We then present a systematic mapping study on safety-critical systems and agile development through this lens in order to map potential benefits, challenges, and solution candidates for guiding future research.Comment: Accepted at Euromicro Conf. on Software Engineering and Advanced Applications 2018, Prague, Czech Republi

Analysis of the impact of test based development techniques (TDD, BDD, AND ATDD) to the software life cycle

Within the world of software development, there is a permanent need to create quality products that are capable of facing challenges in environments of changing requirements. The industry in this area is aware of this, and so, it makes use of software development methodologies such as: traditional or agile. Agile development represents a distancing from traditional approaches, allowing the creation of applications incrementally and iteratively and, thus, adjusting to the changing requirements of customers. For this reason, companies have recently adopted the use of its practices and techniques, e.g.: Test-Driven Development (TDD), Acceptance Test-Driven Development (ATDD), Behavior-Driven Development (BDD), among others. These techniques promise mainly to improve the quality of the software and the productivity of the programmers; therefore, many experiments, especially using TDD, have been made within the academy and the industry; which shows variant results (some with positive effects and others not so much). In addition, expert programmers have put these techniques into practice in software creation, getting satisfactory results due to the advantages offered by its use. The main objective of this work is to verify the impact produced by the techniques of software development based on tests (TDD, ATDD and BDD), analyzing its primordial promises. A literature research has been conducted in order to understand the strengths and weaknesses of each one of these techniques. With the intention of observing the effectiveness of TDD and BDD, an experiment was planned in an academic scenario, considering education and appropriate training to implement enough knowledge about them. With the results obtained, it was possible to understand that the techniques studied ensured the quality of the product developed and improved the productivity of the programmers; validating its effects within software development

Adopting test automation at Effizency to Improve Agility and Software Quality

Digital solutions have long been used as a means to solve everyday problems. Over time these solutions have been improved and refined. These solutions have emerged to help humans, primarily with tasks that can be cumbersome or repetitive. The demand for repetitive tasks and process optimization through digital means is peaked. For this reason, many companies in the software development area have adopted the use of automated tests capable of doing autonomously and quickly the tasks that previously required a lot of resources to perform, thus jumping on the "Automation Bandwagon”. By applying this approach, companies have the goal to improve the quality standards of the software offered by reducing the number of bugs and identifying them as early as possible in the development process. To observe the applicability, optimization, and efficiency of the automation of autonomous testing in a specific system, the concepts and technologies proposed here were applied in a professional scenario of a young company, Effizency. This company aims to facilitate the sale of energy services and electrical optimization. Effizency currently works using an agile approach and is constantly looking for ways to improve its development process. The company is currently facing the challenge of increasing the quality of its software and at the same time reducing the repetitiveness of its validation processes. Through this dissertation, it is expected that an improvement will be identified in terms of both a reduction of process repetition, time consumption and an increase in the test coverage performed. The main objective of this dissertation is to improve the quality of a company's software and the agility of its development process. This objective will be achieved using automated testing.As soluções digitais são usadas para resolver problemas do dia a dia há muito tempo. Ao longo do tempo, estas têm sido melhoradas e aperfeiçoadas. Estas soluções surgiram com o objetivo de ajudar o ser humano nas suas tarefas, maioritariamente tarefas que podem ser pesadas ou repetitivas. A procura pela automatização de tarefas repetitivas e de otimização de processos através de meios digitais está no seu auge. Por essa razão, muitas empresas na área de desenvolvimento de software adotaram o uso de testes automáticos capazes de fazer de forma autónoma e rápida as tarefas que anteriormente necessitariam de muitos recursos para realizar, entrando assim no “Vagão da Automatização”. Ao aplicar esta abordagem, as empresas têm o objetivo de melhorar os padrões de qualidade do software oferecidos reduzindo o número de bugs e identificando-os o mais cedo possível no processo de desenvolvimento. Com o intuito de observar a aplicabilidade, otimização e eficiência da automação de testes autónomos num sistema em concreto foi realizada a aplicação dos conceitos e tecnologias aqui propostos num cenário profissional de uma empresa jovem, Effizency. A Effizency trabalha atualmente utilizando uma abordagem ágil e está constantemente à procura de formas de melhorar o seu processo de desenvolvimento. Atualmente a empresa, enfrenta o desafio de aumentar a qualidade do seu software e, ao mesmo tempo reduzir o a repetitividade dos seus processos de validação. Através desta dissertação, espera-se que seja identificada uma melhoria tanto em termos de uma redução da repetição do processo, do consumo de tempo e de um aumento da cobertura dos testes realizados. O principal objetivo desta dissertação é melhorar a qualidade do software de uma empresa e a agilidade do seu processo de desenvolvimento. Este objetivo será alcançado através de testes automatizados

Showstoppers for Continuous Delivery in Small Scale Projects

Small scale projects outsourced to consultants provide their own difficulties when compared to more standard software development. Some of these problems are a lack of infrastructure and customers inexperienced with software development. This thesis is looking at the possibility of implementing continuous delivery in such an environment. The concrete problems are small projects with very little room for experimentation. But also the inexperience in automated testing which is essential for efficient regression testing. This led this thesis in two directions. The first one is how can you create a situation where continuous delivery could be beneficial, where developers prefer writing automated test cases instead of performing Ad Hoc manual testing during development and relying on a larger testing phase towards the end, much like what is done in waterfall development. The solution is to perform more deliveries to the customer throughout the project, with the customer having the responsibility of providing feedback on these deliveries. For the developers to embrace automated testing, a shift in focus is needed, from functional testing through the GUI to smaller unit and integration tests that will be easier to write and maintain. The other direction is addressing the fact that there is very little to continuously deliver during early stages of development, which could essentially make up half the project length. But also that there are several small projects each year. Making configuration management a support function for projects allows for standardisation and sharing the cost between all the projects

Automated testing of Hypermedia REST applications

Testimine on oluline osa tarkvaraarenduse elutsüklis ja testidel põhinev arendamine on üks peamistest praktikatest Agile metoodikas. Tihti keskenduvad programmeerijad RESTful rakenduse loomise protsessis äriloogika testimisele ja unustavad kontrollida protokolli, mis teostab REST interaktsioone. Selles kontekstis pakutakse välja tööriist, mis automatiseerib testide genereerimist ja teostab interaktsioone RESTful rakendusega. Tööriist võtab sisendiks kasutuslood, mis on koostatud Gherkini kitsendatud versiooniga. See on domeenispetsiifiline keel käitumispõhiseks arenduseks. Kasutuslood, mis on kirjutatud selles Gherkini variandis, hõlmavad REST rakenduse poolt nõutud interaktsioone sellisel viisil, et neist on võimalik genereerida teste. Veelgi enam, tööriist genereerib samalt kasutusloolt täisfunktsionaalse pseudoteostuse.\n\rProgrammeerijad saavad kasutada neid pseudoteostusi kliendipoole arendamiseks, vajamata REST rakenduse tegelikku teostust. Käesolev töö tutvustab tööriista kasutust ja disainiprintsiipe ning esitab näite selle kasutamisest.Testing is one essential part of the software development lifecycle and Test Driven Development is one of the main practices in agile methodology. During the development of a RESTful web application, developers oftentimes focus only in testing the business logic and neglect testing the protocol implementing REST interactions. In this context, we propose a tool to automate the generation of test cases that exercise the interactions required by a RESTful application. The tool takes as input user stories written in restricted version of Gherkin, a widely use domain specific language for behaviour driven development. User stories written in this variant of Gherkin capture the essence of the interactions required by a REST application in a way that it is possible to derive test cases from them. Moreover, the tool derives fully functional mock implementations from the same input user story. Such mock implementations can be then used by programmers to develop the client side without requiring the actual implementation of the REST application. This document introduces the design principles and implementation of the tool and presents a study case showcasing its use

Increasing Software Reliability using Mutation Testing and Machine Learning

Mutation testing is a type of software testing proposed in the 1970s where program statements are deliberately changed to introduce simple errors so that test cases can be validated to determine if they can detect the errors. The goal of mutation testing was to reduce complex program errors by preventing the related simple errors. Test cases are executed against the mutant code to determine if one fails, detects the error and ensures the program is correct. One major issue with this type of testing was it became intensive computationally to generate and test all possible mutations for complex programs. This dissertation used machine learning for the selection of mutation operators that reduced the computational cost of testing and improved test suite effectiveness. The goals were to produce mutations that were more resistant to test cases, improve test case evaluation, validate then improve the test suite’s effectiveness, realize cost reductions by generating fewer mutations for testing and improving software reliability by detecting more errors. To accomplish these goals, experiments were conducted using sample programs to determine how well the reinforcement learning based algorithm performed with one live mutation, multiple live mutations and no live mutations. The experiments, measured by mutation score, were used to update the algorithm and improved accuracy for predictions. The performance was then evaluated on multiple processor computers. One key result from this research was the development of a reinforcement algorithm to identify mutation operator combinations that resulted in live mutants. During experimentation, the reinforcement learning algorithm identified the optimal mutation operator selections for various programs and test suite scenarios, as well as determined that by using parallel processing and multiple cores the reinforcement learning process for mutation operator selection was practical. With reinforcement learning the mutation operators utilized were reduced by 50 – 100%.In conclusion, these improvements created a ‘live’ mutation testing process that evaluated various mutation operators and generated mutants to perform real-time mutation testing while dynamically prioritizing mutation operator recommendations. This has enhanced the software developer’s ability to improve testing processes. The contributions of this paper’s research supported the shift-left testing approach, where testing is performed earlier in the software development cycle when error resolution is less costly

Improving the testing of Profit Software's insurance policy database system

Profit Software's Profit Life and Pension (PLP) is an investment insurance management system. This means that PLP handles investment insurances from the moment they are sold to when they eventually expire. For a system that handles money, it is important that it can be trusted. Therefore, testing is a required part of PLP's development. This thesis is an investigation into PLP's testing strategy. In this thesis we analyse PLP's current testing strategy to find flaws and impediments. We then offer improvement suggestions to the identified problem areas as well as suggest additions which we found could be beneficial

Software Requirements As Executable Code

This project analyzed the effectiveness of using Story Testing frameworks to create an application directly from user specifications. It did this by taking an example business application with traditional specifications and rewriting those specifications in three different Story Testing Frameworks - Cucumber, FitNesse, and JBehave. Analysis of results drew the following conclusions: 1) Story Testing can help prove a project\u27s completeness, 2) Specifications are still too technical, 3) Implementation is not overly complex, and 4) Story Testing is worth it. It proposed future research around evaluating natural languages and seeking more user-friendly ways of writing specifications in a natural language