EMPIRICAL ASSESSMENT OF THE IMPACT OF USING AUTOMATIC STATIC ANALYSIS ON CODE QUALITY

Automatic static analysis (ASA) tools analyze the source or compiled code looking for violations of recommended programming practices (called issues) that might cause faults or might degrade some dimensions of software quality. Antonio Vetro' has focused his PhD in studying how applying ASA impacts software quality, taking as reference point the different quality dimensions specified by the standard ISO/IEC 25010. The epistemological approach he used is that one of empirical software engineering. During his three years PhD, he's been conducting experiments and case studies on three main areas: Functionality/Reliability, Performance and Maintainability. He empirically proved that specific ASA issues had impact on these quality characteristics in the contexts under study: thus, removing them from the code resulted in a quality improvement. Vetro' has also investigated and proposed new research directions for this field: using ASA to improve software energy efficiency and to detect the problems deriving from the interaction of multiple languages. The contribution is enriched with the final recommendation of a generalized process for researchers and practitioners with a twofold goal: improve software quality through ASA and create a body of knowledge on the impact of using ASA on specific software quality dimensions, based on empirical evidence. This thesis represents a first step towards this goa

Modularity and Architecture of PLC-based Software for Automated Production Systems: An analysis in industrial companies

Adaptive and flexible production systems require modular and reusable software especially considering their long term life cycle of up to 50 years. SWMAT4aPS, an approach to measure Software Maturity for automated Production Systems is introduced. The approach identifies weaknesses and strengths of various companie's solutions for modularity of software in the design of automated Production Systems (aPS). At first, a self assessed questionnaire is used to evaluate a large number of companies concerning their software maturity. Secondly, we analyze PLC code, architectural levels, workflows and abilities to configure code automatically out of engineering information in four selected companies. In this paper, the questionnaire results from 16 German world leading companies in machine and plant manufacturing and four case studies validating the results from the detailed analyses are introduced to prove the applicability of the approach and give a survey of the state of the art in industry

Supporting web programming assignment assessment with test automation and RPA

Automated software solutions to support and assist in assessment of student implemented applications are not a rarity, but often need to be custom engineered to fit a specific learning environment or a course. When such a system can be fielded in use properly, it has a tremendous potential to lighten the workload of course personnel by automating the repetitive manual tasks and testing student submissions against assignment requirements. Additionally, these support systems are often able to shorten the feedback loop which is seen to have a direct impact on student learning. In this thesis test automation and robotic process automation are researched to discover how they can be used to support web programming assignment assessment. The background on software testing, automation and feedback related pedagogy are researched mainly by the methods of literature review and expert interview. A third methodology – design science – is then applied for the purpose of verifying and extending the learnt theory in an empirical manner. A research artifact is created in the form of a prototype capable of supporting in assessment tasks. Performance of the prototype is measured by recording set execution metrics while assessing anonymized case study student submissions from a web development course arranged by University of Turku: DTEK2040 Web and Mobile Programming. Thesis concludes that to support assessment through test automation is to focus on unit and system level testing of functionalities while assuming the exact implementation at code level cannot be fully known. Suggestion is made that relying on assignment descriptions as basis for test design is not enough, but rather requirements engineering should be done together with course personnel to take advantage of their experience in what sort of errors are to be tolerated in student submissions. Thesis also concludes that automation can perform interaction with student submissions, file manipulation, record keeping and tracking tasks at a satisfactory level. The potential to shorten the feedback loop and summarizing quantitative feedback for the student is recognized, however, to build an automated system to identify, gather and summarize formative, pedagogically more valuable feedback was noted to be out of scope for this thesis and suggested as future work to possibly extend the prototype with.Automatisoidut ohjelmistoratkaisut, jotka tukevat ja avustavat opiskelijoiden toteuttamien sovellusten arvioinnissa, eivät ole harvinaisia, mutta ne useimmiten joudutaan rakentamaan tiettyyn oppimisympäristöön tai opintosisältöön sopiviksi. Tällaiset järjestelmät omaavat kuitenkin valtavan potentiaalin keventää kurssihenkilöstön työtaakkaa automatisoimalla toistuvia manuaalisia työtehtäviä ja automaatiotestaamalla opiskelijoiden palauttamia tuotoksia asetettuja tehtävävaatimuksia vastaan. Järjestelmät johtavat varsin usein myös opiskelijan näkökulmasta nopeampaan palautesykliin, jolla kyetään todeta olevan suora vaikutus oppimiseen. Tässä opinnäytetyössä tutkitaan testiautomaatiota sekä robottiprosessiautomaatiota pyrkimyksenä selvittää kuinka näitä teknologioita voitaisiin hyödyntää tukemaan web-ohjelmointitehtävien arviointia. Ohjelmistotestauksen, automaation ja palautteen pedagogiikan taustoja tutkitaan pääasiassa kirjallisuuskatsauksen ja asiantuntijahaastattelun menetelmin. Lisäksi sovelletaan kolmatta metodologiaa, suunnittelutiedettä, jonka tavoitteena on vahvistaa teoriaosuuden havaintoja sekä pyrkiä empiirisesti laajentamaan niitä. Suunnittelutieteen kautta tutkimusartifaktina syntyy prototyyppi, jonka suorituskykyä ja hyötyjä mitataan keräämällä dataa hyödyntäen aitoja, anonymisoituja opiskelijapalautuksia Turun yliopiston järjestämän DTEK2040: Web and Mobile Programming -kurssin tiimoilta. Opinnäytetyön johtopäätöksenä on, että arvioinnin tukeminen testiautomaation avulla on keskittymistä yksikkö- ja järjestelmätason toiminnallisuuksien testaukseen. Testaukseen on liitettävä myös oletus, että arvioitavan kohteen tarkkaa toteutusta kooditasolla ei voida täysin tuntea. Tehtäväkuvausten käyttö testitapausten suunnittelun perustana todetaan riittämättömäksi, ja vaatimussuunnittelu ehdotetaan tehtävän yhdessä kurssin henkilökunnan kanssa, jotta heidän kokemuksiaan voidaan hyödyntää yleisimpien opiskelijapalutuksissa ilmenevien virhetapausten kartoittamiseksi sekä testitapausten tarkkuuden ja arvioinnin jyrkkyyden säätämiseksi. Prosessiautomaation osalta todetaan, että automaatio kykenee suorittamaan vuorovaikutusta opiskelijoiden palautusten, tiedostojen käsittelyä, kirjanpito- ja seurantatehtäviä tyydyttävällä tasolla. Mahdollisuus palautesilmukan lyhentämiseen ja summaavan palautteen yhteenvetoon opiskelijalle tunnustetaan myös empiirisesti. Laadullisen, pedagogisesti arvokkaamman palautteen kokoaminen ja jalostaminen todettiin tämän opinnäytetyön mittakaavassa liian suureksi projektiksi ja sen empiiristä toteutusta ehdotettiin yhtenä mahdollisena jatkotutkimusaiheena

Jolie Microservices: An Experiment

Os microsserviços estão cada vez mais presentes no mundo das tecnologias de informação, por providenciarem uma nova forma construir sistemas mais escaláveis, ágeis e flexíveis. Apesar disto, estes trazem consigo o problema da complexidade de comunicação entre microsserviços, fazendo com que o sistema seja difícil de manter e de se perceber. Linguagens de programação específicas a microsserviços como Jolie entram em cena para tentar resolver este problema e simplificar a construção de sistemas com arquiteturas de microsserviços. Este trabalho fornece uma visão ampla do estado da arte da linguagem de programação Jolie onde é primeiramente detalhado o porquê de surgirem linguagens específicas a microsserviços e como a linguagem Jolie está construída de maneira a coincidir com as arquiteturas de microsserviços através de recursos nativos. Para demonstrar todas as vantagens de usar esta linguagem em comparação com as abordagens mais mainstream é pensado um experimento de desenvolvimento de um sistema de microsserviços no âmbito de uma aplicação de e-commerce. Este sistema é construído de forma igual usando duas bases tecnológicas – Jolie e Spring Boot. O Spring Boot é considerado a tecnologia mais usada para desenvolver sistemas de microsserviços sendo o candidato ideal para comparação. É pensada toda a análise e design deste experimento. Em seguida, a implementação da solução é detalhada a partir das configurações do sistema, escolhas arquitetónicas e como elas são implementadas. Componentes como API gateway, mediadores de mensagens, bases de dados, orquestração de microsserviços, e conteinerização para cada microsserviço e outros componentes do sistema. Pol último as soluções são comparadas e analisadas com base na abordagem Goals, Questions, Metrics (GQM). São analisadas relativamente a atributos de qualidade como manutenção, escalabilidade, desempenho e testabilidade. Após esta análise pode-se concluir que a solução construída com Jolie apresenta diferenças na manutenção sendo significativamente superior à solução baseada em Spring Boot e apresenta diferenças em termos de performance sendo ligeiramente inferior à solução construída com Spring Boot. O trabalho termina com a indicação das conquistas, dificuldades, ameaças à validade, possíveis trabalhos futuros e observações finais.Microservices are increasingly present in the world of information technologies, as they provide a new way to build more scalable, agile, and flexible systems. Despite this, they bring with them the problem of communication complexity between microservices, making the system difficult to maintain and understand. Microservices-specific programming languages like Jolie come into play to try to solve this problem and simplify the construction of systems with microservices architectures. This work provides a broad view of the State of Art of the Jolie programming language, where it is first detailed why microservices-specific languages emerge and how the Jolie language is built to match microservices architectures through native resources. To demonstrate all the advantages of using this language compared to more mainstream approaches, an experiment is designed to develop a microservices system within an e-commerce application. This system is built equally using two technological foundations – Jolie and Spring Boot. Spring Boot is considered the most used technology to develop microservices systems and is an ideal candidate for comparison. The entire analysis and design of this experiment are thought through. Then the implementation of the solution is detailed from system configurations, architectural choices, and how they are implemented. Components such as API gateway, message brokers, databases, microservices orchestration, and containerization for each microservice and other components of the system. Finally, the solutions are compared and analyzed based on the Goals, Questions, Metrics (GQM) approach. They are analyzed for quality attributes such as maintainability, scalability, performance, and testability. After this analysis, it can be concluded that the solution built with Jolie presents differences in maintenance being significant superior to the solution based on Spring Boot, and it presents differences in terms of performance being slightly inferior to the solution built with Spring Boot. The work ends with an indication of the achievements, difficulties, threats to validity, possible future work, and final observations

Mikropalveluiden testauskäytännöt julkisen sektorin projekteissa

Online services are constantly evolving, which makes service maintainability challenging. This has led to micro service architecture, where big applications are split into smaller services in order to improve applications' maintainability, scalability, and flexibility. However, splitting a single process application into multiple services causes the testing process to be more challenging. This Master's thesis is exploring these testing problems in a micro service context and finding practical guidance for the test implementation. Moreover, this Master's thesis focuses on public sector software projects. Public sector software projects are clearly predefined and the provider has open information about the project's needs. Thus, the project has a clear goal and known boundaries right from the beginning. The research approach for this study is an exploratory multiple case study consisting of three case projects. The data of the case projects were collected through semi-structural interviews and version history commit analysis. The results of this study present a set of successful practices and recommendations for taking testing into account during a micro service oriented agile development process. Successful testing requires monitoring of the project's maturity level to focus testing resources at the right time. Additionally, the case projects brought up practical testing guidance, such as understanding of the common testing responsibility, the importance of peer review, and the value of assigning a specific tester after the project has reached its end-to-end testing phase.Web-palvelut kehittyvät jatkuvasti, mikä vaikeuttaa palveluiden ylläpitoa. Yhtenä ratkaisuna on palvelun pilkkominen osiin mikropalveluiksi. Palvelun pilkkominen edistää palvelun ylläpitoa, skaalattavuutta ja joustavuutta. Toisaalta palvelun pilkkominen mikropalveluiksi vaikeuttaa testausprosessia. Tämä diplomityö tutkii mikropalveluiden testausprosessiin liittyviä ongelmia ja etsii käytännönläheistä ohjeistuista testien toteuttamiseen mikropalveluympäristössä. Diplomityö keskittyy julkisen sektorin mikropalveluprojekteihin, koska kaikki tässä diplomityössä käytetyt tutkimusprojektit ovat julkisen sektorin hallinnoimia. Julkisen sektorin ohjelmistoprojektit ovat selkeästi esimääriteltyjä ja projektien aineisto on avoimesti saatavilla. Tämän takia projekteilla on selkeä päämäärä ja tunnetut rajat heti projektin alussa. Tutkimusmenetelmänä käytettiin tutkivaa case study -menetelmää. Tutkimus sisälsi kolme tutkimuskohdetta. Tutkimusdata kerättiin osittain jäsennetyillä kontekstuaalisilla haastatteluilla ja ohjelmistokoodin versiohallinnan historian analyysillä. Tuloksena syntyi kokoelma hyväksi todettuja käytäntöjä ja suosituksia, jotka auttavat ottamaan testauksen huomioon mikropalvelun iteratiivisessa ohjelmistokehitysprosessissa. Suositeltaviksi testauskäytännöiksi havaittiin projektin maturiteetin tarkkaileminen, että testauksen resursointi voidaan tehdä oikeaan aikaan. Lisäksi, projekteista nousi esiin muita suosituksia, kuten kehitystiimin yhteisen testaamisvastuun ymmärtäminen, koodikatselmoinnin merkitys ja erillisen testaajan tärkeys, kun projektin maturiteetti on kasvanut riittävästi

Exploiting Similarity Patterns to Build Generic Test Case Templates for Software Product Line Testing

Ph.DDOCTOR OF PHILOSOPH

State of Refactoring Adoption: Towards Better Understanding Developer Perception of Refactoring

Context: Refactoring is the art of improving the structural design of a software system without altering its external behavior. Today, refactoring has become a well-established and disciplined software engineering practice that has attracted a significant amount of research presuming that refactoring is primarily motivated by the need to improve system structures. However, recent studies have shown that developers may incorporate refactoring strategies in other development-related activities that go beyond improving the design especially with the emerging challenges in contemporary software engineering. Unfortunately, these studies are limited to developer interviews and a reduced set of projects. Objective: We aim at exploring how developers document their refactoring activities during the software life cycle. We call such activity Self-Affirmed Refactoring (SAR), which is an indication of the developer-related refactoring events in the commit messages. After that, we propose an approach to identify whether a commit describes developer-related refactoring events, to classify them according to the refactoring common quality improvement categories. To complement this goal, we aim to reveal insights into how reviewers develop a decision about accepting or rejecting a submitted refactoring request, what makes such review challenging, and how to the efficiency of refactoring code review. Method: Our empirically driven study follows a mixture of qualitative and quantitative methods. We text mine refactoring-related documentation, then we develop a refactoring taxonomy, and automatically classify a large set of commits containing refactoring activities, and identify, among the various quality models presented in the literature, the ones that are more in-line with the developer\u27s vision of quality optimization, when they explicitly mention that they are refactoring to improve them to obtain an enhanced understanding of the motivation behind refactoring. After that, we performed an industrial case study with professional developers at Xerox to study the motivations, documentation practices, challenges, verification, and implications of refactoring activities during code review. Result: We introduced SAR taxonomy on how developers document their refactoring strategies in commit messages and proposed a SAR model to automate the detection of refactoring. Our survey with code reviewers has revealed several difficulties related to understanding the refactoring intent and implications on the functional and non-functional aspects of the software. Conclusion: Our SAR taxonomy and model, can work in conjunction with refactoring detectors, to report any early inconsistency between refactoring types and their documentation and can serve as a solid background for various empirical investigations. In light of our findings of the industrial case study, we recommended a procedure to properly document refactoring activities, as part of our survey feedback

Self-Organizing Software Architectures

Looking at engineering productivity is a source for improving the state of software engineering. We present two approaches to improve productivity: bottom-up modeling and self-configuring software components. Productivity, as measured in the ability to produce correctly working software features using limited resources is improved by performing less wasteful activities and by concentrating on the required activities to build sustainable software development organizations. Bottom-up modeling is a way to combine improved productivity with agile software engineering. Instead of focusing on tools and up-front planning, the models used emerge, as the requirements to the product are unveiled during a project. The idea is to build the modeling formalisms strong enough to be employed in code generation and as runtime models. This brings the benefits of model-driven engineering to agile projects, where the benefits have been rare. Self-configuring components are a development of bottom-up modeling. The notion of a source model is extended to incorporate the software entities themselves. Using computational reflection and introspection, dependent components of the software can be automatically updated to reflect changes in the dependence. This improves maintainability, thus making software changes faster. The thesis contains a number of case studies explaining the ways of applying the presented techniques. In addition to constructing the case studies, an empirical validation with test subjects is presented to show the usefulness of the techniques.Itseorganisoituvat ohjelmistoarkkitehtuurit Ohjelmistokehityksen tuottavuus on monen ohjelmistokehitysorganisaation huolenaihe. Erityisesti ylläpitovaiheessa ohjelmistojen heikko muokattavuus tuottaa turhia kustannuksia ja pettymyksiä asiakassuhteissa, kun vaikeasti muokattavaan ohjelmistoon tulisi tehdä muutoksia. Tässä työssä esitetään kaksi menetelmää ohjelmistojen muokattavuuden parantamiseksi: kokoava mallinnuskielten käyttäminen sekä itseorganisoituvat ohjelmistokomponentit. Mallipohjaisessa ohjelmistotuotannossa ohjelmistoille kehitetään soveltuvat mallinnuskielet ja -työkalut, joiden pohjalta kehitettävä ohjelmisto voidaan automaattisesti tuottaa. Uuden mallinnuskielen kehittäminen ja sitä tukevan välineistön rakentaminen on kuitenkin aikaaviepää ja vaikeaa. Vaarana on, että kehitetty kieli on valmistuessaan vanhentunut. Niin kutsutuissa ketterissä ohjelmistomenetelmissä yritetään välttää perinteisten, suunittelupainotteisten kehitysmenetelmien tuottamia sudenkuoppia. Liiallinen ketteryys voi kuitenkin kostautua heikkona tuottavuutena, kun kehitysväen kaikki aika kuluu näppäryysharjoituksiin varsinaisen tuottavan työn sijaan. Kokoava mallipohjainen tuotanto keskittyy kehittämään vain riittävän hyviä malleja, joiden perusteella voidaan yhdistää mallipohjaisen ohjelmistotuotannon ja ketterien prosessimallien tuomat edut. Ulkoisten, erikseen kehiteltyjen mallikielten lisäksi työssä esitellään ajatus ohjelmakoodin itsensä käyttämisestä mallipohjaisen ohjelmistotuotannon työkaluna. Näin syntyy itseorganisoituva ohjelmistoarkkitehtuuri. Tällä tavoin kehitystyön tuottavuus paranee, sillä ohjelmakoodin sisäisten riippuvuuksien määrä laskee, ja näin ollen muokkausten tekeminen on helpompaa. Työssä esitellään tapaustutkimuksia ohjelmakoodiin perustuvasta mallipohjaisen ohjelmistotuotannon ohjelmistokehyksistä sekä empiirinen validointi itseorganisoituvuuden hyödyllisyydestä tuottavuusnäkökulmasta katsoen