What is software quality for AI engineers?:towards a thinning of the fog

Abstract It is often overseen that AI-enabled systems are also software systems and therefore rely on software quality assurance (SQA). Thus, the goal of this study is to investigate the software quality assurance strategies adopted during the development, integration, and maintenance of AI/ML components and code. We conducted semi-structured interviews with representatives of ten Austrian SMEs that develop AI-enabled systems. A qualitative analysis of the interview data identified 12 issues in the development of AI/ML components. Furthermore, we identified when quality issues arise in AI/ML components and how they are detected. The results of this study should guide future work on software quality assurance processes and techniques for AI/ML components

How long do junior developers take to remove technical debt items?

Background. Software engineering is one of the engineering fields with the highest inflow of junior engineers. Tools that utilize source code analysis to provide feedback on internal software quality, i.e. Technical Debt (TD), are valuable to junior developers who can learn and improve their coding skills with minimal consultations with senior colleagues. Objective. We aim at understating which SonarQube TD items junior developers prioritize during the refactoring and how long they take to refactor them. Method. We designed a case study with replicated design and we conducted it with 185 junior developers in two countries, that developed 23 projects with different programming languages and architectures. Results. Junior developers focus homogeneously on different types of TD items. Moreover, they can refactor items in a fraction of the estimated time, never spending more than 50% of the time estimated by SonarQube. Conclusion. Junior Developers appreciate the usage of SonarQube and considered as a useful tool. Companies might ask junior developers to quickly clean their code.acceptedVersionPeer reviewe

Keyword-labeled self-admitted technical debt and static code analysis have significant relationship but limited overlap

Abstract Technical debt presents sub-optimal choices made in development, which are beneficial in the short term but not in the long run. Consciously admitted debt, which is marked with a keyword, e.g., TODO, is called keyword-labeled self-admitted technical debt (KL-SATD). KL-SATD can lead to adverse effects in software development, e.g., to a rise in complexity within the developed software. We investigated the relationship between KL-SATD from source code comments and reports from the highly popular industrial program analysis tool SonarQube. The goal was to find which SonarQube metrics and issues are related to KL-SATD introduction and removal and how many KL-SATD in the context of an issue addresses that issue. We performed a study with 33 software repositories. We analyzed the changes in SonarQube reports (sqale index, reliability and security remediation metrics, and SonarQube issues) and the relationship to KL-SATD addition and removal with mixed model analysis. We manually annotated a sample to investigate how many KL-SATD comments are in the context of SonarQube issues and how many address them directly. KL-SATD is associated with a reduction in code maintainability measured with SonarQube’s sqale index. KL-SATD removal is associated with an increase in code maintainability (sqale index) and reliability measured with SonarQube’s reliability remediation effort. The introduction and removal of KL-SATD have a predominantly relationship with code smells, and not with vulnerabilities and bugs. Manual annotation revealed that 36% of KL-SATD comments are in the context of a SonarQube issue, but only 15% of the comment address an issue. This means that despite of statistical relationship between KL-SATD comments and SonarQube reports there is a large set of KL-SATD comments that are in areas that Sonarqube reports as clean or free of maintainability issues. KL-SATD introduction and removal are connected mainly to code smells, connecting them to maintainability rather than reliability or security. This is reinforced by the relationship with the sqale index, as well as the dominance of code smells in SonarQube issues. Many KL-SATD issues have characteristics going beyond static analysis tools and require future studies extending the capabilities of the current tools. As KL-SATD comments and SonarQube reports appear to have limited overlap, it suggests that they are complementary and both are needed for getting a comprehensive view coverage of code maintainability. The study also presents rules violations developers should be aware of regarding KL-SATD introduction and removal

Preliminary insights to enable automation of the software development process in software StartUps:an investigation study from the use of artificial intelligence and machine learning

Abstract Artificial Intelligence (AI) and Machine Learning (ML) tools and techniques have increasingly effectively supported Software Engineering (SE) tasks, whether for requirements classification, software refactoring, defect prediction, and many others. In the context of software StartUps, where innovative and scalable software products are developed, dealing with the pressure of fast delivery of a working solution becomes a challenging factor. We aim to investigate AI and ML techniques used by SE practitioners and entrepreneurs to support their Software Development Processes (SDP) and thus enable their use by software StartUps. We seek to identify this information through the application of an online Survey instrument, mainly disseminated in Brazil and Finland. This preliminary study provides insights that can support improving the SDP in StartUps

Anomaly detection in cloud-native systems

Abstract Companies develop cloud-native systems deployed on public and private clouds. Since private clouds have limited resources, the systems should run efficiently by keeping performance related anomalies under control. The goal of this work is to understand whether a set of five performance-related KPIs depends on the metrics collected at runtime by Kafka, Zookeeper, and other tools (168 different metrics). We considered four weeks worth of runtime data collected from a system running in production. We trained eight Machine Learning algorithms on three weeks worth of data and tested them on one week’s worth of data to compare their prediction accuracy and their training and testing time. It is possible to detect performance-related anomalies with a very high level of accuracy (higher than 95% AUC) and with very limited training time (between 8 and 17 minutes). Machine Learning algorithms can help to identify runtime anomalies and to detect them efficiently. Future work will include the identification of a proactive approach to recognize the root cause of the anomalies and to prevent them as early as possible

ASPA:a static analyser to support learning and continuous feedback on programming courses:an empirical validation

Abstract For decades there have been arguments how to teach programming in the basic courses. Supportive intervention methods to improve students’ learning and methods to improve assessment process have been widely studied. There are various successful methods to each topic separately, but only a few of them fit for both. In this work, we aimed at validating ASPA a static analyser tool that supports learning and continuous feedback on programming courses. For this purpose, we designed and conduct an empirical study among 236 students enrolled in the basic programming course, that voluntary adopted the tools during the project development activities. We first profiled the students, then, evaluated the attitude toward using ASPA, the perceived ease of use, and the perceived usefulness. Results showed that ASPA is a good helper for the entire course and especially the student’s programming assignments, and it also helps to improve the students’ grades

Regularity or anomaly?:on the use of anomaly detection for fine-grained JIT defect prediction

Abstract Fine-grained just-in-time defect prediction aims at identifying likely defective files within new commits. Popular techniques are based on supervised learning, where machine learning algorithms are fed with historical data. One of the limitations of these techniques is concerned with the use of imbalanced data that only contain a few defective samples to enable a proper learning phase. To overcome this problem, recent work has shown that anomaly detection can be used as an alternative. With our study, we aim at assessing how anomaly detection can be employed for the problem of fine-grained just-in-time defect prediction. We conduct an empirical investigation on 32 open-source projects, designing and evaluating three anomaly detection methods for fine-grained just-in-time defect prediction. Our results do not show significant advantages that justify the benefit of anomaly detection over machine learning approaches

Lessons learned on communication channels and practices in agile software development

Abstract Communication plays an important role in Agile Software Development (ASD). In each ASD practice (e.g., stand-up or retrospective meetings), different communication practices and channels are adopted by different companies. Several works have analyzed the impact of communication channels and practices. However, there are no secondary studies summarizing their impact on ASD. This study presents a Systematic Mapping Study (SMS) that aggregates, summarizes, and discusses the results of 25 relevant primary studies concerning the impact of communication channels and practices in ASD. We followed the well-known systematic mapping methodology in software engineering and analyzed empirical studies published before the end of June 2018. The results of our study have yielded several strategies that can be adopted by practitioners. Communication practices are context dependent. In the case of a distributed team, blended usage of rich-media communication tools, such as shared mind-map tools, videoconferencing, and promoting the exchange of team members between teams, is beneficial. In conclusion, communication can be expensive if teams do not apply the right strategies. Future research direction is to understand how to maximize product quality while reducing communication cost and how to identify the most beneficial communication strategy for the different stages of ASD

Identification and measurement of Requirements Technical Debt in software development:a systematic literature review

Abstract Context: Requirements Technical Debt are related to the distance between the ideal value of the specification and the actual implementation of the system, which are consequences of strategic decisions for immediate gains, or unintended changes in context. To ensure the evolution of the software, it is necessary to to manage TD. Identification and measurement are the first two stages of the management process; however, they are poorly explored in academic research in requirements engineering. Objective: We aimed to investigating which evidence helps to strengthen the TD requirements management process, including identification and measurement. Method: We conducted a Systematic Literature Review through manual and automatic searches considering 7499 studies from 2010 to 2020, and including 66 primary studies. Results: We identified some causes related to Technical Debt requirements, existing strategies to help in the identification and measurement, and metrics to support the measurement stage. Conclusion: The studies on Requirements Technical Debt are still preliminary, especially regarding management software. Yet, however, the interpersonal aspects that prove difficult in the implementation of such activities are not sufficiently addressed. Finally, the provision of metrics to help measure technical debt is part of the contribution of this search, providing insights into the application in its requirements context