Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research

An Empirical Study on Decision making for Quality Requirements

[Context] Quality requirements are important for product success yet often handled poorly. The problems with scope decision lead to delayed handling and an unbalanced scope. [Objective] This study characterizes the scope decision process to understand influencing factors and properties affecting the scope decision of quality requirements. [Method] We studied one company's scope decision process over a period of five years. We analyzed the decisions artifacts and interviewed experienced engineers involved in the scope decision process. [Results] Features addressing quality aspects explicitly are a minor part (4.41%) of all features handled. The phase of the product line seems to influence the prevalence and acceptance rate of quality features. Lastly, relying on external stakeholders and upfront analysis seems to lead to long lead-times and an insufficient quality requirements scope. [Conclusions] There is a need to make quality mode explicit in the scope decision process. We propose a scope decision process at a strategic level and a tactical level. The former to address long-term planning and the latter to cater for a speedy process. Furthermore, we believe it is key to balance the stakeholder input with feedback from usage and market in a more direct way than through a long plan-driven process

Mining app reviews to support software engineering

The thesis studies how mining app reviews can support software engineering. App reviews —short user reviews of an app in app stores— provide a potentially rich source of information to help software development teams maintain and evolve their products. Exploiting this information is however difficult due to the large number of reviews and the difficulty in extracting useful actionable information from short informal texts. A variety of app review mining techniques have been proposed to classify reviews and to extract information such as feature requests, bug descriptions, and user sentiments but the usefulness of these techniques in practice is still unknown. Research in this area has grown rapidly, resulting in a large number of scientific publications (at least 182 between 2010 and 2020) but nearly no independent evaluation and description of how diverse techniques fit together to support specific software engineering tasks have been performed so far. The thesis presents a series of contributions to address these limitations. We first report the findings of a systematic literature review in app review mining exposing the breadth and limitations of research in this area. Using findings from the literature review, we then present a reference model that relates features of app review mining tools to specific software engineering tasks supporting requirements engineering, software maintenance and evolution. We then present two additional contributions extending previous evaluations of app review mining techniques. We present a novel independent evaluation of opinion mining techniques using an annotated dataset created for our experiment. Our evaluation finds lower effectiveness than initially reported by the techniques authors. A final part of the thesis, evaluates approaches in searching for app reviews pertinent to a particular feature. The findings show a general purpose search technique is more effective than the state-of-the-art purpose-built app review mining techniques; and suggest their usefulness for requirements elicitation. Overall, the thesis contributes to improving the empirical evaluation of app review mining techniques and their application in software engineering practice. Researchers and developers of future app mining tools will benefit from the novel reference model, detailed experiments designs, and publicly available datasets presented in the thesis

Safety Engineering with COTS components

Safety-critical systems are becoming more widespread, complex and reliant on software. Increasingly they are engineered through Commercial Off The Shelf (COTS) (Commercial Off The Shelf) components to alleviate the spiralling costs and development time, often in the context of complex supply chains. A parallel increased concern for safety has resulted in a variety of safety standards, with a growing consensus that a safety life cycle is needed which is fully integrated with the design and development life cycle, to ensure that safety has appropriate influence on the design decisions as system development progresses. In this article we explore the application of an integrated approach to safety engineering in which assurance drives the engineering process. The paper re- ports on the outcome of a case study on a live industrial project with a view to evaluate: its suitability for application in a real-world safety engineering setting; its benefits and limitations in counteracting some of the difficulties of safety en- gineering with COTS components across supply chains; and, its effectiveness in generating evidence which can contribute directly to the construction of safety cases

Revisiting Sentiment Analysis for Software Engineering in the Era of Large Language Models

Software development is an inherently collaborative process, where various stakeholders frequently express their opinions and emotions across diverse platforms. Recognizing the sentiments conveyed in these interactions is crucial for the effective development and ongoing maintenance of software systems. Over the years, many tools have been proposed to aid in sentiment analysis, but accurately identifying the sentiments expressed in software engineering datasets remains challenging. Although fine-tuned smaller large language models (sLLMs) have shown potential in handling software engineering tasks, they struggle with the shortage of labeled data. With the emergence of bigger large language models (bLLMs), it is pertinent to investigate whether they can handle this challenge in the context of sentiment analysis for software engineering. In this work, we undertake a comprehensive empirical study using five established datasets. We assess the performance of three open-source bLLMs in both zero-shot and few-shot scenarios. Additionally, we compare them with fine-tuned sLLMs. Our experimental findings demonstrate that bLLMs exhibit state-of-the-art performance on datasets marked by limited training data and imbalanced distributions. bLLMs can also achieve excellent performance under a zero-shot setting. However, when ample training data is available or the dataset exhibits a more balanced distribution, fine-tuned sLLMs can still achieve superior results.Comment: Submitted to TOSE

Reducing the effort for systematic reviews in software engineering

Context: Systematic Reviews (SRs) are means for collecting and synthesizing evidence from the identification and analysis of relevant studies from multiple sources. To this aim, they use a well-defined methodology meant to mitigate the risks of biases and ensure repeatability for later updates. SRs, however, involve significant effort. Goal: The goal of this paper is to introduce a novel methodology that reduces the amount of manual tedious tasks involved in SRs while taking advantage of the value provided by human expertise. Method: Starting from current methodologies for SRs, we replaced the steps of keywording and data extraction with an automatic methodology for generating a domain ontology and classifying the primary studies. This methodology has been applied in the Software Engineering sub-area of Software Architecture and evaluated by human annotators. Results: The result is a novel Expert-Driven Automatic Methodology, EDAM, for assisting researchers in performing SRs. EDAM combines ontology-learning techniques and semantic technologies with the human-in-the-loop. The first (thanks to automation) fosters scalability, objectivity, reproducibility and granularity of the studies; the second allows tailoring to the specific focus of the study at hand and knowledge reuse from domain experts. We evaluated EDAM on the field of Software Architecture against six senior researchers. As a result, we found that the performance of the senior researchers in classifying papers was not statistically significantly different from EDAM. Conclusions: Thanks to automation of the less-creative steps in SRs, our methodology allows researchers to skip the tedious tasks of keywording and manually classifying primary studies, thus freeing effort for the analysis and the discussion

On the engineering of crucial software

The various aspects of the conventional software development cycle are examined. This cycle was the basis of the augmented approach contained in the original grant proposal. This cycle was found inadequate for crucial software development, and the justification for this opinion is presented. Several possible enhancements to the conventional software cycle are discussed. Software fault tolerance, a possible enhancement of major importance, is discussed separately. Formal verification using mathematical proof is considered. Automatic programming is a radical alternative to the conventional cycle and is discussed. Recommendations for a comprehensive approach are presented, and various experiments which could be conducted in AIRLAB are described

Conceptions of Refactoring: An Investigation of Stack Overflow Posts

Refactoring is a common activity in software development. Developers make changes in the code in order to achieve a desired effect such as better performance, conformance to new business rules, or the removal of code anti-patterns such as code smells. However, refactoring operations often fail to achieve the results that are expected of them. There have been many studies conducted to assess their impact and effectiveness in different scenarios, but they have not returned consistent results. At times, studies have even shown decreasing and increasing in code quality from the same operation. This study investigated whether a common set of well-known refactoring actions defined by Fowler aligned with the terms and actions discussed by developers in the process of refactoring. It obtained these scenarios by looking into Stack Overflow posts discussing refactoring to see what actions and goals users were pursing. We hypothesis that there may be discrepancies between the actions discussed and the context (i.e., web-development vs. database) or technology (i.e., Java, PHP, Python) where different refactorings are more easily implemented. It was found that the number of identifiable refactoring scenarios increases when the scenario contains matching components described in the methods (i.e., Extract Method, Rename Class, etc.). Additionally, developers often only have a vague conception of what actions they believe will achieve the goals of their refactor. The conclusion drawn from these results is that refactoring suggestions must be aware of the context they will be applied to in order to align with the developer\u27s expectations. These methods also must be explicitly aligned with specific quality improvements or changes in order for developers to feel more comfortable using them to communicate their refactoring intentions

The strategic use of patents and its implications for enterprise and competition policies

This report was commissioned as a study into the strategic use of patents. In the course of its case investigations and legislative reviews the European Commission became aware of changes in the use of intellectual property, in particular the use of patents. It was noted that firms’ uses of intellectual property are becoming increasingly strategic. This raised concerns about the implications of firms’ patenting behaviour for enterprise and competition policy. The following report contains a comprehensive review of patenting behaviour, the extent to which patenting is becoming more strategic and the implications this has for competition and enterprise policies

Effective social productivity measurements during software development: an empirical study

Much of contemporary scientific discussion regarding factors that influence software development productivity is undertaken in various domains where there is an insufficient empirical basis for exploring socio-technical factors of productivity that are specific to a software development organization. The purpose of the study is to characterize the multidimensional nature of software development productivity and its social aspects as a set of latent constructs (i.e. variables that are not directly observed) for a medium-sized software company. To this end, we designed an exploratory in-depth field study based on the hypothesized productivity constructs, which were modeled by a set of factors identified from literature reviews, and later refined by industrial focus groups. In order to demonstrate the applicability of our approach, we conducted confirmatory factor analysis with the data attained from a questionnaire with 216 participants. To investigate factors of influence further, we analyzed the impact of selected team-based variables over the latent constructs of productivity. Taken together, our findings confirm that such an approach can be used to explore the quantifiable influence of socio-technical factors that would affect productivity of a particular software development organization. Ultimately, the resulting model provides guidance to explore the comparative importance of a set of firm-specific factors that may help to improve the productivity of the organization