Choosing your weapons: On sentiment analysis tools for software engineering research

Recent years have seen an increasing attention to social aspects of software engineering, including studies of emotions and sentiments experienced and expressed by the software developers. Most of these studies reuse existing sentiment analysis tools such as SentiStrength and NLTK. However, these tools have been trained on product reviews and movie reviews and, therefore, their results might not be applicable in the software engineering domain. In this paper we study whether the sentiment analysis tools agree with the sentiment recognized by human evaluators (as reported in an earlier study) as well as with each other. Furthermore, we evaluate the impact of the choice of a sentiment analysis tool on software engineering studies by conducting a simple study of differences in issue resolution times for positive, negative and neutral texts. We repeat the study for seven datasets (issue trackers and STACK OVERFLOW questions) and different sentiment analysis tools and observe that the disagreement between the tools can lead to contradictory conclusions

Exploring Behaviours of RESTful APIs in an Industrial Setting

A common way of exposing functionality in contemporary systems is by providing a Web-API based on the REST API architectural guidelines. To describe REST APIs, the industry standard is currently OpenAPI-specifications. Test generation and fuzzing methods targeting OpenAPI-described REST APIs have been a very active research area in recent years. An open research challenge is to aid users in better understanding their API, in addition to finding faults and to cover all the code. In this paper, we address this challenge by proposing a set of behavioural properties, common to REST APIs, which are used to generate examples of behaviours that these APIs exhibit. These examples can be used both (i) to further the understanding of the API and (ii) as a source of automatic test cases. Our evaluation shows that our approach can generate examples deemed relevant for understanding the system and for a source of test generation by practitioners. In addition, we show that basing test generation on behavioural properties provides tests that are less dependent on the state of the system, while at the same time yielding a similar code coverage as state-of-the-art methods in REST API fuzzing in a given time limit

On negative results when using sentiment analysis tools for software engineering research

Recent years have seen an increasing attention to social aspects of software engineering, including studies of emotions and sentiments experienced and expressed by the software developers. Most of these studies reuse existing sentiment analysis tools such as SentiStrength and NLTK. However, these tools have been trained on product reviews and movie reviews and, therefore, their results might not be applicable in the software engineering domain. In this paper we study whether the sentiment analysis tools agree with the sentiment recognized by human evaluators (as reported in an earlier study) as well as with each other. Furthermore, we evaluate the impact of the choice of a sentiment analysis tool on software engineering studies by conducting a simple study of differences in issue resolution times for positive, negative and neutral texts. We repeat the study for seven datasets (issue trackers and Stack Overflow questions) and different sentiment analysis tools and observe that the disagreement between the tools can lead to diverging conclusions. Finally, we perform two replications of previously published studies and observe that the results of those studies cannot be confirmed when a different sentiment analysis tool is used

Advancing Continuous Model-Based Development in Industry

For the development of complex software systems, two prominent paradigms have become popular in the industry: model-based development and agile software development. Model-based development holds the promise of improving the productivity of software development through abstraction, by focusing on the problem domain and capturing it in models. Agile methods are sometimes seen as conflicting with model-based development due to their favored short development cycles as opposed to the apparent longer development phases in model-based development. In this licentiate thesis, we explore how development can benefit from combining the two practices successfully into continuous model-based development. We present four papers studying this topic in the industrial development of complex embedded systems. The first two papers present investigations of the current state-of-practice and specific challenges of combining the agile practice of continuous integration and model-based development. In the third and fourth paper, we focus on one of those challenges: model synchronization, i.e. the management of consistency between disparate development artifacts describing the same system. We propose a lightweight approach that notifies developers of introductions of inconsistency between different models. Lastly, we consider a second dimension of the management of different development artifacts: variability. We provide support for alleviating manual tasks in maintaining consistency across variants of models in a product line created to describe system variants

Lightweight consistency checking for advancing continuous model-based development in industry

For the development of modern software-intensive systems, a large number of development artifacts are created and maintained to design and implement their intended structure and behavior. In this thesis, we consider continuous model-based development settings in which models and other development artifacts are developed incrementally, in short development cycles. Inconsistencies between these development artifacts occur continuously throughout the system's development and maintenance, due to the incremental nature of the development of these artifacts. However, when not known and never resolved, such inconsistencies may ultimately cause failures in developed systems. Therefore, consistency checking across development artifacts is often desirable. Consistency checking approaches face challenges such as adoption in existing engineering settings, the potential informality and incompleteness of models and other development artifacts, as well as the need for shorter development cycles. We show a tight coupling between introducing short development cycles and the need for increased support for consistency checking across models and other development artifacts. From a surveyed set of industrial settings, we identify the need for lightweight consistency-checking approaches. Four aspects that make a consistency checking approach lightweight are formulated on the basis of the surveyed industrial settings. We then study specific challenges to developing consistency checks in four concrete industrial settings. We summarize our experiences in these settings in reference schemas showing the steps of creation and adoption of lightweight consistency checks from various starting points. In summary, this thesis presents contributions to lightweight consistency checking in industrial settings and ultimately aims to facilitate the adoption of continuous model-based development

Lightweight consistency checking for advancing continuous model-based development in industry

For the development of modern software-intensive systems, a large number of development artifacts are created and maintained to design and implement their intended structure and behavior. In this thesis, we consider continuous model-based development settings in which models and other development artifacts are developed incrementally, in short development cycles. Inconsistencies between these development artifacts occur continuously throughout the system's development and maintenance, due to the incremental nature of the development of these artifacts. However, when not known and never resolved, such inconsistencies may ultimately cause failures in developed systems. Therefore, consistency checking across development artifacts is often desirable. Consistency checking approaches face challenges such as adoption in existing engineering settings, the potential informality and incompleteness of models and other development artifacts, as well as the need for shorter development cycles. We show a tight coupling between introducing short development cycles and the need for increased support for consistency checking across models and other development artifacts. From a surveyed set of industrial settings, we identify the need for lightweight consistency-checking approaches. Four aspects that make a consistency checking approach lightweight are formulated on the basis of the surveyed industrial settings. We then study specific challenges to developing consistency checks in four concrete industrial settings. We summarize our experiences in these settings in reference schemas showing the steps of creation and adoption of lightweight consistency checks from various starting points. In summary, this thesis presents contributions to lightweight consistency checking in industrial settings and ultimately aims to facilitate the adoption of continuous model-based development

Impediments to Introducing Continuous Integration for Model-Based Development in Industry

Model-based development and continuous integration each separately are methods to improve the productivity of development of complex modern software systems. We investigate industrial adoption of these two phenomena in combination, i.e., applying continuous integration practices in model-based development projects. Through semi-structured interviews, eleven engineers at three companies with different modelling practices share their views on perceived and experienced impediments to this adoption. We find some cases in which this introduction is undesired and expected to not be beneficial. For other cases, we find and categorize several impediments and discuss how they are dealt with in industrial practice. Model synchronization and tool interoperability are found the most challenging to overcome and the ways in which they are circumvented in g to overcome and the ways in which they are circumvented in practice are detrimental for introducing continuous integration

Polyglot AST: Towards Enabling Polyglot Code Analysis

International audienceToday, a plethora of programming languages exists, each better suited for a particular concern. For example, Python is suited for data analysis but not web development, whereas JavaScript is the inverse. As software complexity grows and needs to address multiple concerns, different programming languages are often used in combination, despite the burden of bridging them (e.g., using Java Native Interface). Polyglot programming emerged as a solution allowing the seamless mixing of multiple programming languages. GraalVM and PolyNote are examples of runtimes allowing polyglot programming. However, there is a striking lack of support at design time for building and analyzing polyglot code. To the best of our knowledge, there is no uniform language-agnostic way of reasoning over multiple languages to provide seamless code analysis, since each language comes with its own form of Abstract Syntax Trees (AST). In this paper, we present an approach to build a uniform yet polyglot AST over polyglot code, so that it is easier to perform global analysis. We first motivate this challenge and identify the main requirements for building a polyglot AST. We then propose a proof of concept implementation of our solutions on GraalVM's polyglot API. On top of the polyglot AST, we demonstrate the ability to implement several polyglot-specific analysis services, namely auto-completion, consistency checking, type inference, and rename refactoring. Our evaluation on three polyglot projects taken from GitHub, and involving JavaScript and Python code, shows that we can build a polyglot AST without significant overhead. We also demonstrate the usefulness of the polyglot analysis services through the provided automation, as well as their scalability

Polyglot AST: Towards Enabling Polyglot Code Analysis

International audienceToday, a plethora of programming languages exists, each better suited for a particular concern. For example, Python is suited for data analysis but not web development, whereas JavaScript is the inverse. As software complexity grows and needs to address multiple concerns, different programming languages are often used in combination, despite the burden of bridging them (e.g., using Java Native Interface). Polyglot programming emerged as a solution allowing the seamless mixing of multiple programming languages. GraalVM and PolyNote are examples of runtimes allowing polyglot programming. However, there is a striking lack of support at design time for building and analyzing polyglot code. To the best of our knowledge, there is no uniform language-agnostic way of reasoning over multiple languages to provide seamless code analysis, since each language comes with its own form of Abstract Syntax Trees (AST). In this paper, we present an approach to build a uniform yet polyglot AST over polyglot code, so that it is easier to perform global analysis. We first motivate this challenge and identify the main requirements for building a polyglot AST. We then propose a proof of concept implementation of our solutions on GraalVM's polyglot API. On top of the polyglot AST, we demonstrate the ability to implement several polyglot-specific analysis services, namely auto-completion, consistency checking, type inference, and rename refactoring. Our evaluation on three polyglot projects taken from GitHub, and involving JavaScript and Python code, shows that we can build a polyglot AST without significant overhead. We also demonstrate the usefulness of the polyglot analysis services through the provided automation, as well as their scalability

Polyglot AST: Towards Enabling Polyglot Code Analysis

International audienceToday, a plethora of programming languages exists, each better suited for a particular concern. For example, Python is suited for data analysis but not web development, whereas JavaScript is the inverse. As software complexity grows and needs to address multiple concerns, different programming languages are often used in combination, despite the burden of bridging them (e.g., using Java Native Interface). Polyglot programming emerged as a solution allowing the seamless mixing of multiple programming languages. GraalVM and PolyNote are examples of runtimes allowing polyglot programming. However, there is a striking lack of support at design time for building and analyzing polyglot code. To the best of our knowledge, there is no uniform language-agnostic way of reasoning over multiple languages to provide seamless code analysis, since each language comes with its own form of Abstract Syntax Trees (AST). In this paper, we present an approach to build a uniform yet polyglot AST over polyglot code, so that it is easier to perform global analysis. We first motivate this challenge and identify the main requirements for building a polyglot AST. We then propose a proof of concept implementation of our solutions on GraalVM's polyglot API. On top of the polyglot AST, we demonstrate the ability to implement several polyglot-specific analysis services, namely auto-completion, consistency checking, type inference, and rename refactoring. Our evaluation on three polyglot projects taken from GitHub, and involving JavaScript and Python code, shows that we can build a polyglot AST without significant overhead. We also demonstrate the usefulness of the polyglot analysis services through the provided automation, as well as their scalability