An Investigation into the Imposed Cognitive Load of Static & Dynamic Type Systems on Programmers

Static and dynamic type systems have long been a point of contention in the programming language wars. Yet, for many years, arguments on either side were drawn from personal experience and not empirical evidence. A challenge for researchers is that the usability of language constructs is difficult to quantify, especially since usability can be interpreted in many ways. By one definition, language usability can be measured in terms of the level of cognitive load imposed on a developer. This can be done through questionnaires, but ultimately user responses are subject to bias. In recent years, eye-tracking has been shown to be an effective means of measuring cognitive load via direct physiological measures. Towards the goal of measuring type system usability, we present a user study in which participants completed programming tasks in Java and Groovy. This thesis explored the use of the Index of Cognitive Activity (ICA) as a cognitive load measurement tool and considered novices and experts separately in the analysis. We found ICA to be an ineffective means of measuring type system usability and we cannot say conclusively whether it can be generally applied to programming tasks. Despite this, our results contradict previous studies as we found type system did not affect success rate, task completion time, or perceived task difficulty

On The Human Factors Impact of Polyglot Programming on Programmer Productivity

Polyglot programming is a common practice in modern software development. This practice is often considered useful to create software by allowing developers to use whichever language they consider most well suited for the different parts of their software. Despite this ubiquity of polyglot programming there is no empirical research into how this practice affects software developers and their productivity. In this dissertation, after reviewing the state of the art in programming language and linguistic research pertaining to the topic, this matter is investigated by way of two empirical studies with 109 and 171 participants solving programming tasks. Based on the findings, the design of a data management library, a common use-case for polyglot programming, is proposed broadly and then applied specifically to the language Quorum as a case study. The review of previous studies finds that there is a pattern of productivity gain that can be explained by the occurrence of type annotations in programming, which gives insight into how programmers comprehend code. Study results show that there is a significant improvement of programmer productivity when programmers are using polyglot programming in an embedded context (partial eta squared = 0.039) and that less experienced programmers do better in a group with more frequent, but less severe, switches, while more experienced developers perform better with less frequent but more complete switches between languages. A study on language switches on a file level shows that file level programming language switching has a clear negative impact on programmer productivity (partial eta squared = 0.059) and is most likely caused by the increased occurrence of errors when switching

Decoding the "Free/Open Source(F/OSS) Software Puzzle" a survey of theoretical and empirical contributions

F/OSS software has been described by many as a puzzle. In the past five years, it has stimulated the curiosity of scholars in a variety of fields, including economics, law, psychology, anthropology and computer science, so that the number of contributions on the subject has increased exponentially. The purpose of this paper is to provide a sufficiently comprehensive account of these contributions in order to draw some general conclusions on the state of our understanding of the phenomenon and identify directions for future research. The exercise suggests that what is puzzling about F/OSS is not so much the fact that people freely contribute to a good they make available to all, but rather the complexity of its institutional structure and its ability to organizationally evolve over time.F/OSS software, Innovation, Incentives, Governance, Intellectual Property Rights

Quantifying usability of domain-specific languages: An empirical study on software maintenance

A domain-specific language (DSL) aims to support software development by offering abstractions to a particular domain. It is expected that DSLs improve the maintainability of artifacts otherwise produced with general-purpose languages. However, the maintainability of the DSL artifacts and, hence, their adoption in mainstream development, is largely dependent on the usability of the language itself. Unfortunately, it is often hard to identify their usability strengths and weaknesses early, as there is no guidance on how to objectively reveal them. Usability is a multi-faceted quality characteristic, which is challenging to quantify beforehand by DSL stakeholders. There is even less support on how to quantitatively evaluate the usability of DSLs used in maintenance tasks. In this context, this paper reports a study to compare the usability of textual DSLs under the perspective of software maintenance. A usability measurement framework was developed based on the cognitive dimensions of notations. The framework was evaluated both qualitatively and quantitatively using two DSLs in the context of two evolving object-oriented systems. The results suggested that the proposed metrics were useful: (1) to early identify DSL usability limitations, (2) to reveal specific DSL features favoring maintenance tasks, and (3) to successfully analyze eight critical DSL usability dimensions.This work was funded by B. Cafeo CAPES PhD Scholarship, and CNPq scholarship grant number 141688/2013-0; A. Garcia FAPERJ - distinguished scientist grant (number E-26/102.211/2009), CNPq - productivity grants (number 305526/2009-0 and 308490/2012-6), Universal project grants (number 483882/2009-7 and 485348/2011-0), and PUC-Rio (productivity grant).info:eu-repo/semantics/publishedVersio

EMPIRICAL ASSESSMENT OF THE IMPACT OF USING AUTOMATIC STATIC ANALYSIS ON CODE QUALITY

Automatic static analysis (ASA) tools analyze the source or compiled code looking for violations of recommended programming practices (called issues) that might cause faults or might degrade some dimensions of software quality. Antonio Vetro' has focused his PhD in studying how applying ASA impacts software quality, taking as reference point the different quality dimensions specified by the standard ISO/IEC 25010. The epistemological approach he used is that one of empirical software engineering. During his three years PhD, he's been conducting experiments and case studies on three main areas: Functionality/Reliability, Performance and Maintainability. He empirically proved that specific ASA issues had impact on these quality characteristics in the contexts under study: thus, removing them from the code resulted in a quality improvement. Vetro' has also investigated and proposed new research directions for this field: using ASA to improve software energy efficiency and to detect the problems deriving from the interaction of multiple languages. The contribution is enriched with the final recommendation of a generalized process for researchers and practitioners with a twofold goal: improve software quality through ASA and create a body of knowledge on the impact of using ASA on specific software quality dimensions, based on empirical evidence. This thesis represents a first step towards this goa

Empirically-Grounded Construction of Bug Prediction and Detection Tools

There is an increasing demand on high-quality software as software bugs have an economic impact not only on software projects, but also on national economies in general. Software quality is achieved via the main quality assurance activities of testing and code reviewing. However, these activities are expensive, thus they need to be carried out efficiently. Auxiliary software quality tools such as bug detection and bug prediction tools help developers focus their testing and reviewing activities on the parts of software that more likely contain bugs. However, these tools are far from adoption as mainstream development tools. Previous research points to their inability to adapt to the peculiarities of projects and their high rate of false positives as the main obstacles of their adoption. We propose empirically-grounded analysis to improve the adaptability and efficiency of bug detection and prediction tools. For a bug detector to be efficient, it needs to detect bugs that are conspicuous, frequent, and specific to a software project. We empirically show that the null-related bugs fulfill these criteria and are worth building detectors for. We analyze the null dereferencing problem and find that its root cause lies in methods that return null. We propose an empirical solution to this problem that depends on the wisdom of the crowd. For each API method, we extract the nullability measure that expresses how often the return value of this method is checked against null in the ecosystem of the API. We use nullability to annotate API methods with nullness annotation and warn developers about missing and excessive null checks. For a bug predictor to be efficient, it needs to be optimized as both a machine learning model and a software quality tool. We empirically show how feature selection and hyperparameter optimizations improve prediction accuracy. Then we optimize bug prediction to locate the maximum number of bugs in the minimum amount of code by finding the most cost-effective combination of bug prediction configurations, i.e., dependent variables, machine learning model, and response variable. We show that using both source code and change metrics as dependent variables, applying feature selection on them, then using an optimized Random Forest to predict the number of bugs results in the most cost-effective bug predictor. Throughout this thesis, we show how empirically-grounded analysis helps us achieve efficient bug prediction and detection tools and adapt them to the characteristics of each software project