Simsax: A measure of project similarity based on symbolic approximation method and software defect inflow

Background: Profiling software development projects, in order to compare them, find similar sub-projects or sets of activities, helps to monitor changes in software processes. Since we lack objective measures for profiling or hashing, researchers often fall back on manual assessments. Objective: The goal of our study is to define an objective and intuitive measure of similarity between software development projects based on software defect-inflow profiles. Method: We defined a measure of project similarity called SimSAX which is based on segmentation of defect-inflow profiles, coding them into strings (sequences of symbols) and comparing these strings to find so-called motifs. We use simulations to find and calibrate the parameters of the measure. The objects in the simulations are two different large industry projects for which we know the similarity a priori, based on the input from industry experts. Finally, we apply the measure to find similarities between five industrial and six open source projects. Results: Our results show that the measure provides the most accurate simulated results when the compared motifs are long (32 or more weeks) and we use an alphabet of 5 or more symbols. The measure provides the possibility to calibrate for each industrial case, thus allowing to optimize the method for finding specific patterns in project similarity. Conclusions: We conclude that our proposed measure provides a good approximation for project similarity. The industrial evaluation showed that it can provide a good starting point for finding similar periods in software development projects

Deep learning model for end-to-end approximation of COSMIC functional size based on use-case names

Context: COSMIC is a widely used functional size measurement (FSM) method that supports software development effort estimation. The FSM methods measure functional product size based on functional requirements. Unfortunately, when the description of the product\u27s functionality is often abstract or incomplete, the size of the product can only be approximated since the object to be measured is not yet fully described. Also, the measurement performed by human-experts can be time-consuming, therefore, it is worth considering automating it. Objective: Our objective is to design a new prediction model capable of approximating COSMIC-size of use cases based only on their names that is easier to train and more accurate than existing techniques. Method: Several neural-network architectures are investigated to build a COSMIC size approximation model. The accuracy of models is evaluated in a simulation study on the dataset of 437 use cases from 27 software development projects in the Management Information Systems (MIS) domain. The accuracy of the models is compared with the Average Use-Case approximation (AUC), and two recently proposed two-step models—Average Use-Case Goal-aware Approximation (AUCG) and Bayesian Network Use-Case Goal AproxImatioN (BN-UCGAIN). Results: The best prediction accuracy was obtained for a convolutional neural network using a word-embedding model trained on Wikipedia+Gigaworld. The accuracy of the model outperformed the baseline AUC model by ca. 20%, and the two-step models by ca. 5–7%. In the worst case, the improvement in the prediction accuracy is visible after estimating 10 use cases. Conclusions: The proposed deep learning model can be used to automatically approximate COSMIC size of software applications for which the requirements are documented in the form of use cases (or at least in the form of use-case names). The advantage of the model is that it does not require collecting historical data other than COSMIC size and names of use cases

LegacyPro: A DNA-inspired method for identifying process legacies in software development organizations

Changing a software development process is a tricky task—the bigger the change, the trickier it gets. Large companies have the inertia of processes, the change of process takes time, happens over multiple releases and at different pace in different parts of the organization. Unfortunately, there are no effective tools available that help us determine if an organization has really adopted a proclaimed process change, as well as to what extent it is making progress towards this desired state. This paper presents a novel and unique method for determining the factual adoption of new processes in software R&D organizations. We use a DNA-inspired analysis (motifs) to categorize parts and find similarities between projects using defect-inflow profiles. We applied the method to analyze projects from a large infrastructure provider and from open source and show quantification of the evolution of processes

Automated Code Review Comment Classification to\ua0Improve Modern Code Reviews

Modern Code Reviews (MCRs) are a widely-used quality assurance mechanism in continuous integration and deployment. Unfortunately, in medium and large projects, the number of changes that need to be integrated, and consequently the number of comments triggered during MCRs could be overwhelming. Therefore, there is a need for quickly recognizing which comments are concerning issues that need prompt attention to guide the focus of the code authors, reviewers, and quality managers. The goal of this study is to design a method for automated classification of review comments to identify the needed change faster and with higher accuracy. We conduct a Design Science Research study on three open-source systems. We designed a method (CommentBERT) for automated classification of the code-review comments based on the BERT (Bidirectional Encoder Representations from Transformers) language model and a new taxonomy of comments. When applied to 2,672 comments from Wireshark, The Mono Framework, and Open Network Automation Platform (ONAP) projects, the method achieved accuracy, measured using Matthews Correlation Coefficient, of 0.46–0.82 (Wireshark), 0.12–0.8 (ONAP), and 0.48–0.85 (Mono). Based on the results, we conclude that the proposed method seems promising and could be potentially used to build machine-learning-based tools to support MCRs as long as there is a sufficient number of historical code-review comments to train the model

Using Machine Learning to Design a Flexible LOC Counter

The results of counting the size of programs in terms of Lines-of-Code (LOC) depends on the rules used for counting (i.e. definition of which lines should be counted). In the majority of the measurement tools, the rules are statically coded in the tool and the users of the measurement tools do not know which lines were counted and which were not. The goal of our research is to investigate how to use machine learning to teach a measurement tool which lines should be counted and which should not. Our interest is to identify which parameters of the learning algorithm can be used to classify lines to be counted. Our research is based on the design science research methodology where we construct a measurement tool based on machine learning and evaluate it based on open source programs. As a training set, we use industry professionals to classify which lines should be counted. The results show that classifying the lines as to be counted or not has an average accuracy varying between 0.90 and 0.99 measured as Matthew\u27s Correlation Coefficient and between 95% and nearly 100% measured as the percentage of correctly classified lines. Based on the results we conclude that using machine learning algorithms as the core of modern measurement instruments has a large potential and should be explored further