On the impact of service-oriented patterns on software evolvability: a controlled experiment and metric-based analysis

Background: Design patterns are supposed to improve various quality attributes of software systems. However, there is controversial quantitative evidence of this impact. Especially for younger paradigms such as service- and Microservice-based systems, there is a lack of empirical studies. Objective: In this study, we focused on the effect of four service-based patterns - namely Process Abstraction, Service Façade, Decomposed Capability, and Event-Driven Messaging - on the evolvability of a system from the viewpoint of inexperienced developers. Method: We conducted a controlled experiment with Bachelor students (N = 69). Two functionally equivalent versions of a service-based web shop - one with patterns (treatment group), one without (control group) - had to be changed and extended in three tasks. We measured evolvability by the effectiveness and efficiency of the participants in these tasks. Additionally, we compared both system versions with nine structural maintainability metrics for size, granularity, complexity, cohesion, and coupling. Results: Both experiment groups were able to complete a similar number of tasks within the allowed 90 min. Median effectiveness was 1/3. Mean efficiency was 12% higher in the treatment group, but this difference was not statistically significant. Only for the third task, we found statistical support for accepting the alternative hypothesis that the pattern version led to higher efficiency. In the metric analysis, the pattern version had worse measurements for size and granularity while simultaneously having slightly better values for coupling metrics. Complexity and cohesion were not impacted. Interpretation: For the experiment, our analysis suggests that the difference in efficiency is stronger with more experienced participants and increased from task to task. With respect to the metrics, the patterns introduce additional volume in the system, but also seem to decrease coupling in some areas. Conclusions: Overall, there was no clear evidence for a decisive positive effect of using service-based patterns, neither for the student experiment nor for the metric analysis. This effect might only be visible in an experiment setting with higher initial effort to understand the system or with more experienced developers

A study on the relationship between FindBugs warnings, metrics and expert judgments

The usage of static code analysis tools is one of the techniques to help to inspect the software quality. Many types of research have been made to evaluate such tools, but the investigate on what the analysis report further indicates and how these tools are applied in the industrial case is less concerned. This paper presents the study on three project cases in an IT Consultant company, where Findbugs, a bug pattern detection tool and SonarQube, a code quality monitoring tool are used in the development team. First, the correlation between 6 bug pattern categories and 5 source code metrics are investigated. The statistical analysis of the data extracted from tools has shown the most of the correlations are only specific to certain project. The result is partially different from expert judgments. Second, we decompose the maintainability into several characteristics, and a set of metrics measured by SonarQube are chosen to predict each of them on the basis of the practical experience. Third, the importance of metrics about test coverage and complexity is considered to be not constant among different types of projects

Evaluating and improving a scenario-based analysis method for service-based systems

Modifiability and maintenance are important topics in software development. Many metrics, such as McCabe's cyclomatic complexity, therefore attempt to quantify the maintainability of software quality attributes, in particular the sub characteristic modifiability. Quantifying specific changes in systems is not an easy task. With scenario-based methods, certain changes can be described. However, this requires a high manual effort. A previously developed tool-supported method attempted to estimate the difficulty and complexity of changes by focusing on service-based systems and using a scenario-based assessment. Because the quantification of specific changes to a system cannot be fully automated, the tool-assisted approach has tried to be lightweight, so it does not require much manual effort. This work focuses on improving and evaluating the existing tool-supported method. We collected information about the technical background, then analyzed the existing toolsupported method and made a list of improvements. Some of them have been implemented. In a next step, we analyzed the now improved tool-supported method by conducting hands-on-interviews with seven participants to give a qualitative assessment, and by conducting a survey with 40 participants to provide a quantitative assessment. Our most important finding in the evaluation is that the survey participants found the lines of code estimation is neither very accurate nor applicable and therefore not suitable for our tool-supported method according to the participants of the survey. Story Points seem to be best suited as an effort estimation method for our five predefined estimation methods according to the participants of the survey. We found no correlation between the personal background of a participant, whether a participant has worked with scenario-based evaluation in real projects, or his familiarity with service-based systems and our effort estimation methods and our evaluation functions. Our findings from the hands-on interviews have been to improve the ease of use for creating scenarios, to better visualize the evaluation features and to determine which evaluation features were helpful and which not. After collecting a derived improvement list and implementing some suggestions, we believe that our tool-supported method can be used to estimate the effort of changes in a real service-based system, as participants, as our review notes, find the evaluation view and evaluation features helpful. Keywords: scenario-based evaluation method, lightweight, service-based systems, modifiability, maintainability, tool-supported metho