Assessing the effectiveness of sequence diagrams in the comprehension of functional requirements: results from a family of five experiments

Modeling is a fundamental activity within the requirements engineering process and concerns the construction of abstract descriptions of requirements that are amenable to interpretation and validation. The choice of a modeling technique is critical whenever it is necessary to discuss the interpretation and validation of requirements. This is particularly true in the case of functional requirements and stakeholders with divergent goals and different backgrounds and experience. This paper presents the results of a family of experiments conducted with students and professionals to investigate whether the comprehension of functional requirements is influenced by the use of dynamic models that are represented by means of the UML sequence diagrams. The family contains five experiments performed in different locations and with 112 participants of different abilities and levels of experience with UML. The results show that sequence diagrams improve the comprehension of the modeled functional requirements in the case of high ability and more experienced participants.The authors wish to thank all the participants in the experiments. This research was partially supported by the MULTIPLE project (with ref. TIN2009-13838).Abrahao Gonzales, SM.; Gravino, .C.; Insfrán Pelozo, CE.; Scaniello, .G.; Tortora, .G. (2013). Assessing the effectiveness of sequence diagrams in the comprehension of functional requirements: results from a family of five experiments. IEEE Transactions on Software Engineering. 39(3):327-342. https://doi.org/10.1109/TSE.2012.27S32734239

Assessing the Effectiveness of Sequence Diagrams in the Comprehension of Functional Requirements: Results from a Family of Five Experiments

Modeling is a fundamental activity within the requirements engineering process and concerns the construction of abstract descriptions of requirements that are amenable to interpretation and validation. The choice of a modeling technique is critical whenever it is necessary to discuss the interpretation and validation of requirements. This is particularly true in the case of functional requirements and stakeholders with divergent goals and different backgrounds and experience. This paper presents the results of a family of experiments conducted with students and professionals to investigate whether the comprehension of functional requirements is influenced by the use of dynamic models that are represented by means of the UML sequence diagrams. The family contains five experiments performed in different locations and with 112 participants of different abilities and levels of experience with the UML. The results show that sequence diagrams improve the comprehension of the modeled functional requirements in the case of high ability and more experienced participants

Are Forward Designed or Reverse-Engineered UML Diagrams More Helpful for Code Maintenance?: A Controlled Experiment

Context: UML has been the de facto standard notation for modeling object-oriented software systems since its appearance in 1997. UML diagrams are important for maintainers of a system, especially when the software was developed by a different team. These diagrams of the system are not always available, however, and are commonly recovered using Reverse Engineering (RE) techniques. When obtained through RE, UML diagrams have a high level of detail as compared to those developed in the forward design activity. Method: In this paper we report on a comparison of the attitude and performance of maintainers when using these two kinds of diagrams during the maintenance of source code. Our findings were obtained by carrying out a controlled experiment with 40 students of a Master’s degree in Computer Science. Results: The results show a preference for forward design diagrams but do not display significant differences in task performance. The post-experiment survey results have led us to conclude that the subjects did not consider RE diagrams helpful; they found them difficult to understand, particularly the sequence diagrams. In the case of forward design diagrams, subjects considered sequence diagrams as useful, but they did not really employ them. Conclusions: Based on our findings, as regards performance of maintainers, there are no objective results which favor the use of one of these types of diagram in particular, i.e., UML diagrams which come from forwards design, on the one hand, and diagrams obtained from RE, on the other. Subjective opinions do, however, lead us to recommend the use of diagrams created during design. Nevertheless, we realize that the results should be considered as preliminary ones; further replications of this experiment are planned, using students and professionals, the aim being to obtain more conclusive results.Ministerio de Economía y Competitividad TIN2012-37493-C03-0

The Effects of Decomposition Quality and Multiple Forms of Information on Novices’ Understanding of a Domain from a Conceptual Model

Individuals can often use conceptual models to learn about the business domain to be supported by an information system. We investigate the extent to which such models can help novices (i.e., individuals who lack knowledge in the business domain and in conceptual modeling) to obtain an understanding of the domain codified in the model. We focus on two factors that we predict will influence novices’ understanding: (1) decomposition quality: whether the conceptual model manifests a good decomposition of the domain, and (2) multiple forms of information: whether the conceptual model is accompanied by information in another form (e.g., a textual narrative). We hypothesize that both factors will have positive effects on understanding and that these effects depend on whether the individual seeks a surface or deep understanding. Our results are largely in line with our predictions. Moreover, our results suggest that while novices are generally aware that having multiple forms of information affects their understanding, they are unaware that decomposition quality affects their understanding. Based on these results, we recommend that practitioners include complementary forms of information (such as a textual narrative) along with conceptual models and be careful to ensure that their conceptual models manifest a good decomposition of the domain

An evaluation on the comprehensibility of UML activity and state chart diagrams with regard to manual test generation

The activity and state chart diagrams are the most frequently used UML diagrams for testing a system based on its specification. One of the key important qualities of the UML diagrams is their comprehensibility. The content analysis of previous studies highlighted the lack of experts’ evaluation of the comprehensibility of activity and state chart diagrams with regard to test case generation. Thus, the main objective of this study is to evaluate the comprehensibility of the UML activity and state chart diagrams for test case generation. First, a content analysis was performed to identify the comprehensibility criteria. The criteria are perceived difficulty and subjective confidence. Next, a set of evaluation questions was designed based on the content analysis. Then, test cases were generated from activity and state chart diagrams manually of an adapted case study. An interview was conducted with five experts to validate the evaluation questions. The experts evaluated the comprehensibility of the activity and state chart diagrams by using the evaluation questions. The result of the study provided specific details of the different characteristics of activity and state chart diagrams. Further, it suggested that the activity diagram is more comprehensible than the state chart diagram in the aspect of test case generation. The finding of this study could assist software testers in choosing the appropriate UML diagrams for software testing

Towards Diagram Understanding: A Pilot Study Measuring Cognitive Workload Through Eye-Tracking

We investigate model understanding, in particular , how the quality of the UML diagram layout impacts cognitive load. We hypothesize that this w ill have a significant impact on the structure and effectiveness of engineers’ communication. In previous work, we have studied task performance measurements and subjective assessments; here, we also investigate behavioral indicators such as fixation and pupillary dilation. We use such indicators to explore diagram understanding- and reading strategies and how such strategies are impacted, e.g. by diagram type and expertise level. In the pilot eye-tracking experiment run so far, we have only examined a small number of participants (n=4), so our results are preliminary in nature and do not afford far reaching conclusions. They do, however, corroborate findings from earlier experiments, for example, showing that layout quality indeed matters and improves understanding. Our results also give rise to a number of new hypotheses about diagram understanding strategies that we are investigating in an ongoing data acquisition campaign