An Experimental Scrutiny of Visual Design Modelling: VCL up against UML+OCL

The graphical nature of prominent modelling notations, such as the standards UML and SysML, enables them to tap into the cognitive benefits of diagrams. However, these notations hardly exploit the cognitive potential of diagrams and are only partially graphical with invariants and operations being expressed textually. The Visual Contract Language (VCL) aims at improving visual modelling; it tries to (a) maximise diagrammatic cognitive effectiveness, (b) increase visual expressivity, and (c) level of rigour and formality. It is an alternative to UML that does largely pictorially what is traditionally done textually. The paper presents the results of a controlled experiment carried out four times in different academic settings and involving 43 participants, which compares VCL against UML and OCL and whose goal is to provide insight on benefits and limitations of visual modelling. The paper's hypotheses are evaluated using a crossover design with the following tasks: (i) modelling of state space, invariants and operations, (ii) comprehension of modelled problem, (iii) detection of model defects and (iv) comprehension of a given model. Although visual approaches have been used and advocated for decades, this is the first empirical investigation looking into the effects of graphical expression of invariants and operations on modelling and model usage tasks. Results suggest VCL benefits in defect detection, model comprehension, and modelling of operations, providing some empirical evidence on the benefits of graphical software design

Effects of Using Examples on Structural Model Comprehension: A Controlled Experiment

We present a controlled experiment for the empirical evaluation of Example-Driven Modeling (EDM), an approach that systematically uses examples for model comprehension and domain knowledge transfer. We conducted the experiment with 26 graduate and undergraduate students from electrical and computer engineering (ECE), computer science (CS), and software engineering (SE) programs at the University of Waterloo. The experiment involves a domain model, with a UML class diagram representing the domain abstractions and UML object diagrams representing examples of using these abstractions. The goal is to provide empirical evidence of the effects of suitable examples on model comprehension, compared to having model abstractions only, by having the participants perform model comprehension tasks. Our results show that EDM is superior to having model abstractions only, with an improvement of (+39%) for diagram completeness, (+30%) for study questions completeness, (+71%) for efficiency, and a reduction of (-80%) for the number of mistakes. We provide qualitative results showing that participants receiving model abstractions augmented with examples experienced lower perceived difficulty in performing the comprehension tasks, higher perceived confidence in their tasks' solutions, and asked fewer clarifying domain questions (a reduction of 90%). We also present participants' feedback regarding the usefulness of the provided examples, the number of examples, the types of examples, and the use of partial examples

Empirical Assessment of UML Static Object Diagrams

The essential UML logic diagrams are the class diagrams: they represent the classes of objects that make up a program. Object diagrams are used as the basis to show scenarios of dynamic evolution of the software. The use of object diagrams to depict static structure is very rare. The purpose of this study is to investigate whether the use of static object diagrams can improve the comprehension of software systems. We conducted a study with 17 graduate students during a software engineering course. The students were asked to answer questions about a software system. The system was described either with a class diagram or with both a class diagram and an object diagram. The student asked multiple choice questions on four different systems. This study revealed that there is a statistically significant difference in the comprehension achievement for two of the systems. The effect of the presence of object diagrams can be classified of medium size. These results allow us to formulate new research questions that guide our future work in this area