Towards Diagram Understanding: A Pilot Study Measuring Cognitive Workload Through Eye-Tracking

We investigate model understanding, in particular , how the quality of the UML diagram layout impacts cognitive load. We hypothesize that this w ill have a significant impact on the structure and effectiveness of engineers’ communication. In previous work, we have studied task performance measurements and subjective assessments; here, we also investigate behavioral indicators such as fixation and pupillary dilation. We use such indicators to explore diagram understanding- and reading strategies and how such strategies are impacted, e.g. by diagram type and expertise level. In the pilot eye-tracking experiment run so far, we have only examined a small number of participants (n=4), so our results are preliminary in nature and do not afford far reaching conclusions. They do, however, corroborate findings from earlier experiments, for example, showing that layout quality indeed matters and improves understanding. Our results also give rise to a number of new hypotheses about diagram understanding strategies that we are investigating in an ongoing data acquisition campaign

A Decade and More of UML: An Overview of UML Semantic and Structural Issues and UML Field Use

More than 10 years ago in 1997, three modeling advocates brought together their own distinct techniques to forge UML (Unified Modeling Language), and the world of modeling was forever changed (Booch, Rumbaugh, & Jacobson, 1999, 2005). The Object Management Group (OMG) immediately adopted the new language as the standard for their newly expanded object-oriented (OO) modeling scope (OMG, 2008), and the stage seemed set for a modeling explosion with UML leading the way into a brave new world of more accurate and better performing systems

Highlighting model elements to improve OCL comprehension

Models, metamodels, and model transformations play a central role in Model-Driven Development (MDD). Object Constraint Language (OCL) was initially proposed as part of the Unified Modeling Language (UML) standard to add the precision and validation capabilities lacking in its diagrams, and to express well-formedness rules in its metamodel. OCL has several other applications, such as defining design metrics, code-generation templates, or validation rules for model transformations, required in MDD. Learning OCL as part of a UML course at the university would seem natural but is still the exception rather than the rule. We believe that this is mainly due to a widespread perception that OCL is hard to learn, as gleaned from claims made in the literature. Based on data gathered over the past school years from numerous undergraduate students of di↵erent Software Engineering courses, we analyzed how learning design by contract clauses with UML+OCL compares with several other Software Engineering Body Of Knowledge (SWEBOK) topics. The outcome of the learning process was collected in a rigorous setup, supported by an e-learning platform. We performed inferential statistics on that data to support our conclusions and identify the relevant explanatory variables for students’ success/failure. The obtained findings lead us to extend an existing OCL tool with two novel features: one is aimed at OCL apprentices and goes straight to the heart of the matter by allowing to visualize how OCL expressions traverse UML class diagrams; the other is intended for researchers and allows to compute OCL complexity metrics, making it possible to replicate a research study like the one we are presenting.Modelos, metamodelos e transformações de modelo desempenham um papel central em MDD. OCL foi inicialmente proposta como parte da UML para adicionar os recursos de precisão e validação que faltavam nestes diagramas, e também para expressar regras de boa formação no metamodelo. OCL possui outras aplicações, tais como definir métricas de desenho, modelos de geração de código ou regras de validação para transformações de modelo, exigidas em MDD. Aprender OCL como parte de um curso de UML na universidade parecia portanto natural, não sendo no entanto o que se verifica. Acreditamos que isso se deva a uma percepção generalizada de que OCL é difícil de aprender, tendo em conta afirmações feitas na literatura. Com base em dados recolhidos em anos letivos anteriores de vários alunos de licenciatura de diferentes cursos de Engenharia de Software, analisámos como a aprendizagem por cláusulas contratuais de UML + OCL se compara a outros tópicos do SWEBOK. O resultado do processo de aprendizagem foi recolhido de forma rigorosa, apoiado por uma plataforma de e-learning. Realizámos estatísticas inferenciais sobre os dados para apoiar as nossas conclusões, de forma a identificar as variáveis explicativas relevantes para o sucesso / fracasso dos alunos. As conclusões obtidas levaram-nos a estender uma ferramenta OCL com duas novas funcionalidades: a primeira é voltada para os estudantes de OCL e permite visualizar como as expressões percorrem um diagrama de classes UML; a segunda é voltada para investigadores e permite calcular métricas de complexidade OCL, habilitando a réplica de um estudo semelhante ao apresentado

Comprehension of Procedural Visual Business Process Models - A Literature Review

Visual process models are meant to facilitate comprehension of business processes. However, in prac- tice, process models can be difficult to understand. The main goal of this article is to clarify the sources of cog- nitive effort in comprehending process models. The article undertakes a comprehensive descriptive review of empiri- cal and theoretical work in order to categorize and sum- marize systematically existing findings on the factors that influence comprehension of visual process models. Methodologically, the article builds on a review of forty empirical studies that measure objective comprehension of process models, seven studies that measure subjective comprehension and user preferences, and thirty-two arti- cles that discuss the factors that influence the comprehen- sion of process models. The article provides information systems researchers with an overview of the empirical state of the art of process model comprehension and provides recommendations for new research questions to be addressed and methods to be used in future experiments

Quality Evaluation of Requirements Models: The Case of Goal Models and Scenarios

Context: Requirements Engineering approaches provide expressive model techniques for requirements elicitation and analysis. Yet, these approaches struggle to manage the quality of their models, causing difficulties in understanding requirements, and increase development costs. The models’ quality should be a permanent concern. Objectives: We propose a mixed-method process for the quantitative evaluation of the quality of requirements models and their modelling activities. We applied the process to goal-oriented (i* 1.0 and iStar 2.0) and scenario-based (ARNE and ALCO use case templates) models, to evaluate their usability in terms of appropriateness recognisability and learnability. We defined (bio)metrics about the models and the way stakeholders interact with them, with the GQM approach. Methods: The (bio)metrics were evaluated through a family of 16 quasi-experiments with a total of 660 participants. They performed creation, modification, understanding, and review tasks on the models. We measured their accuracy, speed, and ease, using metrics of task success, time, and effort, collected with eye-tracking, electroencephalography and electro-dermal activity, and participants’ opinion, through NASA-TLX. We characterised the participants with GenderMag, a method for evaluating usability with a focus on gender-inclusiveness. Results: For i*, participants had better performance and lower effort when using iStar 2.0, and produced models with lower accidental complexity. For use cases, participants had better performance and lower effort when using ALCO. Participants using a textual representation of requirements had higher performance and lower effort. The results were better for ALCO, followed by ARNE, iStar 2.0, and i* 1.0. Participants with a comprehensive information processing and a conservative attitude towards risk (characteristics that are frequently seen in females) took longer to start the tasks but had a higher accuracy. The visual and mental effort was also higher for these participants. Conclusions: A mixed-method process, with (bio)metric measurements, can provide reliable quantitative information about the success and effort of a stakeholder while working on different requirements models’ tasks