Eye Tracking Experiments on Process Model Comprehension: Lessons Learned

For documenting business processes, there exists a plethora of process modeling languages. In this context, graphical process models are used to enhance the process comprehensibility of the stakeholders involved. The large number of available modeling languages, however, aggravates process model comprehension and increases the knowledge gap between domain and modeling experts. Upon this, one major challenge is to identify factors fostering the comprehension of process models. This paper discusses the experiences we gathered with the use of eye tracking in experiments on process model comprehension and the lessons learned in this context. The objective of the experiments was to study the comprehension of process models expressed in terms of four different modeling languages (i.e., BPMN, eGantt, EPC, and Petri Net). This paper further provides recommendations along nine identified categories that can foster related experiments on process model comprehension

Investigating the Flow Direction in Business Process Models: An Eye Tracking Study

There exist a plethora of different process modelling languages for the graphical documentation of business processes. In this context, the Business Process Model and Notation (BPMN) 2.0 is one of the most well-known modelling languages and has become the de-facto standard in industry. Guidelines in BPMN 2.0 exist that describe to keep a modelling and reading direction from left-to-right or top-to-bottom. However, there exist no work so far providing empirical evidence about the influence of different reading direction during the reading and comprehension of process models. For this reason, the thesis at hand addresses this issue in an empirical study in order to investigate the influence of different reading directions. In particular, an eye tracking study involving 28 participants was conducted. In this study, four reading directions (i.e., left-to-right, right-to-left, top-to-bottom, bottom-to-top) were presented to the participants and their influence on process model comprehension was evaluated. The results of the eye tracking study show that performance in process model comprehension was similar in all four reading directions and, hence, no significant differences were found. Moreover, the results demonstrate that process model readers adapt well to less common reading directions in business process models

USING SECONDARY NOTATION TO IMPROVE THE COGNITIVE EFFECTIVENESS OF BPMN-MODELS

Almost every implementation of a modeling grammar uses secondary notation to further specify a modeling grammar. Yet, secondary notation is usually applied in an unsystematic way, might contradict what is specified in primary notation and implements research results that should rather be implemented in primary notation. With this work we aim at showing how secondary notation can be used to implement recent research results that are not yet available in primary notation without contracting what is already specified in primary notation. We demonstrate a systematic update of recent research of extended Perceptual Discriminability for BPMN secondary notation and that way, show how research results can quickly be made available for practice without contradicting primary notation. We choose Perceptual Discriminability as it can be used to focus the model user’s attention on the most important constructs and can that way, improve model comprehension. For an update of BPMN secondary notation we first specify free BPMN variables and further show how these variables can be used to focus the model user’s attention on those constructs that most foster comprehension

How quickly do we learn conceptual models?

In organizations, conceptual models are used for understanding domain concepts. Learning the domain from models is crucial for the analysis and design of information systems that are intended to support the domain. Past research has proposed theories to structure conceptual models in order to improve learning. It has, however, never been investigated how quickly domain knowledge is acquired when using theory-guided conceptual models. Based on theoretical arguments, we hypothesize that theory-guided conceptual models expedite the initial stages of learning. Using the REA ontology pattern as an example of theoretical guidance, we show in a laboratory experiment how an eye-tracking procedure can be used to investigate the effect of using theory-guided models on the speed of learning. Whereas our experiment shows positive effects on both outcome and speed of learning in the initial stages of learning, the real contribution of our paper is methodological, i.e. an eye-tracking procedure to observe the process of learning from conceptual models

Towards the Applicability of Measuring the Electrodermal Activity in the Context of Process Model Comprehension: Feasibility Study

Process model comprehension is essential in order to understand the Five Ws (i.e., who, what, where, when, and why) pertaining to the processes of organizations. However, research in this context showed that a proper comprehension of process models often poses a challenge in practice. For this reason, a vast body of research exists studying the factors having an influence on process model comprehension. In order to point research towards a neuro-centric perspective in this context, the paper at hand evaluates the appropriateness of measuring the electrodermal activity (EDA) during the comprehension of process models. Therefore, a preliminary test run and a feasibility study were conducted relying on an EDA and physical activity sensor to record the EDA during process model comprehension. The insights obtained from the feasibility study demonstrated that process model comprehension leads to an increased activity in the EDA. Furthermore, EDA-related results indicated significantly that participants were confronted with a higher cognitive load during the comprehension of complex process models. In addition, the experiences and limitations we have learned in measuring the EDA during the comprehension of process models are discussed in this paper. In conclusion, the feasibility study demonstrated that the measurement of the EDA could be an appropriate method to obtain new insights in process model comprehension

The Use of Eye-tracking in Information Systems Research: A Literature Review of the Last Decade

Eye-trackers provide continuous information on individuals’ gaze behavior. Due to the increasing popularity of eye- tracking in the information systems (IS) field, we reviewed how past research has used eye-tracking to inform future research. Accordingly, we conducted a literature review to describe the use of eye-tracking in IS research based on a sample of 113 empirical papers published since 2008 in IS journals and conference proceedings. Specifically, we examined the methodologies and experimental settings used in eye-tracking IS research and how eye-tracking can be used to inform the IS field. We found that IS research that used eye-tracking varies in its methodological and theoretical complexity. Research on pattern analysis shows promise since such research develops a broader range of analysis methodologies. The potential of eye-tracking remains unfulfilled in the IS field since past research has mostly focused on attention-related constructs and used fixation count metrics on desktop computers. We call for researchers to utilize eye-tracking more broadly in IS research by extending the type of metrics they use, the analyses they perform, and the constructs they investigate

Measuring the Cognitive Complexity in the Comprehension of Modular Process Models

Modularization in process models is a method to cope with the inherent complexity in such models (e.g., model size reduction). Modularization is capable to increase the quality, the ease of reuse, and the scalability of process models. Prior conducted research studied the effects of modular process models to enhance their comprehension. However, the effects of modularization on cognitive factors during process model comprehension are less understood so far. Therefore, this paper presents the results of two exploratory studies (i.e., a survey research study with N = 95 participants; a follow-up eye tracking study with N = 19 participants), in which three types of modularization (i.e., horizontal, vertical, orthogonal) were applied to process models expressed in terms of the Business Process Model and Notation (BPMN) 2.0. Further, the effects of modularization on the cognitive load, the level of acceptability, and the performance in process model comprehension were investigated. In general, the results revealed that participants were confronted with challenges during the comprehension of modularized process models. Further, performance in the comprehension of modularized process models showed only a few significant differences, however, the results obtained regarding the cognitive load revealed that the complexity and concept of modularization in process models were misjudged initially. The insights unraveled that the attitude towards the application and the behavioral intention to apply modularization in process model is still not clear. In this context, horizontal modularization appeared to be the best comprehensible modularization approach leading to a more fine-grained comprehension of respective process models. The findings indicate that alterations in modular process models (e.g., change in the representation) are important to foster and enable their comprehension. Finally, based on our results, implications for research and practice as well as directions for future work are discussed in this paper

Design principles for ensuring compliance in business processes

In this thesis, we evaluate the complexity and understandability of compliance languages. First, to calculate the complexity, we apply established software metrics and interpret the results with respect to the languages’ expressiveness. Second, to investigate the languages’ understandability, we use a cognitive model of the human problem-solving process and analyze how efficiently users perform a compliance modeling task. Our results have theoretical and practical implications that give directions for the development of compliance languages, and rule-based languages in general.Diese Arbeit beurteilt die Komplexität und Verständlichkeit von Compliance-Sprachen. Zur Messung der Komplexität wenden wir etablierte Software-Metriken an und interpretieren die Ergebnisse in Hinblick auf die Aussagekraft der Sprachen. Zur Untersuchung der Verständlichkeit verwenden wir ein kognitives Modell und analysieren, wie effizient eine Compliance-Sprache zur Lösung eines Modellierungsproblems eingesetzt wird. Unsere Ergebnisse haben theoretische und praktische Implikationen für die Entwicklung von Compliance-Sprachen und anderen regelbasierten Sprachen