While the understandability of process models has been extensively investigated for different process modeling notations, it has not been yet broadened to cover hybrid models. This paper proposes a new research model to investigate the understandability of hybrid model representations using a variety of psycho-physiological measurements including eye tracking and galvanic skin response (GSR) together with verbal data analysis. The aim of this research is to ensure a smooth integrationof hybrid modelling technologies in public administrations by investigating the way end-users (i.e., case workers) rely on the different parts of the hybrid process model representation in DCR Graphs, i.e., the graph, the textual annotations describing the law, and the simulation tools to interpret the process model

Abbad Andaloussi, Amine

Burattin, Andrea

Hildebrandt, Thomas Troels

Slaats, Tijs

Weber, Barbara

Online Research Database In Technology

                       General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.  • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal   If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.     Downloaded from orbit.dtu.dk on: Jun 17, 2018Understandability of Hybrid Process Models Using DCR GraphsAbbad Andaloussi, Amine; Slaats, Tijs; Burattin, Andrea; Hildebrandt, Thomas Troels; Weber, BarbaraPublished in:Proceedings of 30th International Conference on Advanced Information Systems EngineeringPublication date:2018Document VersionPeer reviewed versionLink back to DTU OrbitCitation (APA):Abbad Andaloussi, A., Slaats, T., Burattin, A., Hildebrandt, T. T., & Weber, B. (2018). Understandability of HybridProcess Models Using DCR Graphs. In Proceedings of 30th International Conference on Advanced InformationSystems EngineeringA Research Model to Test the Understandabilityof Hybrid Process Models Using DCR GraphsAmine Abbad Andaloussi1, Tijs Slaats2, Andrea Burattin1, Thomas T.Hildebrandt2, and Barbara Weber11 Software and Process Engineering, Technical University of Denmark2800 Kgs. Lyngby, Denmark{amab, andbur, bweb}@dtu.dk2 Department of Computer Science, University of Copenhagen2100 København Ø{slaats, hilde}@di.ku.dkAbstract. While the understandability of process models has been ex-tensively investigated for different process modeling notations, it hasnot been yet broadened to cover hybrid models. This paper proposes anew research model to investigate the understandability of hybrid modelrepresentations using a variety of psycho-physiological measurements in-cluding eye tracking and galvanic skin response (GSR) together withverbal data analysis. The aim of this research is to ensure a smooth in-tegration of hybrid modelling technologies in public administrations byinvestigating the way end-users (i.e., case workers) rely on the differentparts of the hybrid process model representation in DCR Graphs, i.e.,the graph, the textual annotations describing the law, and the simulationtools to interpret the process model.1 IntroductionThe Ecoknow3 project aims at developing solutions for the effective digitaliza-tion of knowledge work processes. Using hybrid modelling technologies, Ecoknowproposes a new Advanced Case Management (ACM) system that combines bothflexibility and compliance with the law. Since this technology empowers endusers (i.e, case workers) and place them in the center of the decision makingprocess, it is crucial to ensure its ease of use. By investigating the way end usersinteract with the different parts of the hybrid process model representation inDCR graphs (i.e., the graph, the textual annotations describing the law, and thesimulation tools; cf. Figure 1), the understandability and effectiveness of this ap-proach can be evaluated. This paper proposes a new research model to investigatethe understandability of hybrid models using a variety of physio-psychologicalmeasurements derived from questionnaires, user interactions, eye tracking data,verbal data, and GSR signal. The outcome can then be used to further improvethe platform and contribute to higher user satisfaction and compliance with thelaw. Section 2 briefly presents the related work, Section 3 introduces our researchmethod, and finally Section 4 concludes the paper.3 See https://ecoknow.org1Fig. 1: A hybrid representation of DCR graphs. The hybrid representation in-cludes the DCR graph, the textual annotations, and the simulation tools.2 Related workHybrid model representations are usually used in the following contexts:(a) Combination of two modeling notations. In this context, several hybrid nota-tions have been proposed in [2, 3, 7] and [11] to combine imperative and declar-ative notations. (b) Combination of model notation with textual annotation,namely, Pinggera et al. in [6] proposed the Literate Process Modeling technique(LiProMo) that aims at improving the communication during the process of pro-cess modeling by inter-waving textual descriptions with process models. Also,Wang et al. in [10] suggested a hybrid representation that combines a graph-ical representation of the process model and a textual annotation prescribingthe underlying linked rules. (c) Support of model notation with test cases. Thisapproach has been proposed and evaluated by Zugal et al. in [14–16]. The ap-proach uses Test Driven Modelling (TDM) to improve the understandability ofdeclarative models.The hybrid representation introduced by the DCR portal combines the modelnotation, with the textual annotation, and offers the support of the simulationtool. Thus, its context is similar to the approaches proposed in (a) and (b).3 Research MethodThis study investigates the factors affecting the understandability of hybridmodel representations using multi-modal data collection. In particular, an eyetracking study supported by questionnaires, GSR signal recording, users inter-actions, and retrospective think-aloud [5, p. 104-108] is conducted to provideinsights from different angles.2Afterwards, a quantitative analysis is performed on the eye tracking data, thequestionnaire data, the user interactions, and the GSR signal. While the verbaldata obtained from the retrospective think-aloud is transcribed and processedbased on grounded theory [8], such that the recurrent aspects are identified andgrouped into categories that are further analyzed and refined at the analysisphase. The same approach has been successfully used in a similar empiricalresearch study held by Haisjack et al. in [4], where the authors have investigatedthe understanding of declarative process models.The following sections describe the key aspects considered by the planningphase of our exploratory study. Section 3.1 defines the main research problem,Section 3.2 describes the experiment subjects, Section 3.3 presents the experi-ment objects, and Section 3.4 explains the measures.3.1 Research ProblemThe aim of this study is to understand how subjects with different levels ofexpertise in DCR will use the different artifacts provided by the DCR portal(graph, textual annotations describing the law, simulation tools) to answer a setof model related questions. The research problem is formulated into the followingresearch questions:– RQ1: How is the subjects’ attention distributed over the different artifactsprovided by the hybrid representation during a comprehension session?– RQ2: In what circumstances do the subjects rely more on one artifact ratherthan another to answer the model questions?– RQ3: To what extend does the hybrid representation contribute to answerthe model questions correctly?– RQ4: How does the subjects process and merge the information coming fromthe graph, the textual annotations describing the law, and the simulationtools?The research questions presented in this section aim at investigating theaspects we suspect to be relevant to our exploratory study. Namely, QuestionRQ1 investigates whether the subjects will be using the artifacts provided bythe hybrid representation accordingly. In other words, we expect that dependingon the experiment question, one or many artifacts should be used to answerthat question. The total time spent looking at/interacting with each artifactcan provide indications about its usability (cf. Section 3.4 for more details).Question RQ2 identifies the circumstances/conditions when the subjects relymore on one artifacts than another. These circumstances are expected to varydepending on the model questions type (i.e., asking about a specific control-flowexecution vs. asking about a general behaviour in the model), and probably thecomplexity of the targeted part of the DCR graph (i.e., the use of relations suchas condition, response, include, exclude, milestone). Question RQ3 examinesthe extent to which the use of different artifacts contributes in answering theexperiment questions correctly. In this context, it is expected that the subjects3who rely on specific artifacts are most likely to provide correct answers. Finally,Question RQ4 addresses the cognitive aspect of using a hybrid representation. Byexploring the subjects’ gaze data and the interaction logs we expect to identifythe common reading patterns and to analyze the information merging strategiesused by the subjects.3.2 SubjectsThe subjects who participated in this experiment have different backgroundsand different levels of expertise in using DCR graphs. The initial sample com-prises subjects employed at the Syddjurs municipality4 (i.e., case workers), andstudents/employees from two danish universities5. The subjects from Syddjursmunicipality have limited experience with DCR graphs while the subjects fromthe universities are more advanced in using DCR graphs.As a minimal knowledge about the hybrid representation offered by the DCRportal (cf. Figure 1) and the experiment procedure are required for all partici-pants regardless of their level of expertise, all the subjects had a basic trainingprior to the experiment.3.3 ObjectsThe DCR model used in this study originate from Section §45 of the “Con-solidation Act on Social Services”. The experiment was designed in both Englishand Danish. The law texts for both versions are available online6. In order toevaluate the understandability of the different artifacts of the hybrid represen-tation, we have designed a set of questions that can be categorized as follows:– Trace questions asking about a specific control-flow execution (trace). Weexpect that these questions will be more engaging to use the simulationtools in order to provide correct answers.– Model-law questions asking about details that we assume to be easily rec-ognized from the law text fragments shown at the description of the DCRgraph activities.– Model questions asking about details that can be inferred by looking closelyat the DCR graph semantics.– General Purpose questions asking to identify and describe specific sub-sections of the law text in the DCR graph. The aim is to engage the subjectsin the process of model comprehension and assess their understandability ofthe DCR graph semantics.4 Danish Minicipality, See https://www.syddjurs.dk5 IT University of Copenhagen (ITU) and Technical University of Denmark (DTU)6 For English version see http://english.sm.dk/media/14900/consolidation-act-on-social-services.pdf, and for Danish version seehttps://www.retsinformation.dk/Forms/R0710.aspx?id=1970364In addition, we use extra questions to assess the cognitive load and the feel-ings of the subject at the end of each trial. The answers provided are validatedagainst the data recorded by the eye tracker, the GSR device, and the userinteractions.3.4 MeasuresTo obtain an understanding of the understandability of hybrid models weuse multi-modal data collection. More specifically, we rely on questionnaires,user interactions, eye tracking data, verbal data, and GSR signal. This sectionprovides an overview about each of the measurements and highlights its domainof application.The questionnaire data is used to measure the answering accuracy, perceivedcognitive load [13], as well emotional responses (i.e., valence and arousal us-ing MANIKIN [1]). The model questions are crafted as dichotomous questions(i.e., true/false), while the perceived cognitive load and emotional responses aremeasured using a 9-point Likert scale.The gaze data recorded during the experiment is also of high importance. Thetwo primary gaze events inferred from the gaze data are fixations and saccades.A fixation refers to the period of time where the eye remains still [5, p. 21]. Forexample, when reading a sentence, the period of time the eye stops at a word isconsidered as a fixation. A saccade refers to the rapid movement of the eye fromone fixation to another [5, p. 23]. Fixations and saccades can be detected usingdifferent oculomotor events estimation algorithms [5, p. 147-186]. The availabilityof these two events allow to perform several type of eye tracking analysis. Forinstance, one can compare the fixation duration and the fixation count betweendifferent areas of interest. Indeed, it it is possible to split the view provided bythe user interface into areas, such that each area is seen as a distinct area ofinterest (AOI) (i.e., graph, law text, simulation tools). Given a drawing of theAOIs as shown in Figure 2 one can compare the fixation duration and the fixationcount between the different artifacts. Furthermore, using the saccades one canbuild a transition matrix showing all the transitions between the different areaof interests [5, p. 193-197]. Fixations and saccades can be also used to infer othermeasures such as Attention Maps, and Scanpaths. Attention maps provide a goodoverview over the data. they use different colors to emphasis on the number offixations the subjects have made on different parts of the stimulus [5, p. 231-251].While Scanpaths allow to perceive the viewing patterns of subjects by displayingon the stimulus a route showing the sequence of fixations and saccades recordedduring a certain timespan [5, p. 253].The interaction logs, in turn, allow mapping the interaction events (i.e.,mouse hovering and clicks over model artifacts and use of simulation) with thegaze events for a better analysis of the subjects’ behavior. In addition, they canbe be transformed to event logs and analyzed using state-of-art process miningtechniques [9].This exploratory study takes also advantage from additional channels suchas the verbal data extracted from the retrospective think-aloud session held by5AOI: Graph AOI: Simulation ToolsAOI: Law TextFig. 2: Example of AOIs depicted on the hybrid version of DCR portal. Theexample contains 3 areas of interest: Graph, Law Text, Simulation Tools. Byprojecting these area of interests on a video recording the user interactions andthe gaze data during a comprehension task, it becomes possible to map eachgaze event (i.e., fixation), or user interaction event to its corresponding area ofinterest.the end of each experiment session and the galvanic skin response data providedby the GSR device. The variations in the GSR signal provide insights aboutarousal and valance which in turn allow to get insights about the cognitive loadand the emotional reactions of the subject during the experiment [5, p. 231-251].This part is tightly related to a previous work done by Westerman et al. in [12],where the authors analyzed usability tasks based on GSR data.4 Conclusion and Future WorkThis paper summarizes the research model, we have developed to investigatethe understandability of hybrid models. The various data channels enable amulti-model data collection, and open up opportunities for several types of dataanalysis.As future work, we are planning to analyze the experiment data. Our mainresearch problem is about understanding the way subjects with different levelsof expertise will use the hybrid representation proposed by DCR graphs. Bythe end of this study, we aim at providing convincing answers to the researchquestions presented in Section 3.1.6References1. M. Bradley and P. J. Lang. Measuring Emotion: The Self-Assessment Seman-tic Differential Manikin and the. Journal of Behavior Therapy and ExperimentalPsychiatry, 25(I):49–59, 1994.2. G. De Giacomo, M. Dumas, F. M. Maggi, and M. Montali. Declarative ProcessModeling in BPMN. pages 84–100. 2015.3. J. De Smedt, J. De Weerdt, J. Vanthienen, and G. Poels. Mixed-Paradigm Pro-cess Modeling with Intertwined State Spaces. Business & Information SystemsEngineering, 58(1):19–29, feb 2016.4. C. Haisjackl, I. Barba, S. Zugal, P. Soffer, I. Hadar, M. Reichert, J. Pinggera, andB. Weber. Understanding Declare models: strategies, pitfalls, empirical results.Software & Systems Modeling, 15(2):325–352, may 2016.5. K. Holmqvist, M. Nystro¨m, R. Andersson, R. Dewhurst, H. Jarodzka, and J. van deWeijer. Eye Tracking: A comprehensive guide to methods and measures. OUPOxford, 2011.6. J. Pinggera, T. Porcham, S. Zugal, and B. Weber. LiProMo-Literate process mod-eling. CEUR Workshop Proceedings, 855:163–170, 2012.7. T. Slaats, D. M. M. Schunselaar, F. M. Maggi, and H. A. Reijers. The Semanticsof Hybrid Process Models. pages 531–551. 2016.8. A. Strauss and J. Corbin. Basics of qualitative research: grounded theory proceduresand techniques. Sage Publications, 1990.9. W. M. Van Der Aalst. Process mining: Discovery, Conformance, and Enhancementof Business processes. Springer International Publishing, 2010.10. W. Wang, M. Indulska, S. Sadiq, and B. Weber. Effect of Linked Rules on BusinessProcess Model Understanding. pages 200–215. 2017.11. M. Westergaard and T. Slaats. Mixing Paradigms for More Comprehensible Mod-els. pages 283–290. 2013.12. S. J. Westerman, E. J. Sutherland, L. Robinson, H. Powell, and G. Tuck. AMulti-method Approach to the Assessment of Web Page Designs. In A. C. R.Paiva, R. Prada, and R. W. Picard, editors, Affective Computing and IntelligentInteraction, pages 302–313, Berlin, Heidelberg, 2007. Springer Berlin Heidelberg.13. S. Zugal. Applying Cognitive Psychology for Improving the Creation, Understand-ing and Maintenance of Business Process Models. 2013.14. S. Zugal, J. Pinggera, and B. Weber. The Impact of Testcases on the Maintainabil-ity of Declarative Process Models. In T. Halpin, S. Nurcan, J. Krogstie, P. Soffer,E. Proper, R. Schmidt, and I. Bider, editors, Enterprise, Business-Process andInformation Systems Modeling, pages 163–177, Berlin, Heidelberg, 2011. SpringerBerlin Heidelberg.15. S. Zugal, J. Pinggera, and B. Weber. Creating Declarative Process Models UsingTest Driven Modeling Suite. In S. Nurcan, editor, IS Olympics: Information Sys-tems in a Diverse World, pages 16–32, Berlin, Heidelberg, 2012. Springer BerlinHeidelberg.16. S. Zugal, J. Pinggera, and B. Weber. Toward enhanced life-cycle support fordeclarative processes. Journal of Software: Evolution and Process, 24(3):285–302,apr 2012.7

Understandability of Hybrid Process Models Using DCR Graphs

http://orbit.dtu.dk/ws/files/149169776/UnderstandabilityPP.pdf

Understandability of Hybrid Process Models Using DCR Graphs

Abstract

Similar works

Full text

Available Versions

Online Research Database In Technology