Assessing the effectiveness of goal-oriented modeling languages: A family of experiments

[EN] Context Several goal-oriented languages focus on modeling stakeholders' objectives, interests or wishes. However, these languages can be used for various purposes (e.g., exploring system solutions or evaluating alternatives), and there are few guidelines on how to use these models downstream to the software requirements and design artifacts. Moreover, little attention has been paid to the empirical evaluation of this kind of languages. In a previous work, we proposed value@GRL as a specialization of the Goal Requirements Language (GRL) to specify stakeholders' goals when dealing with early requirements in the context of incremental software development. Objective: This paper compares the value@GRL language with the i* language, with respect to the quality of goal models, the participants' modeling time and productivity when creating the models, and their perceptions regarding ease of use and usefulness. Method: A family of experiments was carried out with 184 students and practitioners in which the participants were asked to specify a goal model using each of the languages. The participants also filled in a questionnaire that allowed us to assess their perceptions. Results: The results of the individual experiments and the meta-analysis indicate that the quality of goal models obtained with value@GRL is higher than that of i*, but that the participants required less time to create the goal models when using i*. The results also show that the participants perceived value@GRL to be easier to use and more useful than i* in at least two experiments of the family. Conclusions: value@GRL makes it possible to obtain goal models with good quality when compared to i*, which is one of the most frequently used goal-oriented modeling languages. It can, therefore, be considered as a promising emerging approach in this area. Several insights emerged from the study and opportunities for improving both languages are outlined.This work was supported by the Spanish Ministry of Science, Innovation and Universities (Adapt@Cloud project, grant number TIN2017-84550-R) and the Programa de Ayudas de Investigación y Desarrollo (PAID-01-17) from the Universitat Politècnica de València.Abrahao Gonzales, SM.; Insfran, E.; González-Ladrón-De-Guevara, F.; Fernández-Diego, M.; Cano-Genoves, C.; Pereira De Oliveira, R. (2019). Assessing the effectiveness of goal-oriented modeling languages: A family of experiments. Information and Software Technology. 116:1-24. https://doi.org/10.1016/j.infsof.2019.08.003S12411

The Structured Process Modeling Theory (SPMT): a cognitive view on why and how modelers benefit from structuring the process of process modeling

After observing various inexperienced modelers constructing a business process model based on the same textual case description, it was noted that great differences existed in the quality of the produced models. The impression arose that certain quality issues originated from cognitive failures during the modeling process. Therefore, we developed an explanatory theory that describes the cognitive mechanisms that affect effectiveness and efficiency of process model construction: the Structured Process Modeling Theory (SPMT). This theory states that modeling accuracy and speed are higher when the modeler adopts an (i) individually fitting (ii) structured (iii) serialized process modeling approach. The SPMT is evaluated against six theory quality criteria

Recommendation and weaving of reusable mashup model patterns for assisted development

With this article, we give an answer to one of the open problems of mashup development that users may face when operating a model-driven mashup tool, namely the lack of modeling expertise. Although commonly considered simple applications, mashups can also be complex software artifacts depending on the number and types of Web resources (the components) they integrate. Mashup tools have undoubtedly simplified mashup development, yet the problem is still generally nontrivial and requires intimate knowledge of the components provided by the mashup tool, its underlying mashup paradigm, and of how to apply such to the integration of the components. This knowledge is generally neither intuitive nor standardized across different mashup tools and the consequent lack of modeling expertise affects both skilled programmers and end-user programmers alike. In this article, we show how to effectively assist the users of mashup tools with contextual, interactive recommendations of composition knowledge in the form of reusable mashup model patterns. We design and study three different recommendation algorithms and describe a pattern weaving approach for the one-click reuse of composition knowledge. We report on the implementation of three pattern recommender plugins for different mashup tools and demonstrate via user studies that recommending and weaving contextual mashup model patterns significantly reduces development times in all three cases

Ontology-driven conceptual modeling: A'systematic literature mapping and review

All rights reserved. Ontology-driven conceptual modeling (ODCM) is still a relatively new research domain in the field of information systems and there is still much discussion on how the research in ODCM should be performed and what the focus of this research should be. Therefore, this article aims to critically survey the existing literature in order to assess the kind of research that has been performed over the years, analyze the nature of the research contributions and establish its current state of the art by positioning, evaluating and interpreting relevant research to date that is related to ODCM. To understand and identify any gaps and research opportunities, our literature study is composed of both a systematic mapping study and a systematic review study. The mapping study aims at structuring and classifying the area that is being investigated in order to give a general overview of the research that has been performed in the field. A review study on the other hand is a more thorough and rigorous inquiry and provides recommendations based on the strength of the found evidence. Our results indicate that there are several research gaps that should be addressed and we further composed several research opportunities that are possible areas for future research

Evaluating the Efficacy of Value-driven Methods: A Controlled Experiment

A value model is an abstract representation of an organization and is used for capturing and describing the rationale of how the organization creates, delivers, and captures business value. Value-driven development methods use the notion of “economic value exchange” to define more efficient business strategies and align Information Systems with the organization goals. However, current value-driven methods are complex and there is not enough empirical evidence about which of the existing methods is more effective under what circumstances. This paper addresses this issue by presenting a controlled experiment aimed at comparing the Dynamic Value Description (DVD) method, which is a recently defined cognitive early requirements approach, with the well-known e3value method, with respect to their effectiveness, efficiency, perceived ease of use, perceived usefulness and intention to use. The results show that DVD has proved to be a promising method for specifying business value

The Structured Process Modeling Method (SPMM) : what is the best way for me to construct a process model?

More and more organizations turn to the construction of process models to support strategical and operational tasks. At the same time, reports indicate quality issues for a considerable part of these models, caused by modeling errors. Therefore, the research described in this paper investigates the development of a practical method to determine and train an optimal process modeling strategy that aims to decrease the number of cognitive errors made during modeling. Such cognitive errors originate in inadequate cognitive processing caused by the inherent complexity of constructing process models. The method helps modelers to derive their personal cognitive profile and the related optimal cognitive strategy that minimizes these cognitive failures. The contribution of the research consists of the conceptual method and an automated modeling strategy selection and training instrument. These two artefacts are positively evaluated by a laboratory experiment covering multiple modeling sessions and involving a total of 149 master students at Ghent University

The Effects of Decomposition Quality and Multiple Forms of Information on Novices’ Understanding of a Domain from a Conceptual Model

Individuals can often use conceptual models to learn about the business domain to be supported by an information system. We investigate the extent to which such models can help novices (i.e., individuals who lack knowledge in the business domain and in conceptual modeling) to obtain an understanding of the domain codified in the model. We focus on two factors that we predict will influence novices’ understanding: (1) decomposition quality: whether the conceptual model manifests a good decomposition of the domain, and (2) multiple forms of information: whether the conceptual model is accompanied by information in another form (e.g., a textual narrative). We hypothesize that both factors will have positive effects on understanding and that these effects depend on whether the individual seeks a surface or deep understanding. Our results are largely in line with our predictions. Moreover, our results suggest that while novices are generally aware that having multiple forms of information affects their understanding, they are unaware that decomposition quality affects their understanding. Based on these results, we recommend that practitioners include complementary forms of information (such as a textual narrative) along with conceptual models and be careful to ensure that their conceptual models manifest a good decomposition of the domain

Toward a Taxonomy of Modeling Difficulties: A Multi-Modal Study on Individual Modeling Processes

Conceptual modeling is an essential activity during information systems development and, accordingly, a learning task faced by students of Information Systems. Presently, surprisingly little is known about how learning processes of conceptual modeling proceed, and about modeling difficulties learners experience. In this study, we integrate complementary modes of observation of learners\u27 modeling processes to identify modeling difficulties these learners face while performing a data modeling task using a modeling tool. We use the concept of cognitive breakdowns to analyze verbal protocols, recordings of learner-tool interactions and video recordings of learners\u27 modeling processes and survey learners about modeling difficulties. Our study identifies five types of modeling difficulties relating to different aspects of constructing conceptual data models, i.e., entity types, relationship types, attributes, and cardinalities. The identified types of modeling difficulties motivate a taxonomic theory of modeling difficulties intended to inform design science research on tool support for learners of conceptual modeling