Un Framework para la generación automática de ejercicios mediante técnicas de mutación

En este trabajo se describe el diseño e implementación de un entorno que genera ejercicios tipo test de forma automática mediante técnicas de mutación, llamado Wodel-Edu. Wodel-Edu es una extensión de post-procesado para el Lenguaje de Dominio Específico (DSL) Wodel, desarrollado por el grupo MISO, y que proporciona primitivas de alto nivel para mutación de modelos. Para ello, se ha extendido el DSL Wodel con nuevas primitivas de mutación, nuevas estrategias de selección, un registro de las mutaciones aplicadas, un control de la generación de mutantes duplicados, y una comprobación de que los mutantes que se generan son modelos correctos (conformes a su meta-modelo). También se ha dotado a Wodel de un mecanismo extensible que permite registrar distintas acciones de post-procesado sobre los mutantes generados, extensión sobre la que se ha implementado el entorno Wodel-Edu. Wodel-Edu es independiente del dominio, y genera tres formatos diferentes de ejercicios tipo test: el primero, en el que se presentan varios diagramas, y el estudiante ha de decidir cuál es el correcto; el segundo, en el que se presenta un único diagrama, y el estudiante ha de decidir si es correcto, o no; el tercer formato, se presentan una serie de posibles cambios a realizar sobre el diagrama para corregirlo, y el estudiante ha de seleccionar cuáles de estos cambios son correctos. En este trabajo se ha elegido utilizar Wodel-Edu para generar ejercicios de autómatas finitos. Se presenta además una evaluación de la aplicación de ejercicios generada.This work presents the design and development of a framework for the automatic generation of test exercises using mutation techniques, that we call Wodel-Edu. Wodel-Edu is a post-processing extension for the Domain-Specific Language (DSL) Wodel - developed by MISO group - that provides high level primitives for model mutation. We extend the DSL Wodel with new mutation primitives, new selection strategies, a registry of the applied mutations, a duplicated mutant generation control, and a verification that the generated mutants are conforming to the meta-model. We also improve Wodel with an extensible mechanism that allows applying post-processing actions to the generated mutants, and we use this feature to include the Wodel-Edu extension in the Wodel environment. Wodel-Edu is domain independent, and generates three kind of test exercises: the first one, where several diagrams are shown to the student, and he has to choose which one is correct; the second one, where just one diagram is shown to the student, and he has to choose if it is correct or not; and the third kind of exercise, where several changes, that can be applied to the diagram, are presented to the student, and he has to choose which of these changes are correct. In this work, we chose to apply Wodel-Edu to generate finite automata exercises. We also present an evaluation of the generated test application

Un lenguaje de dominio específico para mutación de modelos

Tesis doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Ingeniería Informática. Fecha de lectura: 14-07-2020Las técnicas de mutación de software se utilizan en campos diversos como las pruebas de mutación, la pruebas de programas, la prueba de fórmulas lógicas, los algoritmos genéticos y la generación automática de ejercicios. Las soluciones existentes suelen utilizar un enfoque a medida, construyendo desde cero una solución específica para el problema que pretenden abordar. Este enfoque conlleva un esfuerzo elevado de implementación para cada caso, es propenso a errores, supone un alto coste de mantenimiento, y tiene como resultado la creación de numerosas soluciones cada una de las cuáles sólo resuelve un problema en concreto. Con el objetivo de proporcionar un enfoque genérico que ayude a superar estos inconvenientes, esta tesis presenta un lenguaje de dominio específico para mutación de modelos llamado Wodel, y su entorno de desarrollo. Wodel es independiente del dominio y puede utilizarse con cualquier lenguaje definido por medio de un meta-modelo. Incluye primitivas de mutación para creación, modificación, borrado, retipado y clonado de objetos, y para creación, modificación y borrado de referencias. Wodel proporciona facilidades de soporte al proceso de mutación, como la validación de los mutantes, un registro de las mutaciones aplicadas y la detección de mutantes equivalentes. También da soporte a la ingeniería de operadores de mutación mediante la generación de métricas de mutación y la síntesis automática de modelos semilla que aseguran la cobertura de todas las instrucciones de un programa Wodel, facilitando de este modo su prueba y validación. Además, Wodel es extensible y permite aprovechar sus funcionalidades para la mutación de modelos en aplicaciones de post-procesado. Para ilustrar dicha extensibilidad, esta tesis presenta dos de estas extensiones a Wodel: una para la generación automática de ejercicios de auto-evaluación para estudiantes, a la que se ha llamado Wodel-Edu; y otra para facilitar la creación de herramientas de pruebas de mutación para lenguajes de programación o de modelado, a la que se ha llamado Wodel-TestSoftware mutation techniques are used in different fields such as mutation testing, software testing, logic formulas testing, genetic algorithms and the automated generation of exercises. The existing solutions are usually ad-hoc, creating from scratch a specific solution for the faced problem. This approach involves a high implementation effort, it is error-prone, entails a high maintenance cost, and results in a variety of solutions each of which only solves a particular problem. With the purpose of providing a generic approach that alleviates these inconveniences, this thesis introduces a domain-specific language for model mutation called Wodel, and its development environment. Wodel is domain independent and can be used with any arbitrary language defined by a meta-model. It includes mutation primitives to create, modify, delete, retype and clone objects, and to create, modify and delete references. Wodel provides facilities to simplify the mutation process, such as model validation, a registry of the applied mutations and the detection of equivalent mutants. It also supports the engineering of mutation operators by the generation of metrics of the mutation programs and the automated synthesis of seed models which ensure full coverage of the statements in a Wodel program, hence easing its testing and validation. Additionally, Wodel is extensible and permits taking advantage of all its model mutation functionalities in post-processing applications. To illustrate this extensibility capability, this thesis reports on two of these extensions to Wodel: the first one targeted to the automated generation of self-assessment exercises for students, called Wodel-Edu; and the second one targeted to ease the creation of mutation testing tools for programming or modelling languages, called Wodel-Tes

Automated generation and correction of diagram-based exercises for Moodle

One of the most time‐consuming task for teachers is creating and correcting exercises to evaluate students. This is normally performed by hand, which incurs high time costs and is error‐prone. A way to alleviate this problem is to provide an assistant tool that automates such tasks. In the case of exercises based on diagrams, they can be represented as models to enable their automated model‐based generation for any target environment, like web or mobile applications, or learning platforms like MOODLE. In this paper, we propose an automated process for synthesizing five types of diagram‐based exercises for the MOODLE platform. Being model‐based, our solution is domain‐agnostic (i.e., it can be applied to arbitrary domains like automata, electronics, or software design). We report on its use within a university course on automata theory, as well as evaluations of generality, effectiveness and efficiency, illustrating the benefits of our approachComunidad de Madrid, Grant/Award Number: S2018/TCS‐4314; Ministerio de Ciencia e Innovación, Grant/Award Numbers: PID2021‐ 122270OB‐I00, TED2021‐129381B‐C2

Wodel-Edu: a tool for the generation and evaluation of diagram-based exercises

Creating and grading exercises are recurring tasks within higher education. When these exercises are based on diagrams – like logic circuits, automata or class diagrams – we can represent them as models, and use model-driven engineering techniques for the large-scale generation of quizzes, which can be automatically graded. This way, we propose a domain-independent tool for the generation and automated evaluation of diagram-based exercises called WODEL-EDU. WODEL-EDU is built atop WODEL, an extensible tool for model mutation, and offers seven kinds of diagram exercises. It supports code generation from the exercises for the MOODLE platform, the web, ANDROID and IOS applications. Evaluations from the professor and student perspectives show good resultsSpecial gratitude to Andrés Rico-Fernández and Jaime Velázquez Pazos for their help with the WODEL-EDU implementation, building the code generators for the ANDROID and IOS exercises applications, respectively, and to all participants in the evaluation. Project partially funded by the Spanish MICINN (PID2021-122270OB-I00, TED2021-129381B-C21

Systematic engineering of mutation operators

In the context of software engineering, mutation consists in injecting small changes in artefacts – like models, programs, or data – for purposes like (mutation) testing, test data generation, and all sorts of search-based methods. These tasks typically require defining sets of mutation operators, which are often built ad-hoc because there is currently poor support for their development and testing. To improve this situation, we propose a methodology and corresponding tool support for the proper engineering of mutation operators. Our proposal is model-based, representing the artefacts to be mutated as models. It includes a domain-specific language to describe the mutation operators, facilities to synthesize models that can be used to test the operators, different metrics to analyse operator coverage, and services to generate operators when the coverage is insufficient. We show automated support atop the WODEL tool, and illustrate its use by defining mutation operators for UML Class Diagrams.This work has been partially funded by the Spanish Ministry of Science (RTI2018-095255-B-I00), by the R&D programme of the Madrid Region (S2018/TCS-4314) and by the Spanish MINECO-FEDER (grant number FAME RTI2018-093608-BC31

A tool for domain-independent model mutation

Mutation is a systematic technique to create variants of a seed artefact by means of mutation operators. It has many applications in computer science, like software testing, automatic exercise generation and design space exploration. Typically, mutation frameworks are developed ad-hoc by implementing mutation operators and their application strategies from scratch, using general-purpose programming languages. However, this is costly and error-prone. To improve this situation, we propose WODEL: a domain-specific language and tool for model-based mutation that is independent of the domain meta-model. WODELenables the rapid development and application of model mutations. It provides built-in advanced functionalities like automatic generation of seed models, and static and dynamic metrics of operator coverage and applicability. It offers extension points, e.g., to post-process mutants and describe domain-specific equivalence criteria. As an example, we illustrate the usage of WODEL for the mutation of security policies, and present an empirical evaluation of its expressiveness.Work partially funded by project FLEXOR (Spanish MINECO, TIN2014- 52129-R), project DArDOS (Spanish MINECO/FEDER TIN2015-65845-C3-1- R) and the R&D programme of the Madrid Region (S2013/ICE-3006

How to Apply the User Profile Usability Technique in the User Modelling Activity for an Adaptive Food Recommendation System for People on Special Diets

Interest among software professionals in the possibility of adapting software to user requirements has grown as a result of the evolution of software analysis, design and implementation thinking and the growth in the number of software systems users. Moving away from the traditional approach where the user has to settle for the options offered by software systems, different factors (e.g. user needs, aspirations, preferences, knowledge level, goals) have to be taken into account for this purpose. Technically, this possibility is referred to as adaptiveness, and it requires user data. It is these data (user model) that determine the adaptiveness conditions. Our aim is to build a user model for adaptive systems applied to nutritional requirements, modelling user characteristics that affect their diets and help to improve their health. To build the user model, we apply the user profile usability technique. In order to validate our proposal, we analyse and design a preliminary prototype of an adaptive system capable of making food recommendations to satisfy specific user needs. This study revealed that diet is a propitious field for the development of adaptive systems and that user modelling is a good choice for design of this type of systems

Automated engineering of domain-specific metamorphic testing environments

Context: Testing is essential to improve the correctness of software systems. Metamorphic testing (MT) is an approach especially suited when the system under test lacks oracles, or they are expensive to compute. However, building an MT environment for a particular domain (e.g., cloud simulation, model transformation, machine learning) requires substantial effort. Objective: Our goal is to facilitate the construction of MT environments for specific domains. Method: We propose a model-driven engineering approach to automate the construction of MT environments. Starting from a meta-model capturing the domain concepts, and a description of the domain execution environment, our approach produces an MT environment featuring comprehensive support for the MT process. This includes the definition of domain-specific metamorphic relations, their evaluation, detailed reporting of the testing results, and the automated search-based generation of follow-up test cases. Results: Our method is supported by an extensible platform for Eclipse, called GOTTEN. We demonstrate its effectiveness by creating an MT environment for simulation-based testing of data centres and comparing with existing tools; its suitability to conduct MT processes by replicating previous experiments; and its generality by building another MT environment for video streaming APIs. Conclusion: GOTTEN is the first platform targeted at reducing the development effort of domain-specific MT environments. The environments created with GOTTEN facilitate the specification of metamorphic relations, their evaluation, and the generation of new test casesWork supported by the Madrid government, Spain and the Complutense University (grant PR65/19-22452), the Spanish Ministry of Science (PID2021-122270OB-I00), and the Madrid region, Spain (P2018/TCS-4314, S2018/TCS-4339

Automatic Test Amplification for Executable Models

International audienceBehavioral models are important assets that must be thoroughly verified early in the design process. This can be achieved with manually-written test cases that embed carefully hand-picked domain-specific input data. However, such test cases may not always reach the desired level of quality, such as high coverage or being able to localize faults efficiently. Test amplification is an interesting emergent approach to improve a test suite by automatically generating new test cases out of existing manually-written ones. Yet, while ad-hoc test amplification solutions have been proposed for a few programming languages, no solution currently exists for amplifying the test cases of behavioral models. In this paper, we fill this gap with an automated and generic approach. Given an executable DSL, a conforming behavioral model, and an existing test suite, our approach generates new regression test cases in three steps: (i) generating new test inputs by applying a set of generic modifiers on the existing test inputs; (ii) running the model under test with new inputs and generating assertions from the execution traces; and (iii) selecting the new test cases that increase the mutation score. We provide tool support for the approach atop the Eclipse GEMOC Studio and show its applicability in an empirical study. In the experiment, we applied the approach to 71 test suites written for models conforming to two different DSLs, and for 67 of the 71 cases, it successfully improved the mutation score between 3.17 % and 54.11 % depending on the initial setup. CCS CONCEPTS • Software and its engineering → Software testing and debugging; Model-driven software engineering