Automatic assessment of sequence diagrams

In previous work we showed how student-produced entity-relationship diagrams (ERDs) could be automatically marked with good accuracy when compared with human markers. In this paper we report how effective the same techniques are when applied to syntactically similar UML sequence diagrams and discuss some issues that arise which did not occur with ERDs. We have found that, on a corpus of 100 student-drawn sequence diagrams, the automatic marking technique is more reliable that human markers. In addition, an analysis of this corpus revealed significant syntax errors in student-drawn sequence diagrams. We used the information obtained from the analysis to build a tool that not only detects syntax errors but also provides feedback in diagrammatic form. The tool has been extended to incorporate the automatic marker to provide a revision tool for learning how to model with sequence diagrams

DSpot: Test Amplification for Automatic Assessment of Computational Diversity

Context: Computational diversity, i.e., the presence of a set of programs that all perform compatible services but that exhibit behavioral differences under certain conditions, is essential for fault tolerance and security. Objective: We aim at proposing an approach for automatically assessing the presence of computational diversity. In this work, computationally diverse variants are defined as (i) sharing the same API, (ii) behaving the same according to an input-output based specification (a test-suite) and (iii) exhibiting observable differences when they run outside the specified input space. Method: Our technique relies on test amplification. We propose source code transformations on test cases to explore the input domain and systematically sense the observation domain. We quantify computational diversity as the dissimilarity between observations on inputs that are outside the specified domain. Results: We run our experiments on 472 variants of 7 classes from open-source, large and thoroughly tested Java classes. Our test amplification multiplies by ten the number of input points in the test suite and is effective at detecting software diversity. Conclusion: The key insights of this study are: the systematic exploration of the observable output space of a class provides new insights about its degree of encapsulation; the behavioral diversity that we observe originates from areas of the code that are characterized by their flexibility (caching, checking, formatting, etc.).Comment: 12 page

Automatic assessment of creativity in heuristic problem-solving based on query diversity

Indexación: Web of Science; Scopus.Research, development and innovation are the pillars on which companies rely to offer new products and services capable of attracting consumer demand. This is why creative problem-solving emerges as one of the most relevant skills of the 21st century. Fortunately, there are many creativity training programs that have proven effective. However, many of these programs and methods base on a previous measurement of creativity and require experienced reviewers, they consume time for being manual, and they are far from everyday activities. In this study, we propose a model to estimate the creative quality of users' solutions dealing with heuristic problems, based on the automatic analysis of query patterns issued during the information search to solve the problem. This model has been able to predict the creative quality of solutions produced by 226 users, reaching a sensitivity of 78.43%. Likewise, the level of agreement among reviewers in relation to the creative characteristics is evaluated through two rubrics, and thereby, observing the difficulties of the manual evaluation: subjectivity and effort. The proposed model could be used to foster prompt detection of non-creative solutions and it could be implemented in diverse industrial processes that can range from the recruitment of talent to the evaluation of performance in R&D&I processes.https://www.revistadyna.com/search/automatic-assessment-of-creativity-in-heuristic-problem-solving-based-on-query-diversit