Mechanix: An Intelligent Web Interface for Automatic Grading of Sketched Free-Body Diagrams

Sketching free body diagrams is an essential skill that students learn in introductory physics and engineering classes; however, university class sizes are growing and often have hundreds of students in a single class. This situation creates a grading challenge for instructors as there is simply not enough time nor resources to provide adequate feedback on every problem. We have developed a web-based application called Mechanix to provide automated real-time feedback on hand-drawn free body diagrams for students. The system is driven by novel sketch recognition algorithms developed for recognizing and comparing trusses, general shapes, and arrows in diagrams. We have discovered students perform as well as paper homework or other online homework systems which only check the final answer through deployment to five universities with 450 students completing homework on the system over the 2018 and 2019 school years. Mechanix has reduced the amount of manual grading required for instructors in those courses while ensuring students can correctly draw the free body diagram

Wodel-Edu: a tool for the generation and evaluation of diagram-based exercises

Creating and grading exercises are recurring tasks within higher education. When these exercises are based on diagrams – like logic circuits, automata or class diagrams – we can represent them as models, and use model-driven engineering techniques for the large-scale generation of quizzes, which can be automatically graded. This way, we propose a domain-independent tool for the generation and automated evaluation of diagram-based exercises called WODEL-EDU. WODEL-EDU is built atop WODEL, an extensible tool for model mutation, and offers seven kinds of diagram exercises. It supports code generation from the exercises for the MOODLE platform, the web, ANDROID and IOS applications. Evaluations from the professor and student perspectives show good resultsSpecial gratitude to Andrés Rico-Fernández and Jaime Velázquez Pazos for their help with the WODEL-EDU implementation, building the code generators for the ANDROID and IOS exercises applications, respectively, and to all participants in the evaluation. Project partially funded by the Spanish MICINN (PID2021-122270OB-I00, TED2021-129381B-C21

Automatic assessment of sequence diagrams

In previous work we showed how student-produced entity-relationship diagrams (ERDs) could be automatically marked with good accuracy when compared with human markers. In this paper we report how effective the same techniques are when applied to syntactically similar UML sequence diagrams and discuss some issues that arise which did not occur with ERDs. We have found that, on a corpus of 100 student-drawn sequence diagrams, the automatic marking technique is more reliable that human markers. In addition, an analysis of this corpus revealed significant syntax errors in student-drawn sequence diagrams. We used the information obtained from the analysis to build a tool that not only detects syntax errors but also provides feedback in diagrammatic form. The tool has been extended to incorporate the automatic marker to provide a revision tool for learning how to model with sequence diagrams

Platform Architecture for the Diagram Assessment Domain

Using e-learning and e-assessment environments in higher education bears considerable potential for both students and teachers. In this contribution we present an architecture for a comprehensive e-assessment platform for the modeling domain. The platform – currently developed in the KEA-Mod project – features a micro-service architecture and is based on different inter-operable components. Based on this idea, the KEA-Mod platform will provide e-assessment capabilities for various graph-based modeling languages such as Unified Modeling Language (UML), EntityRelationship diagrams (ERD), Petri Nets, Event-driven Process Chains (EPC) and the Business Process Model and Notation (BPMN) and their respective diagram types

Exploring Feedback and Gamification in a Data Modeling Learning Tool

Data modeling is an essential part of IT studies. Learning how to design and structure a database is important when storing data in a relational database and is common practice in the IT industry. Most students need much practice and tutoring to master the skill of data modeling and database design. When a student is in a learning process, feedback is important. As class sizes grow and teaching is no longer campus based only, providing feedback to each individual student may be difficult. Our study proposes a tool to use when introducing database modeling to students. We have developed a web-based tool named LearnER to teach basic data modeling skills, in a collaborative project between the University of South-Eastern Norway (USN) and Kristiania University College (KUC). The tool has been used in six different courses over a period of four academic years. In LearnER, the student solves modeling assignments with different levels of difficulty. When they are done, or they need help, they receive automated feedback including visual cues. To increase the motivation for solving many assignments, LearnER also includes gamifying elements. Each assignment has a maximum score. When students ask for help, points are deducted from the score. When students manage to solve many assignments with little help, they may end up at a leaderboard. This paper tries to summarize how the students use and experience LearnER. We look to see if the students find the exercises interesting, useful and of reasonable difficulty. Further, we investigate if the automated feedback is valuable, and if the gamifying elements contribute to their learning. As we have made additions and refinements to LearnER over several years, we also compare student responses on surveys and interviews during these years. In addition, we analyze usage data extracted from the application to learn more about student activity. The results are promising. We find that student activity increases in newer versions of LearnER. Most students report that the received feedback helps them to correct mistakes when solving modeling assignments. The gamifying elements are also well received. Based on LearnER usage data, we find and describe typical errors the students do and what types of assignments they prefer to solve.publishedVersio

Extending Artemis With a Rule-Based Approach for Automatically Assessing Modeling Tasks

The Technische Universität Dresden has multiple e-learning projects in use. The Chair of Software Technology uses Inloop to teach students object-oriented programming through automatic feedback. In the last years, interest has grown in giving students automated feedback on modeling tasks. This is why there was an extension developed by Hamann to automate the assessment of modeling tasks in 2020. The TU Dresden currently has plans to replace Inloop with Artemis, a comparable system. Artemis currently supports the semi-automatic assessment of modeling exercises. In contrast, the system proposed by Hamann, called Inloom, is based on a rule-based approach and provides instant feedback. A rule-based system has certain advantages over a similarity-based system. One advantage is the mostly better feedback that these systems generate. To give instructors more flexibility and choice, this work tries to identify possible ways of extending Artemis with the rule-based approach Inloom. In the second step, this thesis will provide a proof of concept implementation. Furthermore, a comparison between different systems is developed to help instructors choose the best suitable system for their usecase.:Introduction, Background, Related Work, Analysis, System Design, Implementation, Evaluation, Conclusion and Future Work, Bibliography, Appendi

An Automated Grading and Feedback System for a Computer Literacy Course

Computer Science departments typically offer a computer literacy course that targets a general lay audience. At Appalachian State University, this course is CS1410 - Introduction to Computer Applications. computer literacy courses have students work with various desktop and web-based software applications, including standard office applications. CS1410 strives to have students use well known applications in new and challenging ways, as well as exposing them to some unfamiliar applications. These courses can draw large enrollments which impacts efficient and consistent grading. This thesis describes the development and successful deployment of the Automated Grading And Feedback (AGAF) system for CS1410. Specifically, a suite of automated grading tools targeting the different types of CS1410 assignments has been built. The AGAF system tools have been used on actual CS1410 submissions and the resulting grades were verified. AGAF tools exist for Microsoft Office assignments requiring students to upload a submission file. Another AGAF tool accepts a student “online text submission” where the text encodes the URL of a Survey Monkey survey and a blog. Other CS1410 assignments require students to upload an image file. AGAF can process images in multiple ways, including decoding of a QR two-dimensional barcode and identification of an expected image pattern

Automated generation and correction of diagram-based exercises for Moodle

One of the most time‐consuming task for teachers is creating and correcting exercises to evaluate students. This is normally performed by hand, which incurs high time costs and is error‐prone. A way to alleviate this problem is to provide an assistant tool that automates such tasks. In the case of exercises based on diagrams, they can be represented as models to enable their automated model‐based generation for any target environment, like web or mobile applications, or learning platforms like MOODLE. In this paper, we propose an automated process for synthesizing five types of diagram‐based exercises for the MOODLE platform. Being model‐based, our solution is domain‐agnostic (i.e., it can be applied to arbitrary domains like automata, electronics, or software design). We report on its use within a university course on automata theory, as well as evaluations of generality, effectiveness and efficiency, illustrating the benefits of our approachComunidad de Madrid, Grant/Award Number: S2018/TCS‐4314; Ministerio de Ciencia e Innovación, Grant/Award Numbers: PID2021‐ 122270OB‐I00, TED2021‐129381B‐C2