Experiments in the automatic marking of ER-Diagrams

In this paper we present an approach to the computer understanding of diagrams and show how it can be successfully applied to the automatic marking (grading) of student attempts at drawing entity-relationship (ER) diagrams. The automatic marker has been incorporated into a revision tool to enable students to practice diagramming and obtain feedback on their attempts

Using patterns in the automatic marking of ER-Diagrams

This paper illustrates how the notion of pattern can be used in the automatic analysis and synthesis of diagrams, applied particularly to the automatic marking of ER-diagrams. The paper describes how diagram patterns fit into a general framework for diagram interpretation and provides examples of how patterns can be exploited in other fields. Diagram patterns are defined and specified within the area of ER-diagrams. The paper also shows how patterns are being exploited in a revision tool for understanding ER-diagrams

Automatic assessment of sequence diagrams

In previous work we showed how student-produced entity-relationship diagrams (ERDs) could be automatically marked with good accuracy when compared with human markers. In this paper we report how effective the same techniques are when applied to syntactically similar UML sequence diagrams and discuss some issues that arise which did not occur with ERDs. We have found that, on a corpus of 100 student-drawn sequence diagrams, the automatic marking technique is more reliable that human markers. In addition, an analysis of this corpus revealed significant syntax errors in student-drawn sequence diagrams. We used the information obtained from the analysis to build a tool that not only detects syntax errors but also provides feedback in diagrammatic form. The tool has been extended to incorporate the automatic marker to provide a revision tool for learning how to model with sequence diagrams

Semi-automatic assessment of students' graph-based diagrams

Diagrams are increasingly used in many design methods, and are being taught in a variety of contexts in higher education such as database conceptual design or software design in computer science. They are an important part of many assessments. Currently computer aided assessments are widely used for multiple choice questions. They lack the ability to assess a student’s knowledge in a more comprehensive way, which is required for diagram-type student work. The aim of this research is to develop a semi-automatic assessment framework, which enables the use of computer to support the assessment process of diagrammatic solutions, with the focus of ensuring the consistency of grades and feedback on solutions. A novel trace model, that captures design traces of student solutions, was developed as a part of the framework and was used to provide the matching criteria for grouping the solutions. A new marking style, partial marking, was developed to mark these solution groups manually. The Case-Based Reasoning method is utilised in the framework to mark some of the groups automatically. A guideline for scenario writing was proposed to increase the efficiency of automatic marking. A prototype diagram editor, a marking tool and scenario writing environment were implemented for the proposed framework in order to demonstrate proof of concept. The results of experiments show that the framework is feasible to use in the formative assessment and it provides consistent marking and personalised feedback to the students. The framework also has the potential to significantly reduce the time and effort required by the examiner to mark student diagrams. Although the constructed framework was specifically used for the assessment of database diagrams, the framework is generic enough to be used for other types of graph-based diagram

Automatically assessing free-form diagrams in e-assessment systems

In this paper, we describe the advances we have made in extending the quiz component of our VLE (Moodle) to support the drawing and automatic assessment of free-form diagrams. The paper illustrates how diagram-based questions are incorporated into normal Moodle quizzes and the nature of the feedback provided. It describes the types of diagrams that can be supported by the system (graph-based) and outlines how the automatic marking is performed. We briefly summarise the results of our tests of the marking algorithm which show that it performs well. Finally, the paper outlines two important support applications: one for specifying the type of a diagram to be used in a quiz and one for creating questions, model solutions and marking schemes for diagram questions

A diagram drawing tool for semi–automatic assessment of conceptual database diagrams

The increased number of diagram based questions in higher education has recently attracted researchers to look into marking diagrams automatically. Student diagrammatic solutions are naturally very dissimilar to each others. However, it has been observed that there are a number of identical diagram components. This observation forms the basis of our semi–automatic assessment. Identifying identical diagram components in student diagrams needs contextual information about each component. This paper proposes a diagram tool which obtains the contextual information of each component in a conceptual database diagram

Drawing and marking graph diagrams

The marking of graph diagrams (that is to say diagrams that are composed of nodes, possibly joined by edges) is tedious if the diagrams are presented on paper. If the key content of the diagrams is available in electronic form then the marking can be much more efficient. This is achieved because the tutor only has to mark each different diagram element once and this mark is transmitted to all diagrams that contain the element. This benefit to the tutor is obtained by requiring the students to use a diagram drawing program of some kind. However using such an editor can simplify the process for the students by allowing them to concentrate more on the problem and less on its graphical representation. The students can also be rewarded for going to this extra effort by receiving a much more detailed, personalised commentary on their work than would have been possible before, given the same amount of tutor time. We present the evolution of a drag-and-drop diagram editor specialised for the area of ER diagrams and an associated marking system with a simple but effective feedback mechanism. Some results from initial trials are presented along with some ideas for improvement and extension

A multi-touch ER diagram editor to capture students' design rationale

The increased presence of diagram-type student work in higher education has recently attracted researchers to look into the automation of diagram marking. Research into the semi-automatic diagram assessment at Loughborough University has identified the requirements of a diagram editor in order to capture the students’ design rationale. To fulfill these requirements, several experimental diagram editors have been developed. This paper introduces an ER diagram editor which uses multi touch technology. The initial experiments and findings for the editor are described in the paper

Personalised feedback with semi-automated assessment tool for conceptual database model

The increased presence of diagram-type student work in higher education has recently attracted researchers to look into the automation of diagram marking. This paper introduces web-based diagram drawing and marking tools for a new (semi-automatic) assessment approach. The approach reduces the number of diagram components marked by the human marker and provides individualised and detailed feedback to students. The tools which have been used in tutorials of a first year database module in the Computer Science department at Loughborough University are described together with findings from the usage of the tools