Automatically assessing free-form diagrams in e-assessment systems

In this paper, we describe the advances we have made in extending the quiz component of our VLE (Moodle) to support the drawing and automatic assessment of free-form diagrams. The paper illustrates how diagram-based questions are incorporated into normal Moodle quizzes and the nature of the feedback provided. It describes the types of diagrams that can be supported by the system (graph-based) and outlines how the automatic marking is performed. We briefly summarise the results of our tests of the marking algorithm which show that it performs well. Finally, the paper outlines two important support applications: one for specifying the type of a diagram to be used in a quiz and one for creating questions, model solutions and marking schemes for diagram questions

The Automatic Assessment of Multiple Artefacts: An Investigation into Design Diagrams and Their Implementations

As the Higher Education sector has moved towards student-centred learning so too has the growth in electronic support for learning. E-assessment has been a part of this growth as increasingly assessment and its feedback is seen as an integral part of the students’ learning process. Mature e-assessment systems exist, particularly where answers to questions are restricted to a prescribed list of alternatives. However, for free response artefacts, where there is a limited restriction placed on answers to questions, automated assessment systems are embryonic. This dissertation presents an investigation into the automated assessment of free response artefacts. Design diagrams and their accompanying source code implementations are examples of free response artefacts. A case study is developed that investigates how to automatically generate formative feedback for a design diagram by utilizing its accompanying implementation. The dissertation presents a two-staged solution, initially analysing the design diagram in isolation before comparing it with the implementation. A framework for this approach has been developed and tested using a tool applied to coursework submitted by undergraduate computer science students. The tool was evaluated by comparing the formative feedback comments generated by the tool with those produced by a team of computer science educators. Evaluation was undertaken via two Likert questionnaires, one completed by students and one completed by a team of computer scientists. The results presented are favourable, with the majority of comments produced by the tool being seen to be as least as good as those generated by the computer science educators

Diagram interpretation and e-learning systems

We describe a system capable of grading free-form diagrammatic answers. Our system matches meaningful parts of a diagram with equivalents in a model solution. This is complicated by errors, omissions, and superfluous items in the student answer. The result of matcing is used to calculate the grade and generate appropritae feedback; it performs at least as well as a human marker on a variety of diagram types. We describe tools that allow the easy creation of questions, marking schemees, and diagram editors suitalbe for embedding in a VLE quiz engine