EVALUATING SEMANTIC ANALYSIS METHODS FOR SHORT ANSWER GRADING USING LINEAR REGRESSION

The assessment of free-text answers may demand significant human effort, especially in scenarios with many students. This paper focuses on the automatic grading of short answer written in Portuguese language using techniques of natural language processing and semantic analysis. A previous study found that a similarity scoring model might be more suitable to a question type than to another. In this study, we combine latent semantic analysis (LSA) and a WordNet path-based similarity method using linear regression to predict scores for 76 short answers to three questions written by high school students. The predicted scores compared well to human scores and the use of combined similarity scores showed an improvement in overall results in relation to a previous study on the same corpus. The presented approach may be used to support the automatic grading of short answer using supervised machine learning to weight different similarity scoring models. &nbsp

Adapting the automatic assessment of free-text answers to the students

In this paper, we present the first approach in the field of Computer Assisted Assessment (CAA) of students' free-text answers to model the student profiles. This approach has been implemented in a new version of Atenea, a system able to automatically assess students' short answers. The system has been improved so that it is now able to take into account the students' preferences and personal features to adapt not only the assessment process but also to personalize the appearance of the interface. In particular, it is now able to accept students’ answers written in Spanish or in English indistinctly, by means of Machine Translation. Moreover, we have observed that Atenea’s performance does not decrease drastically when combined with automatic translation, provided that the translation does not reduce greatly the variability in the vocabulary

Question types in english language diagnostic testing

English language proficiency testing, like large-scale testing in many other domains, often uses multiple-choice questions, to exploit the efficiency of automatic marking. An experiment supplementing established mcq tests with very short free-text answer questions in English diagnostic testing has shown that the latter are better discriminators at the lower end of student ability. Although not economic with paper-based marking on a large scale, the Assess By Computer e-assessment software offers marking options for such answers which make constructed-answer tests a realistic option

Toward the automated assessment of entity-relationship diagrams

The need to interpret imprecise diagrams (those with malformed, missing or extraneous features) occurs in the automated assessment of diagrams. We outline our proposal for an architecture to enable the interpretation of imprecise diagrams. We discuss our preliminary work on an assessment tool, developed within this architecture, for automatically grading answers to a computer architecture examination question. Early indications are that performance is similar to that of human markers. We will be using Entity-Relationship Diagrams (ERDs) as the primary application area for our investigation of automated assessment. This paper will detail our reasons for choosing this area and outline the work ahead

Using patterns in the automatic marking of ER-Diagrams

This paper illustrates how the notion of pattern can be used in the automatic analysis and synthesis of diagrams, applied particularly to the automatic marking of ER-diagrams. The paper describes how diagram patterns fit into a general framework for diagram interpretation and provides examples of how patterns can be exploited in other fields. Diagram patterns are defined and specified within the area of ER-diagrams. The paper also shows how patterns are being exploited in a revision tool for understanding ER-diagrams

Experiments in the automatic marking of ER-Diagrams

In this paper we present an approach to the computer understanding of diagrams and show how it can be successfully applied to the automatic marking (grading) of student attempts at drawing entity-relationship (ER) diagrams. The automatic marker has been incorporated into a revision tool to enable students to practice diagramming and obtain feedback on their attempts