A unique style of computer‐assisted assessment

This paper examines a project at the University of Wolverhampton that has been producing its own unique style of computerized test for several years. The tests are all designed to deliver a different set of questions each time they are run, a fact which enables many of them to double as learning resources. Most of the tests are used for both formative and summative assessments on Level 1 modules, in conjunction with more traditional assessment methods

Automatic Image Marking Process

Abstract-Efficient evaluation of student programs and timely processing of feedback is a critical challenge for faculty. Despite persistent efforts and significant advances in this field, there is still room for improvement. Therefore, the present study aims to analyse the system of automatic assessment and marking of computer science programming students’ assignments in order to save teachers or lecturers time and effort. This is because the answers are marked automatically and the results returned within a very short period of time. The study develops a statistical framework to relate image keywords to image characteristics based on optical character recognition (OCR) and then provides analysis by comparing the students’ submitted answers with the optimal results. This method is based on Latent Semantic Analysis (LSA), and the experimental results achieve high efficiency and more accuracy by using such a simple yet effective technique in automatic marking

VPL: laboratorio virtual de programación para Moodle

En este trabajo se presenta VPL (Virtual Programming Lab), una herramienta software de código abierto que permite la gestión de prácticas de programación en Moodle. Esta herramienta está compuesta de un módulo Moodle, un applet editor de código fuente y un demonio Linux que permite la ejecución remota de programas de forma segura. Pretende ahorrar tiempo y mejorar la gestión general de este tipo de actividades, además de permitir la realización de las prácticas sólo con un navegador. Sus características más destacadas son: la posibilidad de editar el código fuente y ejecutar las prácticas de forma interactiva desde el navegador, ejecutar pruebas que revisen las prácticas y analizar la similitud entre prácticas para el control del plagio. Esta herramienta se ha empleado con éxito en diversas asignaturas durante el año 2009. VPL está disponible para su descarga en http://vpl.dis.ulpgc.es.Peer Reviewe

Automatic Marking with Sakai

Large student numbers often drive teaching staff to consider greater degrees of automation of assessment activities. In introductory Computer Science classes - where submitted programs need to repeatedly be compiled, executed and tested - automation is an obvious route to investigate. This paper reports on an experimental automation system for assessing programming assignments, and its integration with the open source Sakai learning management system. While the system has been an administrative success, feedback from students has identified numerous areas for improvement at the interface of the student and the automatic marker. Furthermore, the use of automation has highlighted the need for teaching software development methodology from an early stage

The automatic assessment of Z specifications

The need to automate the process of assessing a specification in a learning environment is identified to be one of the fundamental ways to improve the use of formal notation in specifying a real system. General issues involved in building an automatic marking system for computer-based courses are explored. Techniques that have been proposed for assessing a specification are also discussed. By considering the issues and the techniques, we describe how they can be used to build a system that is able to give a quality grade to a specification that is written in the Z language. In the system, four quality factors are taken into consideration; maintainability of a specification (which considers the typographic arrangement of a specification and the specification complexity), and correctness of a specification (which reflects the static correctness and the dynamic correctness of a specification). By using suitable quality metrics for specification maintainability, the results that are produced are compared to some values which can either be absolute values or relative to the model answer. The marks awarded for this factor are based on this comparison. Static correctness is carried out by applying a syntax and type checker. The marks granted for this factor depend on the outcome of the checker. Dynamic correctness is determined by employing a testing technique. In the context of a specification, the behaviour of a system-state, which is represented by so-called state variables, is analysed. The specification is 'executed' by using animation. The marks are given according to the correctness of the output and the final state. The system is implemented within the well-known courseware management system, Ceilidh. There are fundamental differences between Z specifications, and the subject matter of other courses taught using the Ceilidh system (which are mostly computer programming courses). For this reason we take some time in this thesis to explain (in some detail) the incorporation of the system within Ceilidh. The need for the fundamental components (i.e the editor, the syntax and type checker, the animator and the automatic marker) are discussed and described. The system has been used by a group of 13 students who attended a Z course within the School of Computer Science and Information Technology at the University of Nottingham during the 1997-1998 academic year. The students were given a questionnaire about the system. An analysis of these questionnaires shows that the currently implemented tools are beneficial and helpful to the students. We also test the results of the system and compare them with a small selected group of human markers. The testing reveals very encouraging results and shows that the system can mark student scripts with a good degree of accuracy. We conclude that this system can provide a very useful aid for teachers of the Z Specification language

The automatic assessment of Z specifications

The need to automate the process of assessing a specification in a learning environment is identified to be one of the fundamental ways to improve the use of formal notation in specifying a real system. General issues involved in building an automatic marking system for computer-based courses are explored. Techniques that have been proposed for assessing a specification are also discussed. By considering the issues and the techniques, we describe how they can be used to build a system that is able to give a quality grade to a specification that is written in the Z language. In the system, four quality factors are taken into consideration; maintainability of a specification (which considers the typographic arrangement of a specification and the specification complexity), and correctness of a specification (which reflects the static correctness and the dynamic correctness of a specification). By using suitable quality metrics for specification maintainability, the results that are produced are compared to some values which can either be absolute values or relative to the model answer. The marks awarded for this factor are based on this comparison. Static correctness is carried out by applying a syntax and type checker. The marks granted for this factor depend on the outcome of the checker. Dynamic correctness is determined by employing a testing technique. In the context of a specification, the behaviour of a system-state, which is represented by so-called state variables, is analysed. The specification is 'executed' by using animation. The marks are given according to the correctness of the output and the final state. The system is implemented within the well-known courseware management system, Ceilidh. There are fundamental differences between Z specifications, and the subject matter of other courses taught using the Ceilidh system (which are mostly computer programming courses). For this reason we take some time in this thesis to explain (in some detail) the incorporation of the system within Ceilidh. The need for the fundamental components (i.e the editor, the syntax and type checker, the animator and the automatic marker) are discussed and described. The system has been used by a group of 13 students who attended a Z course within the School of Computer Science and Information Technology at the University of Nottingham during the 1997-1998 academic year. The students were given a questionnaire about the system. An analysis of these questionnaires shows that the currently implemented tools are beneficial and helpful to the students. We also test the results of the system and compare them with a small selected group of human markers. The testing reveals very encouraging results and shows that the system can mark student scripts with a good degree of accuracy. We conclude that this system can provide a very useful aid for teachers of the Z Specification language

VPL: Laboratorio Virtual de Programación para Moodle

En este trabajo se presenta VPL (Virtual Programming Lab), una herramienta software de código abierto que permite la gestión de prácticas de programación en Moodle. Esta herramienta está compuesta de un módulo Moodle, un applet editor de código fuente y un demonio Linux que permite la ejecución remota de programas de forma segura. Pretende ahorrar tiempo y mejorar la gestión general de este tipo de actividades, además de permitir la realización de las prácticas sólo con un navegador. Sus características más destacadas son: la posibilidad de editar el código fuente y ejecutar las prácticas de forma interactiva desde el navegador, ejecutar pruebas que revisen las prácticas y analizar la similitud entre prácticas para el control del plagio. Esta herramienta se ha empleado con éxito en diversas asignaturas durante el año 2009. VPL está disponible para su descarga en http://vpl.dis.ulpgc.es

Use of proofs-as-programs to build an anology-based functional program editor

This thesis presents a novel application of the technique known as proofs-as-programs. Proofs-as-programs defines a correspondence between proofs in a constructive logic and functional programs. By using this correspondence, a functional program may be represented directly as the proof of a specification and so the program may be analysed within this proof framework. CʸNTHIA is a program editor for the functional language ML which uses proofs-as-programs to analyse users' programs as they are written. So that the user requires no knowledge of proof theory, the underlying proof representation is completely hidden. The proof framework allows programs written in CʸNTHIA to be checked to be syntactically correct, well-typed, well-defined and terminating. CʸNTHIA also embodies the idea of programming by analogy — rather than starting from scratch, users always begin with an existing function definition. They then apply a sequence of high-level editing commands which transform this starting definition into the one required. These commands preserve correctness and also increase programming efficiency by automating commonly occurring steps. The design and implementation of CʸNTHIA is described and its role as a novice programming environment is investigated. Use by experts is possible but only a sub-set of ML is currently supported. Two major trials of CʸNTHIA have shown that CʸNTHIA is well-suited as a teaching tool. Users of CʸNTHIA make fewer programming errors and the feedback facilities of CʸNTHIA mean that it is easier to track down the source of errors when they do occur

Computer-based assessment: A versatile educational tool

Abstract There are many types of Computer-Based Assessment in use today in higher education, from formative self-tests to summative ®nal exams. CBA is used in various ways as an integral part of many currently used Computer Aided Learning environments. This paper surveys some of the reasons for using computers for assessment and some of the types of test in use, including evidence that coverage can extend beyond ®rst year assessments, where it seems to be stuck in some disciplines. One project and one type are examined in more detail: randomly generated open access tests. With this type of test students are allowed to practise in their own time before sitting the same test for a grade. Evidence from one particular test, in statistics, was taken from computer logs, questionnaires and individual interviews and is used to show that the test style motivated students to study, and, for some students, directed their revision even when they were away from the computer. The results show that random-based tests can have a number of major advantages over ®xed assessments, including: increased lifespan, security and exibility, improved student motivation for study; and use as a learning resource. They also indicate that CBA does not have to be viewed in isolation from the learning environment in which it is situated, but can have an impact upon a student's study strategy, for example through increased revision. Taken together with previous studies and the survey of uses, there is now a body of evidence to suggest that CBA is an extremely versatile educational tool.