Automated free text marking with Paperless School

The Paperless School automarking system utilises a number of novel approaches to address the challenge of providing both summative and formative assessments with little or no human intervention. The Paperless School system is designed primarily for day-to-day, low stakes testing of essay and short-text student inputs. It intentionally sacrifices some degree of accuracy to achieve ease of set up, but nevertheless provides an accurate view of the abilities of each student by averaging marks over a number of essays. The system is designed to function as a back-end service to an Learning Management System (LMS), thus facilitating the marking of large numbers of texts. This should enable considerable teacher resources to be freed up for other teaching tasks. In this paper we will discuss some of the issues involved in bringing computational linguistics to bear in the educational context. We will cover • how Blooms Taxonomy (the pedagogical model underlying most formal grading schemes) can be represented in software. • an overview of the steps required to derive a grade that will sufficiently closely predict the grade a human marker would give. • extending the system to include formative assessment, via intelligent comment banks

Focus Group. Building a Compliance Audit for BS7988 "Code of Practice for the Use of Information Technology (IT) for the Delivery of Assessments

The BS7988 standard is now formally published, and is broadly in line with the provisional specification presented by John Kleeman, David Keech and Stephen Wright to this conference last year. Compliance with the standard requires quality assurance across a wide range of areas - hardware, software, network, data storage and human issues are all covered. It requires consideration of security, reliability and accessibility. In this session we will work collectively to build a provisional framework for a QA system to cover those areas of the Standard that fall within the remit of conference delegates, principally those relating to software development. The output of the session will be a draft of a quality audit that can be published as part of the conference proceedings. While clearly this will be very much a first-stage document, we hope that it will provide pointers for further work in this area