The Preface for the first special issue for the International Journal of e-Assessment highlights the role of the assessment for learning agenda. It also points to papers within the special issue which relate to:

(a) Overcoming the barriers to supporting student learning with computer-assisted assessment; (b) Marking and computer-assisted assessment; (c) Harnessing the technology to improve efficiency and pedagogy.

The papers in this issue stress the general recognition that times are changing and that assessment needs to become embedded in the teaching/learning cycle and not purely as a checking device for the awarding institution. These papers encourage us to rise to the challenge of developing more sophisticated computer-assisted assessment systems that support student learning and there is still a lot to do

Warburton, Bill

Whitelock, Denise

English

Open Research Online

Open Research OnlineThe Open University’s repository of research publicationsand other research outputsPreface : CAA 2010: Computer assisted assessment:supporting student learningJournal ItemHow to cite:Whitelock, Denise and Warburton, Bill (2011). Preface : CAA 2010: Computer assisted assessment: supportingstudent learning. International Journal of e-Assessment, 1(1)For guidance on citations see FAQs.c© 2011 The AuthorsVersion: Accepted ManuscriptLink(s) to article on publisher’s website:http://journals.sfu.ca/ijea/index.php/journal/article/viewFile/6/5Copyright and Moral Rights for the articles on this site are retained by the individual authors and/or other copyrightowners. For more information on Open Research Online’s data policy on reuse of materials please consult the policiespage.oro.open.ac.ukPreface CAA 2010: Computer assisted assessment: supporting student learning  Guest Editors: Denise Whitelock, The Open University and Bill Warburton, University of Southampton  Assessment for learning The assessment for learning movement which sprang into action after a series of deliberations by the Assessment Reform Group in 2002 wanted to achieve a better alignment between teaching, learning and assessment. They defined the term ‘assessment for learning’ as ’the process of seeking and interpreting evidence for use by learners and their teachers to decide where the learners are in their learning, where they need to go and how best to get there’ (ARG 2002). It is acknowledged that feedback plays a crucial role in supporting the student learning process and computer-assisted assessment has played an important role in delivering timely and effective feedback to students since the early introduction of multiple-choice questions through to more current sophisticated and innovative forms of e-assessment (Whitelock et al. 2006). The recognition that times are changing and that assessment needs to become embedded within the teaching/learning cycle has also meant that not only the assessment tasks must change but also the whole way we conduct our examinations using technology requires a major rethink. For is it not unreasonable in this day and age to ask students to handwrite their examination answers when they word process all their other written assignments (Bennett 2002; Mogey et al; 2007; Warburton 2009)?  Even though some progress has been made since Whitelock and Brasher (2006) made a number of recommendations about how to achieve institutional and individual support for a more widespread introduction of computer-assisted assessment there are still a number of barriers that need to be overcome and the first set of papers in this edition of IJEA address these pertinent issues. Overcoming the barriers to supporting student learning with computer-assisted assessment To set the scene for this discussion, Mogey (this issue) highlights  a number of barriers identified to date, such as time, staff training and student attitudes to computer-assisted assessment, which prevent  the adoption of e-assessment. However, she suggests it is the lack of physical spaces for e-assessment that is the most significant obstruction to widespread uptake of this form of assessment. Mogey builds on Hunley and Schaller’s notion (2009) that assessment is the key to creating spaces for learning. She argues that changing assessment spaces will also transform pedagogical practice and perhaps then we will be able to take more advantage of Web 2.0 tools for assessment as advocated by Whitelock (2010). Changing physical spaces for computer-assisted assessment leads to another issue, which concerns many awarding bodies, and concerns the verification of user identity, especially for distance learners. Impersonation in these conditions cannot be considered as accidental (Stoner 1995) and is perceived as a major risk (Quinn et al. International Journal of e-Assessment vol.1 no.1 2011  22003) to the adoption of e-assessment. Apampa et al. (this issue) suggest a more novel approach to authentication through using a blob-based presence verification system for summative e-assessment. The advantage of their approach is that it is a non- interruptive reauthentication process that requires low processing power that is implemented with the assistance of presence monitoring software, which takes advantage of processing low numbers of moving objects (Zang and Klette 2003). Ampana et al.’s findings suggest the adoption of blob-based authentication systems can allay identification fears associated with summative e-assessments. Making sure we have a level playing field for all the examination candidates undertaking e-assessments also entails overcoming another barrier and that is to ensure all cohorts of students receive equivalent tests when the questions are selected at random from a common item bank. One solution to this problem, which has been researched by Dermo (this issue), is to use the ‘OSIRIS (Objective Standardization in Random item Selection) method of modifying student grades based on the difficulty of questions they were requested to answer. The statistical evidence produced in this paper suggests that the OSIRIS modification can indeed be beneficial but will only be used by large numbers of academics if the process becomes automated. Ensuring fairness for all, when employing large-scale summative e-assessment, was the driving force behind the Al-Hajri and Ricketts (this issue) paper. They investigated a number of factors that might affect Omani student performance on computerized tests. They discovered that although there were no differences in computer experience or computer self-efficacy measure between males and females, the females performed less well in computerized tests than their male colleagues. Al-Hajri and Ricketts therefore recommend that students practise more with computerized assessments and hence become more familiar with them before they sit the final summative examination.  A number of recommendations for changes that can support student learning with computer-assisted assessment have been raised above and one way to facilitate change at an institutional level has been proposed by Whitelock and Cross (this issue). They have developed a benchmarking instrument that uses key stakeholders to survey the complete assessment process. This means that not only baselines can be set but, the student experience of assessment can also be monitored, while simultaneously providing staff with meaningful data about their performance as assessors. This type of evaluation can certainly add value to institutions through supporting a continuous improvement trajectory based on a maturity model of best practice. One of the new technological suggestions for best practice is proposed by Kleeman et al. (this issue) which is to embed electronic assessments within the learning materials and not to have them as two separate entities. This is now technologically possible and they state that the key requirement to embedding assessments in this way is for the software to ‘sense the display characteristics of the device or window in which the assessment is being delivered and display the questions appropriately’. The authors provide clear examples of where these types of assessments can prove beneficial and where they should not be used. This paper also provides a thoughtful account of how embedded assessments might also shape the future of the assessment for learning agenda. International Journal of e-Assessment vol.1 no.1 2011  3Marking and computer-assisted assessment Tutors want to give their students timely and constructive feedback to assist them to take control of their own learning. One way to do this is to introduce some form of peer assessment whereby the students themselves become more assessment literate. Barker and Bennett (this issue) describe how they employed an electronic voting system in their Masters’ students’ class to mark all the websites created for one of the summative assignments. The second part of the assignment was marked by the tutors alone using an automated feedback tool. The authors found that by employing computer-assisted marking techniques that the marking time was reduced by 30% and the feedback was delivered within three weeks of the assignment submission date. Another concern for markers is whether the change over from marking paper assignments to the onscreen marking of essays will prove too onerous and more importantly will the final grade awarded be as accurate for the on screen process as it is for the paper marking scenario? Johnson et al. (this issue) systematically investigated this question, with data collected from 11 experienced markers working for a large UK-based awarding body. The study found that examiners can mark with equal accuracy onscreen essays but that the examiners experienced greater cognitive workload when in on screen marking mode. This finding raises important issues of training and support for tutors during the marking mode transition period. Harnessing the technology to improve efficiency and pedagogy The big question for most tutors is whether adopting new technologies for assessment will actually support student learning and is worth all the effort and capital investment involved in devising and implementing a staff training regime. Peter Crompton (this issue) not only provides an overview of the functionality  of computer-based algebra assessment packages but also evaluates the effectiveness of these packages to deliver learning outcomes through the examination of empirical survey data. He concludes that at present engaging in this type of activity for an individual academic is too costly and that in general engaging support staff or postgraduate students to assist in the design process of these types of assessments prevents them from being adopted into the mainstream of mathematics undergraduate teaching. Two other papers in this issue try to address the problems outlined by Crompton by examining methods that can assist with the formaliztion and automatic generation of e-assessments. The first response by Cubric and Tosic is to expand the meta ontology for generating questions and also to add a semantic interpretation mapping between the domain ontology and the target question ontology using Bloom’s (1956) taxonomy for this semantic interpretation. In this way they wanted to refine the automatic generation of multiple-choice questions (MCQs). The authors report the findings from their implementation of the prototype system, where they have defined four new question types and three additional strategies for generating meaningful distracters. Another approach, by which assessment can support the learning process, which is advocated by Sitthisak and Gilbert, is by making explicit how assessment and feedback should play an integral role within the learning design process. They propose an extension to the IMS Learning Design specification (IMS LD) which elevates the role of competence to a more salient position and elevates assessment to the position of a key learning resource. The authors argue that these extensions International Journal of e-Assessment vol.1 no.1 2011  4assist in the formal description of a leaning and teaching process making it more available to machine processing. Their work illustrates another step towards the simplification of automatic question and feedback generation. Towards assessment for learning The papers in this issue highlight the general recognition that times are changing and that assessment needs to become embedded in the teaching/learning cycle, and not purely as a checking device for the awarding institution. Practitioners of e-assessment have tried to address this issue by providing timely and constructive feedback to students through the development of a number of interactive tasks that can be automatically marked, often presented to the student in simple formats such as multiple choice question, but which more importantly can provide immediate feedback to the learner. There is also recognition that assessment tasks themselves must change and even the physical spaces for e-assessment require immediate attention. However in all our research endeavours we need to provide students with advice that supports future learning and many opportunities for substantive improvement remain. Therefore whether a task is assessed by peers, self or tutors, the advice generated from the assessment process should take the students forward on to the next stage of their learning journey. These papers encourage us to rise to the challenge of developing more sophisticated computer-assisted assessment systems that support student learning and there is still a lot to do! References  ARG, 2002. Assessment for learning: 10 principles, http://www.assessment-reform-group.org (accessed October 29, 2009). Bennett, R. 2002. Inexorable and inevitable: The continuing story of technology and assessment. Journal of Technology, Learning, and Assessment 1, 1: 3-23. Bloom, B.S. and  D.R. Krathwohl. 1956. Taxonomy of educational objectives: The classification of educational goals by a committee of college and university examiners. Handbook 1. Cognitive domain. New York: Addison-Wesley. Hunley, S. and M. Schaller. 2009. Assessment the key to creating spaces that promote learning. Educause Review 44, 2..  Mogey, N., G. Sarab, J. Haywood, S. van Heyningen, D. Dewhurst, D. Hounsell and R. Neilson. 2007. The end of handwriting? Using computers in traditional essay examinations. Journal of Computer Assisted Learning 24, 1: 39-46. Quinn, P., N. Muldoon and M. Mozer. 2003. An institutional re-positioning of examinations. In 16th ODLAA Biennial Forum Conference Proceedings.  Stoner, G. 1996. Implementing learning technology. Learning technology dissemination initiative. Edinburgh: Heriot-Watt University. Warburton, W. 2009. Quick win or slow burn: modelling UK HE CAA uptake. Assessment & Evaluation in Higher Education 34, 3: 257-272.  Whitelock, D. and A. Brasher. 2006. Developing a roadmap for e-assessment: Which way now?  In Proceedings of the 10th International CAA Conference, ed. M. Danson, 487-501. Loughborough: Loughborough University. Whitelock, D., D. MacKenzie, C. Whitehouse, C. Ruedel and S. Rae. 2006. Identifying innovative and effective practice in e-assessment: findings from seventeen UK case studies.  In Proceedings of the 10th International CAA Conference, ed. M. Danson, 505-511. Loughborough: Loughborough University. International Journal of e-Assessment vol.1 no.1 2011  5Whitelock, D. 2010. Activating assessment for learning: Are we on the way with Web 2.0? In Web 2.0-based-e-learning: Applying social informatics for tertiary teaching eds M.J.W. Lee and C. McLoughlin. Hershey: IGI Global.  Zhang, X. and R. Klette. 2003. Object classification and tracking in traffic scenes. Computer Graphics and Imaging 2003: 232-237.   

Preface : CAA 2010: Computer assisted assessment: supporting student learning

http://oro.open.ac.uk/29083/1/Preface_IJEA_1%281%29.pdf

Preface : CAA 2010: Computer assisted assessment: supporting student learning

Abstract

Similar works

Full text

Available Versions

Open Research Online