Bennett, Judith Merryn

Dunlop, Lynda

Knox, Kerry Jane

Whitehouse, Anne Mary

White Rose Research Online

How to use multiple choice questions for formative assessment

This is a repository copy of How to use multiple choice questions for formative assessment.White Rose Research Online URL for this paper:https://eprints.whiterose.ac.uk/125605/Version: Accepted VersionArticle:Whitehouse, Anne Mary, Bennett, Judith Merryn orcid.org/0000-0002-5033-0804, Dunlop, Lynda orcid.org/0000-0002-0936-8149 et al. (1 more author) (2017) How to use multiple choice questions for formative assessment. Education in Chemistry. ISSN 1749-5326 eprints@whiterose.ac.ukhttps://eprints.whiterose.ac.uk/Reuse Items deposited in White Rose Research Online are protected by copyright, with all rights reserved unless indicated otherwise. They may be downloaded and/or printed for private study, or other acts as permitted by national copyright laws. The publisher or other rights holders may allow further reproduction and re-use of the full text version. This is indicated by the licence information on the White Rose Research Online record for the item. Takedown If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request. 1 170807 RSC EiC Using MCQ Whitehouse.docx  Mary Whitehouse Using multiple choice questions to assess chemical understanding Mary Whitehouse, Judith Bennett, Lynda Dunlop, Kerry J. Knox  Earlier this year we carried out a synthesis of research related to the summative assessment of chemistry subject knowledge for the RSC (Bennett, Dunlop, Knox, & Whitehouse, 2017). From this study it became clear that much work has been undertaken into the development of effective assessment of chemical understanding using multiple choice questions (MCQ); it is some of that work that provides the background to this article. As part of a suite of tools for summative assessment MCQ offer a number of inherent advantages, for example they can be marked reliably and quickly, making them cost and time efficient for large cohorts and they can be used to cover a broader range of content within a shorter test time than would be possible with open response questions alone (Black, 1998). However making the most of the benefits of MCQs requires careful preparation, including the challenge of writing good questions where the distractors are appropriate and do not mislead students. Ideally MCQ should be pretested before they are used for high stakes testing. Concern has been raised by some critics is that it is possible that some students will gain marks by guessing the correct answer; various strategies have been reported to reduce the effect of guessing on the marks awarded (see, for example, Campbell, 2015). An area of particular interest for teachers is the use of multiple choice questions for formative assessment.  Combining good MCQs with mini whiteboards, Plickers < www.plickers.com >, Socrative < www.socrative.com >, or other ways of collecting responses makes it quick for a teacher to collect information about students’ understanding, allowing the teaching to be adapted to meet the needs of the students (see for example, Allan, 2017). Some of the constraints imposed when MCQ are used for high stakes testing do not apply when questions are used for formative assessment, and the answer architecture can be tailored to provide useful diagnostic information. For example, so-called ‘ordered multiple choice questions’ have been identified as being particularly useful for formative assessment. Hadenfeldt, Bernholt, Liu, Neumann, & Parchmann (2013) developed MCQ in which the possible responses represented different  levels of understanding of ideas about the nature of matter. This development was based on a learning progression that reflected increasingly sophisticated understanding of ideas about the structure and composition of matter. The authors found that these questions discriminated as effectively as open response questions on the same topic. Development of such questions is shown to be an iterative process, in which the outcomes of using the questions not only informs the teaching of the current students, but also leads to further development of the original learning progression, as suggested by Wilson (2009). A second development of the traditional MCQ answer architecture of particular value for formative assessment involves presenting the MCQ in a ‘confidence grid’ format as shown in Figure 1 (Whitehouse, 2014). We have found that students often do not want to make a clear choice between the possible answers to a multiple choice question. Converting a simple MCQ into a ‘confidence grid’ format enables the student to show their uncertainties, and the teacher to understand better where problems lie. The example question shown was developed from a question used in the Assessing 2 170807 RSC EiC Using MCQ Whitehouse.docx  Mary Whitehouse Students’ Concept of a Substance project at Durham University (Johnson & Tymms, 2011). In questions of this type the distractors are incorrect ideas commonly held by students. In the original research Johnson and Tymms found that most students (61%) selected options A or B, with only 21% selecting the correct answer.  Figure 1 .  A multiple-choice question presented in a ‘confidence grid’ format (adapted from Johnson & Tymms (2011) Teachers have found the confidence grid a useful format for questions where many students have common alternative conceptions. Development of questions of this type will provide teachers with a better understanding of the ideas their students hold and as a consequence enable to the teacher to better tailor the lesson to the class.  Research about students’ ideas about chemistry provides plenty of inspiration for writing questions of this type (see, for example, Kind, 2004, Taber, 2002). Other developments in multiple choice questions have been made possible by the increased use of on-screen assessment, including the use of two-tier questions and adaptive questioning where the route taken through the questions is determined by students’ responses. Whatever the means of collecting the responses and whatever the question format, the quality of the answer options is key to their effective use to support learning. Mary Whitehouse (@MaryUYSEG) is an Honorary Fellow in the University of York Science Education Group. Judith Bennett is the Salters' Professor of Science Education at the University of York Lynda Dunlop is a Lecturer in Science Education at the University of York Kerry J. Knox is a Lecturer in Science Education at the University of York  References Allan, B. (2017). Responsive teaching.  Retrieved from https://eic.rsc.org/ideas/responsive-teaching/3007116.article Bennett, J., Dunlop, L., Knox, K. J., & Whitehouse, M. (2017). The assessment of chemistry subject knowledge in secondary education: a critical evaluation of the literature: Final report to the Royal Society of Chemistry. Retrieved from York:  3 170807 RSC EiC Using MCQ Whitehouse.docx  Mary Whitehouse Black, P. (1998). Testing, friend or foe? Theory and practice of assessment and testing. London: Falmer Press. Campbell, M. L. (2015). Multiple-Choice Exams and Guessing: Results from a One-Year Study of General Chemistry Tests Designed to Discourage Guessing. Journal of Chemical Education, 92(7), 1194-1200.  Hadenfeldt, J. C., Bernholt, S., Liu, X., Neumann, K., & Parchmann, I. (2013). Using Ordered Multiple-Choice Items to Assess Students' Understanding of the Structure and Composition of Matter. Journal of Chemical Education, 90(12), 1602-1608.  Johnson, P., & Tymms, P. (2011). The Emergence of a Learning Progression in Middle School Chemistry. Journal of Research in Science Teaching, 48(8), 849-877.  Kind, Vanessa Beyond appearances: Students’ misconceptions about basic chemical ideas. School of Education, Durham University, UK. (2004): http://www.rsc.org/learn-chemistry/resource/res00002202/beyond-appearances  Taber, K. S. (2002). Chemical Misconceptions: Prevention, Diagnosis and Cure: Classroom resources  Volume 2: Royal Society of Chemistry. Whitehouse, M. (2014). Using a backward design approach to embed assessment in teaching. School Science Review, 95(352).  Wilson, M. (2009). Measuring progressions: Assessment structures underlying a learning progression. Journal of Research in Science Teaching, 46(6), 716-730.     

https://eprints.whiterose.ac.uk/125605/1/RSC_EiC_assessment_article_Aug_17.pdf

How to use multiple choice questions for formative assessment

Abstract

Similar works

Full text

Available Versions

White Rose Research Online

White Rose Research Online