Setting objective tests in mathematics using QM perception

We here describe technical issues in setting objective tests in various areas of mathematics using Question Mark Perception’s QML language and format files, coupled with MathML mathematics mark-up and the Scalable Vector Graphics (SVG) syntax for producing diagrams. The plain text MathML and SVG coding can replace graphics files commonly used to display equations and diagrams in CAA packages and web pages, and have the overwhelming advantage that random parameters can be dropped into the interpreted plain text at runtime, thereby producing many millions of realisations of the underlying question style

Issues with setting online objective mathematics questions and testing their efficacy

The Mathletics database now comprises many mathematical topics from GCSE to level 2 undergraduate. The aim of this short paper is to document, explore and provide some solutions to the pedagogic issues we are facing whilst setting online objective questions across this range. Technical issues are described in the companion paper by Ellis, Greenhow and Hatt (2006). That paper refers to “question styles to stress that we author according to the pedagogic and algebraic structure of the content of a question; random parameters are chosen at runtime ... This results in each style having thousands, or even millions, of realisations seen by the users.” With this emphasis, and with new topics being included, new question types beyond the usual multi-choice (MC) etc have been developed to ask appropriate and challenging questions. We feel that their pedagogic structure (and underlying code) is widely applicable to testing beyond the scope of Mathematics. This paper describes three of the new question types: Word Input, Responsive Numerical Input and 4/True/False/Undecidable/Statement/Property. Of generic importance is the fact that each of these question types can include post-processing of submitted answers; sample Javascript coding that checks the validity of the input(s) before marking takes place is described. In common with most of the rest of the question style’s content this could be exported to other CAA systems. Ellis et al (2005) and Gill & Greenhow (2006) describe initial results of a trial of level 1 undergraduate mechanics questions. This academic year we have expanded the range of tests to foundation and level 1 undergraduate algebra and calculus, involving several hundred students. First and foremost we have underlined the value of Random Numerical Input (RNI) question types compared with traditional Numerical Input (NI) types for which answer files resulting from questions with randomised parameters are exceptionally difficult to interpret. Despite our current lack of a consistent and fullymeaningful way of encoding the mal-rules within the question outcome metadata, mal-rule-based question types (MC, RNI etc) have been analysed in terms of difficulty, discrimination and item analysis. In the case of multiplechoice questions any weaknesses are separately identified as skill-based or conceptual

Recent developments in setting and using objective tests in mathematics using QM Perception

This short paper builds upon work described at the last CAA Conference, Greenhow & Gill (2004), in setting objective tests in various areas of mathematics using Question Mark Perception. Current activities continue to exploit the QML language and template files, coupled with MathML mathematics mark-up and the Scalable Vector Graphics (SVG) syntax for producing diagrams. There are many advantages to using such mark-up languages, primarily the use of random parameters at runtime that thereby produce dynamic equations, distracters, feedback and diagrams. An unlooked for, but welcome, advantage, is that one can also resize and recolour these elements by reading the preferences that have been set up in a user-defined cookie. This means that “reasonable provision” for disabled students as required by the SENDA legislation, is built-in. The MathML and SVG technology can be exported to any web-based system, or indeed ordinary web pages that can provide an inexhaustible set of realisations at the click of the reload button. Being central to the display of mathematics on the web, MathML’s WebEQ applet has recently been considerably extended to include graphing of MathML expressions, naturalistic input of equations with syntax checking and math-action tags. These math-action tags can be used to define a specific part of an equation, and mouse actions can then be acted upon, for example to provide a commentary on that part of the equation, toggle to another equation (perhaps a derivation of the tagged term or similar) or, possibly, to set a variable that can be used for marking (as in a hot spot question). The first part of this paper will show how these new facilities can be input into new question types for effective questions and feedback design. It is clear that much useful technology already exists, but setting effective questions that benefit students’ learning requires equal attention to their content and pedagogy. The second part of this paper looks at a possible methodology for setting much more advanced questions than hitherto, looking closely at an example from the ordinary differential equations section of Mathletics. The third part of this paper looks at a series of experiments with a first year mechanics group at Brunel University, as part of the Formative Assessment and Feedback (FAST) project. Students’ reactions were studied, especially the effect of the feedback on their subsequent behaviour when faced with similar/dissimilar questions after a variable time delay. Students spent a lot of time and energy considering the feedback provided, sometimes copying it down or printing it out. Somewhat surprisingly, it seems that a “learning resource” has actually been written, whose formative nature is of equal or more importance than the assessment function originally intended. It can be concluded that plentiful formative feedback is of great importance in the students’ ability to learn mathematics from the tests, rather than simply get their grades or marks in an efficient manner