Investigating computer aided assessment of mathematical proof by varying the format of students' answers and the structure of assessment design by STACK

Abstract

Students are increasingly being expected to use Computer Aided Assessment (CAA) systems as support for traditional courses. Assessing a full mathematical proof in an educational context and providing feedback and other outcomes to students is currently well beyond the capabilities of CAA systems. One possible approach to assessing students' answers has been to break up larger tasks into smaller individual steps to which automatic assessment can then be applied. However, the method of marking depends on the format of a mathematical response. This thesis aims to investigate the effectiveness of different formats in computer aided assessment of mathematics. A format effect occurs when the format of an exercise affects the rate of successful outcomes of the exercise. Having established the need to study the format effect when writing mathematical arguments particularly online, I consider three formats for writing open-ended questions: two-column, typing, and Separated Concerns. The first of three studies explored the impact of the two-column format in writing simple mathematical arguments. In conducting this research, I developed a coding scheme to describe and analyse the structure of individual mathematical arguments. The second study focused on the difference and the format effects between uploading handwritten and typing in writing mathematical responses. Another outcome of this study is to provide a further application of using the coding scheme on analysing students’ arguments. The third study focused on updating STACK potential response tree based on Separated Concerns. STACK is a System for Teaching and Assessment using a Computer algebra Kernel, is an open source computer aided assessment system for mathematics, and other STEM subjects. Separated Concerns is a phrase used to describe materials in which potential misconceptions are addressed directly. In this study, I focused exclusively on students’ responses, and misconceptions in learning proof by induction using STACK. Mathematical induction is used as a vehicle to illustrate the idea of Separated Concerns. The main goal of the third study is to understand how engagement with learning materials packaged into online quizzes to replace live lectures, a ``lecture quiz'', related to success on the weekly assessed quiz, and the course total. A second goal of the third study is to explore the common mistakes made by students when using online materials to prepare for mathematical induction. This study also illustrates how to use research to update the algorithms which assess students' answers, known as ``STACK potential response trees'', in questions written to support learning mathematical induction based on Separated Concerns