Evolution of an assessment project

We describe the evolution of a continuing project that started life as a study of students’ conceptions and reasoning patterns in elementary physics and morphed into a study of exam marking. The narrative structure of the paper reflects the evolutionary character of the project: aims and methods were not predetermined but developed as they interacted with each other. Our investigation began several years ago (Sharma, Millar, Smith and Sefton 2004) as a study of the way that students answer qualitative examination questions in physics and of what those answers tell us about patterns of conceptual understanding and reasoning. Specifically, we analysed answers to the following question: In a spaceship orbiting the earth, an astronaut tries to weigh himself on bathroom scales and finds that the scale indicates a zero reading. However, he is also aware that his mass hasn’t changed since he left the earth. Using physics principles, explain this apparent contradiction. The question was included in the final examination in 1998 for two alternative first-year first semester courses: a Fundamentals course for beginners and a Regular course for students who had done physics for the Higher School Certificate. We analysed a sample of 100 answers from each of the two courses

Differences in two evaluations of answers to a conceptual physics question: a preliminary analysis

In their exploration of student understanding of gravity, Sharma et al. (2004 and 2005) discovered a discrepancy between phenomenographic analysis of student answers to one short examination question and the distribution of marks for the same question between two first-year university physics classes. We report on a preliminary investigation of factors which, we hypothesised, may have contributed to that discrepancy. Additional analysis and evaluation of the original set of answers included a detailed study of the use of physics terminology (PhysicsSpeak) and diagrams in the answers, with the aim of discovering how those features may have affected the marks. A selection of the answers was reviewed for evidence of other characteristics which may have influenced the marker. The views and recollections of the original marker were also recorded. There is no single explanation for the discrepancy, but we found that the use of diagrams has a significant effect on marks, whereas the influence of PhysicsSpeak was weaker than expected

Students' Understandings of Gravity in an Orbiting Space-Ship

We report on an investigation of students' ideas about gravity after a semester of instruction in physics at university. There are two aspects to the study which was concerned with students' answers to a carefully designed qualitative examination question on gravity. The first aspect is a classification of the answers and a comparative study of the ways the problem was tackled by two large groups of students who had different backgrounds in physics and were exposed to different teaching styles. The second aspect is to investigate how students link concepts to solve the problem. We used a phenomenographic analysis of student responses to extract patterns of reasoning and alternative conceptions behind the solutions. We found no differences between the classes of answers given by students in the two courses. Our analysis also identifies a hierarchy in the complexity of the hypothetical reasoning pathways, which we interpret as reflecting the ways in which students may link concepts and resolve conflicts as they solve the problem. The hypothetical reasoning pathways may help educators to develop instructional material or lecture room dialogue in order to tease out key issues. An unexpected finding is that there is a discrepancy between our conclusion that the two groups of answers are similar and the distribution of marks awarded by the examiner – which implies that the quality of the answers is different for the two groups