Going SOLO to assess novice programmers

This paper explores the programming knowledge of novices using Biggs' SOLO taxonomy. It builds on previous work of Lister et al. (2006) and addresses some of the criticisms of that work. The research was conducted by studying the exam scripts for 120 introductory programming students, in which three specific questions were analyzed using the SOLO taxonomy. The study reports the following four findings: when the instruction to students used by Lister et al. - "In plain English, explain what the following segment of Java code does" - is replaced with a less ambiguous instruction, many students still provide multistructural responses; students are relatively consistent in the SOLO level of their answers; student responses on SOLO reading tasks correlate positively with performance on writing tasks; postgraduates students manifest a higher level of thinking than undergraduates. Copyright 2008 ACM

The Effects of Visual Information on Users' Mental Models: An Evaluation of Pathfinder Analysis as a Measure of Icon Usability

Research has shown that individuals' knowledge structures change as a result of learning and experience. This article investigates the possibility that the content of graphical user interfaces can play a role in determining the nature of the knowledge structures users develop. Users employed either concrete, abstract, or arbitrary icon sets in a computer-based problem-solving task. The effects of these icons were assessed using standard measures of performance. On the basis of the assumption that users' mental models should be better if appropriate icons were presented on the interface, Pathfinder analysis was used to elicit users' knowledge structures as they gained experience with the interface. The efficacy of this measure was then compared with performance measures. Our findings show that users' knowledge structures do depend on the nature of the graphical information presented at the interface but do not rely as much on the use of the visual metaphor as previously thought. Although most measures were sensitive to initial differences between icon sets, only some measures were sensitive to the long-term differences that remained after users had gained experience with the icon set. The implications of these findings for interface design are discussed

The Effects of Visual Information on Users' Mental Models: An Evaluation of Pathfinder Analysis as a Measure of Icon Usability

Research has shown that individuals' knowledge structures change as a result of learning and experience. This article investigates the possibility that the content of graphical user interfaces can play a role in determining the nature of the knowledge structures users develop. Users employed either concrete, abstract, or arbitrary icon sets in a computer-based problem-solving task. The effects of these icons were assessed using standard measures of performance. On the basis of the assumption that users' mental models should be better if appropriate icons were presented on the interface, Pathfinder analysis was used to elicit users' knowledge structures as they gained experience with the interface. The efficacy of this measure was then compared with performance measures. Our findings show that users' knowledge structures do depend on the nature of the graphical information presented at the interface but do not rely as much on the use of the visual metaphor as previously thought. Although most measures were sensitive to initial differences between icon sets, only some measures were sensitive to the long-term differences that remained after users had gained experience with the icon set. The implications of these findings for interface design are discussed