Studies of visual attention in physics problem solving

Doctor of PhilosophyDepartment of PhysicsN. Sanjay RebelloThe work described here represents an effort to understand and influence visual attention while solving physics problems containing a diagram. Our visual system is guided by two types of processes -- top-down and bottom-up. The top-down processes are internal and determined by ones prior knowledge and goals. The bottom-up processes are external and determined by features of the visual stimuli such as color, and luminance contrast. When solving physics problems both top-down and bottom-up processes are active, but to varying degrees. The existence of two types of processes opens several interesting questions for physics education. For example, how do bottom-up processes influence problem solvers in physics? Can we leverage these processes to draw attention to relevant diagram areas and improve problem-solving? In this dissertation we discuss three studies that investigate these open questions and rely on eye movements as a primary data source. We assume that eye movements reflect a person’s moment-to-moment cognitive processes, providing a window into one’s thinking. In our first study, we compared the way correct and incorrect solvers viewed relevant and novice-like elements in a physics problem diagram. We found correct solvers spent more time attending to relevant areas while incorrect solvers spent more time looking at novice-like areas. In our second study, we overlaid these problems with dynamic visual cues to help students’ redirect their attention. We found that in some cases these visual cues improved problem-solving performance and influenced visual attention. To determine more precisely how the perceptual salience of diagram elements influenced solvers’ attention, we conducted a third study where we manipulated the perceptual salience of the diagram elements via changes in luminance contrast. These changes did not influence participants’ answers or visual attention. Instead, similar to our first study, the time spent looking in various areas of the diagram was related to the correctness of an answer. These results suggest that top-down processes dominate while solving physics problems. In sum, the study of visual attention and visual cueing in particular shows that attention is an important component of physics problem-solving and can potentially be leveraged to improve student performance

New approaches to the analysis of eye movement behaviour across expertise while viewing brain MRIs

Abstract Brain tumour detection and diagnosis requires clinicians to inspect and analyse brain magnetic resonance images. Eye-tracking is commonly used to examine observers’ gaze behaviour during such medical image interpretation tasks, but analysis of eye movement sequences is limited. We therefore used ScanMatch, a novel technique that compares saccadic eye movement sequences, to examine the effect of expertise and diagnosis on the similarity of scanning patterns. Diagnostic accuracy was also recorded. Thirty-five participants were classified as Novices, Medics and Experts based on their level of expertise. Participants completed two brain tumour detection tasks. The first was a whole-brain task, which consisted of 60 consecutively presented slices from one patient; the second was an independent-slice detection task, which consisted of 32 independent slices from five different patients. Experts displayed the highest accuracy and sensitivity followed by Medics and then Novices in the independent-slice task. Experts showed the highest level of scanning pattern similarity, with medics engaging in the least similar scanning patterns, for both the whole-brain and independent-slice task. In the independent-slice task, scanning patterns were the least similar for false negatives across all expertise levels and most similar for experts when they responded correctly. These results demonstrate the value of using ScanMatch in the medical image perception literature. Future research adopting this tool could, for example, identify cases that yield low scanning similarity and so provide insight into why diagnostic errors occur and ultimately help in training radiologists

Cultural differences in attention: Eye movement evidence from a comparative visual search task.

Individual differences in visual attention have been linked to thinking style: analytic thinking (common in individualistic cultures) is thought to promote attention to detail and focus on the most important part of a scene, whereas holistic thinking (common in collectivist cultures) promotes attention to the global structure of a scene and the relationship between its parts. However, this theory is primarily based on relatively simple judgement tasks. We compared groups from Great Britain (an individualist culture) and Saudi Arabia (a collectivist culture) on a more complex comparative visual search task, using simple natural scenes. A higher overall number of fixations for Saudi participants, along with longer search times, indicated less efficient search behaviour than British participants. Furthermore, intra-group comparisons of scan-path for Saudi participants revealed less similarity than within the British group. Together, these findings suggest that there is a positive relationship between an analytic cognitive style and controlled attention

Cultural differences in scene perception

Do individuals from different cultures perceive scenes differently? Does culture have an influence on visual attention processes? This thesis investigates not only what these influences are, and how they affect eye movements, but also examines some of the proposed mechanisms that underlie the cultural influence in scene perception. Experiments 1 & 2 showed that Saudi participants directed a higher number of fixations to the background of images, in comparison to the British participants. British participants were also more affected by background changes, an indication of their tendency to bind the focal objects to their contexts. Experiments 3 & 4 revealed a higher overall number of fixations for Saudi participants, along with longer search times. The intra-group comparisons of scanpaths for Saudi participants revealed less similarity than within the British group, demonstrating a greater heterogeneity of search behaviour within the Saudi group. These findings could indicate that the British participants have the advantage of being more able to direct attention towards the goals of the task. The mechanisms that have been proposed for cultural differences in visual attention are due to particular thinking styles that emerge from the prevailing culture: analytic thinking (common in individualistic cultures) promotes attention to detail and a focus on the most important part of a scene, whereas holistic thinking (common in collectivist cultures) promotes attention to the global structure of a scene and the relationship between its parts. Priming methodology was used in Experiments 5, 6 & 7 to cue these factors, although it did not reveal any significant effects on eye movement behaviours or on accuracy at recognition of objects. By testing these explanations directly (Experiment 8), findings have mainly suggested the holistic-analytic dimension is one of the main mechanisms underlying cultural diversity in scene perception. Taken together, these experiments conclude that the allocation of visual attention is also influenced by an individual’s culture

Cultural differences in scene perception

Do individuals from different cultures perceive scenes differently? Does culture have an influence on visual attention processes? This thesis investigates not only what these influences are, and how they affect eye movements, but also examines some of the proposed mechanisms that underlie the cultural influence in scene perception. Experiments 1 & 2 showed that Saudi participants directed a higher number of fixations to the background of images, in comparison to the British participants. British participants were also more affected by background changes, an indication of their tendency to bind the focal objects to their contexts. Experiments 3 & 4 revealed a higher overall number of fixations for Saudi participants, along with longer search times. The intra-group comparisons of scanpaths for Saudi participants revealed less similarity than within the British group, demonstrating a greater heterogeneity of search behaviour within the Saudi group. These findings could indicate that the British participants have the advantage of being more able to direct attention towards the goals of the task. The mechanisms that have been proposed for cultural differences in visual attention are due to particular thinking styles that emerge from the prevailing culture: analytic thinking (common in individualistic cultures) promotes attention to detail and a focus on the most important part of a scene, whereas holistic thinking (common in collectivist cultures) promotes attention to the global structure of a scene and the relationship between its parts. Priming methodology was used in Experiments 5, 6 & 7 to cue these factors, although it did not reveal any significant effects on eye movement behaviours or on accuracy at recognition of objects. By testing these explanations directly (Experiment 8), findings have mainly suggested the holistic-analytic dimension is one of the main mechanisms underlying cultural diversity in scene perception. Taken together, these experiments conclude that the allocation of visual attention is also influenced by an individual’s culture

Paying attention to the evidence: a comparison of perception and decision making processes in novice and experienced scene of crime officers using eye tracking in simulated crime scene scenarios

Research on crime scene investigation has strongly focused on the technical aspects of the process, while cognitive aspects (searching, reasoning and perception) have often been overlooked. Textbooks on forensic sciences tend to focus on identifying and processing evidence, and the use of equipment while it can be argued that cognitive factors in processing such evidence and using equipment are equally important. This thesis studies the cognitive aspects of crime scene investigation by comparing eye movement patterns in experts and novices. Studies in various domains, including surgery, sports, and chess playing have shown that eye movements differ between experts and novices, providing a tool towards a more objective assessment of skill than is possible with peer assessment. In four experiments eye movements of experts and novices were examined during (1) inspection of photographs of crime scenes on a computer screen (2) a change blindness task on crime and non-crime scene images, (3) active exploration of a simulated crime scene and (4) the assessment of emotional crime and natural scenes. While some trends in eye movement differences, such as a tendency on longer fixation durations and a broader focus on the overall scene and less on the direct evidence could be found in experts compared to novices, differences between experts and novices were considerably smaller than in other domains, despite the broad range of measures extracted from the data. This lack of clear expertise effects may relate to the rather diverse range of perceptual layouts of crime scenes, reducing possible top-down effects of expertise on the deployment of attention. The results will be discussed with a view of possible directions of future research in this domain

A systematic exploration of uncertainty in interactive systems

Uncertainty is an inherent part of our everyday life. Humans have to deal with uncertainty every time they make a decision. The importance of uncertainty additionally increases in the digital world. Machine learning and predictive algorithms introduce statistical uncertainty to digital information. In addition, the rising number of sensors in our surroundings increases the amount of statistically uncertain data, as sensor data is prone to measurement errors. Hence, there is an emergent need for practitioners and researchers in Human-Computer Interaction to explore new concepts and develop interactive systems able to handle uncertainty. Such systems should not only support users in entering uncertainty in their input, but additionally present uncertainty in a comprehensible way. The main contribution of this thesis is the exploration of the role of uncertainty in interactive systems and how novel input and output methods can support researchers and designers to efficiently and clearly communicate uncertainty. By using empirical methods of Human-Computer Interaction and a systematic approach, we present novel input and output methods that support the comprehensive communication of uncertainty in interactive systems. We further integrate our results in a simulation tool for end-users. Based on related work, we create a systematic overview of sources of uncertainty in interactive systems to support the quantification of uncertainty and identify relevant research areas. The overview can help practitioners and researchers to identify uncertainty in interactive systems and either reduce or communicate it. We then introduce new concepts for the input of uncertain data. We enhance standard input controls, develop specific slider controls and tangible input controls, and collect physiological measurements. We also compare different representations for the output of uncertainty to make recommendations for their usage. Furthermore, we analyze how humans interpret uncertain data und make suggestions on how to avoid misinterpretation and statistically wrong judgements. We embed the insights gained from the results of this thesis in an end-user simulation tool to make it available for future research. The tool is intended to be a starting point for future research on uncertainty in interactive systems and foster communicating uncertainty and building trust in the system. Overall, our work shows that user interfaces can be enhanced to effectively support users with the input and output of statistically uncertain information