Attention and visual memory in visualization and computer graphics

Abstract—A fundamental goal of visualization is to produce images of data that support visual analysis, exploration, and discovery of novel insights. An important consideration during visualization design is the role of human visual perception. How we “see ” details in an image can directly impact a viewer’s efficiency and effectiveness. This paper surveys research on attention and visual perception, with a specific focus on results that have direct relevance to visualization and visual analytics. We discuss theories of low-level visual perception, then show how these findings form a foundation for more recent work on visual memory and visual attention. We conclude with a brief overview of how knowledge of visual attention and visual memory is being applied in visualization and graphics. We also discuss how challenges in visualization are motivating research in psychophysics

Recognition by directed attention to recursively partitioned images

A learning/recognition model (and instantiating program) is described which recursively combines the learning paradigms of conceptual clustering (Michalski, 1980) and learning-from-examples to resolve the ambiguities of real-world recognition. The model is based on neuropsychological and psychological evidence that the visual system is analytic, hierarchical, and composed of a parallel/serial dichotomy (many, see conclusions by Crick, 1984). Emulating the experimental evidence, parallel processes in the model decompose the image into components and cluster the constituents in much the same way as the image processing technique known as moment analysis (Alt, 1962). Serial, attentive mechanisms then reassemble the decompositions by investigating spatial relationships between components. The use of attentive mechanisms extends the moment analysis technique to handle alterations in structure and solves the contention problem created by combining the two learning paradigms. The contention results from a disagreement between the teacher and the model on what constitutes the salient features at the highest level of the symbol. There are four cases ZBT must handle, two of which result from the disagreement with the teacher. The parallel/serial dichotomy represents a vertical/horizontal tradeoff between the invariant and variant features of a domain. The resultant learned hierarchy allows ZBT to recognize structural differences while avoiding problems of exponential growth

Saliency Prediction in the Data Visualization Design Process

L'abstract è presente nell'allegato / the abstract is in the attachmen

Visual search and VDUS

This wide-ranging study explored various parameters of visual search in relation to computer screen displays. Its ultimate goal was to help identify factors which could result in improvements in commercially available displays within the 'real world’. Those improvements are generally reflected in suggestions for enhancing efficiency of locatabolity of information through an acknowledgement of the visual and cognitive factors involved. The thesis commenced by introducing an ergonomics approach to the presentation of information on VDUs. Memory load and attention were discussed. In the second chapter, literature on general and theoretical aspects of visual search (with particular regard for VDUs) was reviewed. As an experimental starting point, three studies were conducted involving locating a target within arrays of varying configurations. A model concerning visual lobes was proposed. Two text-editing studies were then detailed showing superior user performances where conspicuity and the potential for peripheral vision are enhanced. Relevant eye movement data was combined with a keystroke analysis derived from an automated protocol analyser. Results of a further search task showed icons to be more quickly located within an array than textual material. Precise scan paths were then recorded and analyses suggested greater systematicity of search strategies for complex items. This led on to a relatively 'pure' search study involving materials of varying spatial frequencies. Results were discussed in terms of verbal material generally being of higher spatial frequencies and how the ease of resolution and greater cues available in peripheral vision can result in items being accessed more directly. In the final (relatively applied) study, differences in eye movement indices were found across various fonts used. One main conclusion was that eye movement monitoring was a valuable technique within the visual search/VDU research area in illuminating precise details of performance which otherwise, at best, could only be inferred

An investigation into the emotion-cognition interaction and sub-clinical anxiety

This thesis combines behavioural and electrophysiological approaches in the study of the emotion-cognition interaction and sub-clinical anxiety. The research questions addressed in this thesis concern, specifically: the impact of emotion on attention; the interplay between attention and emotion in anxiety;and the cognitive construct of affect. Chapter 1 provides an introduction to emotion research, cognitive models of anxiety and motivates the thesis. Chapter 2 investigates whether affective processing is automatic. More specifically, to elucidate whether facilitated processing of threat in anxiety, evidenced by emotion-related ERP modulations, requires attentional resources. It was previously reported that emotional expression effects on ERP waveforms were completely eliminated when attention was directed away from emotional faces to other task-relevant locations (Eimer et al., 2003). However, Bishop et al. (2004) reported that threat-related stimuli can evoke amygdala activity without attentional engagement or conscious awareness in high-anxious but not low-anxious participants. Spatial attention was manipulated using a similar paradigm as Vuilleumier et al. (2001) and Holmes et al. (2003), to investigate the mechanism underlying the threat-related processing bias in anxiety by examining the influence of spatial attention and trait anxiety levels on established ERP modulations by emotional stimuli. Participants were instructed to match two peripheral faces or two peripheral Landolt squares. The Landolt squares task was selected since this is an attentionally demanding task and would likely consume most, if not all, attention resources. The ERP data did not offer support to the claim that affective stimuli are processed during unattended conditions in high-anxious but not low-anxious participants. Rather, it questions whether a preattentive processing bias for emotional faces is specific to heightened anxiety. This is based on the finding of an enhanced LPP response for threat/happy versus neutral faces and an enhanced slow wave for threat versus neutral faces, neither modulated by the focus of attention for both high and low anxiety groups. Chapter 3 investigated the delayed disengagement hypothesis proposed by Fox and colleagues (2001) as the mechanism underlying the threat-related attentional bias in anxiety. This was done by measuring N2pc and LRP latencies while participants performed an adapted version of the spatial cueing task.Stimuli consisted of a central affective image (either a face or IAPS picture, depending on condition) flanked to the left and right by a letter/number pair. Participants had to direct their attention to the left or right of a central affective image to make an orientation judgement of the letter stimulus. It was hypothesised that if threat-related stimuli are able to prolong attentional processing, N2pc onset should be delayed relative to the neutral condition. However, N2pc latency was not modulated by emotional valence of the central image, for either high or low anxiety groups. Thus, this finding does not provide support for the locus of the threat-related bias to the disengage component of attention. Chapter 4 further investigated the pattern of attentional deployment in the threat-related bias in anxiety. This was done by measuring task-switching ability between neutral and emotional tasks using an adapted version of Johnson’s (in press) attentional control capacity for emotional representations (ACCE) task. Participants performed either an emotional judgement or a neutral judgement task on a compound stimulus that consisted of an affective image (either happy versus fearful faces in the faces condition, or positive versus negative IAPS pictures in the IAPS condition) with a word located centrally across the image (real word versus pseudo-word). Participants scoring higher in trait anxiety were faster to switch from a neutral to a threatening mental set. This improved ability to switch attention to the emotional judgement task when threatening faces are presented is in accordance with a hypervigilance theory of anxiety. However, this processing bias for threat in anxiety was only apparent for emotional faces and not affective scenes, despite the fact that pictures depicting aversive threat scenes were used (e.g., violence, mutilation). This is discussed in more detail with respect to the social significance of salient stimuli. Chapter 5 in a pair of experiments sought to investigate how affect is mentally represented and specifically questions whether affect is represented on the basis of a conceptual metaphor linking direction and affect. The data suggest that the vertical position metaphor underlies our understanding of the relatively abstract concept of affect and is implicitly active, where positive equates with ‘upwards’ and negative with ‘downwards’. Metaphor-compatible directional movements were demonstrated to facilitate response latencies, such that participants were relatively faster to make upward responses to positively-evaluated words and downward responses to negatively-evaluated words than to metaphorincompatible stimulus-response mappings. The finding suggests that popular use of linguistic metaphors depicting spatial representation of affect may reflect our underlying cognitive construct of the abstract concept of valence. Chapter 6 summarises the research in the thesis and implications of the present results are discussed, in particular in relation to cognitive models of anxiety. Areas of possible future research are provided

Asymmetric visual representation of sex from facial appearance

We efficiently infer others' traits from their faces, and these inferences powerfully shape our social behaviour. Here, we investigated how sex is represented in facial appearance. Based on previous findings from sex-judgment tasks, we hypothesized that the perceptual encoding of sex is not balanced but rather polarized: for the processes that generate a sex percept, the default output is "male," and the representation of female faces extends that of the male, engaging activity over unique detectors that are not activated by male faces. We tested this hypothesis with the logic of Treisman's studies of visual search asymmetries, predicting that observers should more readily detect the presence of female faces amongst male distractors than vice versa. Across three experiments (N = 32 each), each using different face stimuli, we confirmed this prediction in response time and sensitivity measures. We apply GIST analyses to the face stimuli to exclude that the search asymmetry is explained by differences in image homogeneity. These findings demonstrate a property of the coding that links facial appearance with a significant social trait: the female face is coded as an extension of a male default. We offer a mechanistic description of perceptual detectors to account for our findings and posit that the origins of this polarized coding scheme are an outcome of biased early developmental experience

The Effectiveness of Multimedia Digital Graphics in the Context of a Simulated Content Environment

The overall purpose of this thesis was to evaluate the effectiveness and potential benefits or shortcomings of digital motion graphics when utilized in the context of a real-world marketing effort. It is vital to evaluate whether or not the understanding we currently have of motion graphics as a tool for effective communication still remains consistent when principles of visual noise, attention capacity limits, pairing with physical materials, and coordination across mediums are taken into consideration. Participants in this study were asked to interact with prepared content within the framing of a simulated social media feed viewed on a computer or mobile device, while also being exposed to paired print campaign materials either in a physical or non-physical format in order to evaluate the overall retention, recall, and message effectiveness of brand information. Data collected from this study was both quantitative and qualitative in nature, skewing more heavily in the direction of quantitative data. The former consists of evaluation of ability to recall information presented to the viewer through prepared marketing materials in print format along with a simulated social media campaign, while the latter applies to questions regarding personal evaluation of the materials and content provided to the viewer during testing and the effectiveness of particular forms of graphics. This study encompassed a total of 253 final responses from participants grouped across a total of four possible combinations of variables between two categories: the form of content with which they were presented in the simulated social media feed, in either multimedia motion format or static graphic format, and the form of medium with which they were presented the magazine marketing materials, either physical print or non-physical. This testing procedure was followed by a questionnaire segment meant to evaluate recall of various elements from the marketing materials, followed by an exit survey. The resulting data suggests a number of specific differences in the assessment and recall of certain forms of content information, including more accurate recall of primary information for a specific form of content as well as a demonstrable relationship between the selected form of content and the medium it was presented in as it relates to the participant’s ability to recall information. Additional observations were also made in regards to the perception of advertisement frequency between medium formats. This study’s results could potentially inform processes of decision-making for coordination of advertising materials across differing formats and mediums, as well as provide insight into the purposes and value that differing forms of content might provide from a communicative or informative aspect. Those attempting to create or promote content across physical, digital, and social channels may find this study’s findings beneficial towards their own efforts in content creation, as well as use it to inform priorities in advertising or communication efforts that go beyond a single format or medium