A multimodal measurement approach using narratives and eye tracking to investigate visual behaviour in perceiving naturalistic and urban environments

The notion that exposure to the natural environment positively affects human well-being has been validated by studies showing measured cognitive, psychological, and physiological benefit. This research is unique in exploring gaze behaviour on environmental images possessing different levels of saliency by using eye tracking along with traditional data collection techniques for example narratives, connectivity to nature scores and interviews. The majority of existing landscape research has been derived by conducting heuristic evaluations without having empirical insight of real participant visual response. In this research, a modern multimodal measurement approach (using Narratives and Eye tracking) was applied to investigate visual behaviour in perceiving naturalistic and urban environments. Eye behaviour is predominantly attracted by salient locations. The concept of methodology of this research on naturalistic and urban environments is drawn from the approaches in market research. Borrowing methodologies from market research that examine visual responses and qualities provided a critical and hitherto unexplored approach. This research has been conducted by using mixed methodological quantitative and qualitative approaches. This thesis focuses on two aspects of Human Environment Interaction (HEI). a) The evaluation of existing environmental research and b) The use of eye tracking as a supplementary objective environmental evaluation technique. A combined qualitative and quantitative approach has been used, including a state-of-the-art technique, eye tracking. The eye movement data were complemented by participant-profile data elicited through background questionnaires and participant-perception data as captured through semistructured interviews. This provides an insight into the participant experience that spans behavioural aspects such as visual search behaviour and visual search performance data, and subjective aspects such as participant expectations and preferences. As a result of this study, when Eye tracking data was collected and analysed two types of responses were observed: i. Immediate Involuntary Response ii. Delayed Learned Response In terms of key finding of this research, it is noticed that each participant has an individual unique navigation style, while surfing through different elements of landscape images. This individual navigation style is termed the ‘Visual Signature’, which is an immediate involuntary response. On the whole, the results of this research corroborated existing landscape research findings, but they also identified potential refinements. The research contributes both methodologically and empirically to Human-Environment Interaction (HEI). This research focused on initial impressions of environmental images with the help of eye tracking. Taking under consideration the importance of the image, this research explored the factors that influence initial fixations in relation to expectations and preferences. This research adds the necessary clarity that would complete the picture and bring an insight for future landscape researchers

Eye-tracking analysis in landscape perception research : influence of photograph properties and landscape characteristics

The European Landscape Convention emphasises the need for public participation in landscape planning and management. This demands understanding of how people perceive and observe landscapes. This can objectively be measured using eye tracking, a system recording eye movements and fixations while observing images. In this study, 23 participants were asked to observe 90 landscape photographs, representing 18 landscape character types in Flanders (Belgium) differing in degree of openness and heterogeneity. For each landscape, five types of photographs were shown, varying in view angle. This experiment design allowed testing the effect of the landscape characteristics and photograph types on the observation pattern, measured by Eye-tracking Metrics (ETM). The results show that panoramic and detail photographs are observed differently than the other types. The degree of openness and heterogeneity also seems to exert a significant influence on the observation of the landscape

Visual scanning patterns and executive function in relation to facial emotion recognition in aging

OBJECTIVE: The ability to perceive facial emotion varies with age. Relative to younger adults (YA), older adults (OA) are less accurate at identifying fear, anger, and sadness, and more accurate at identifying disgust. Because different emotions are conveyed by different parts of the face, changes in visual scanning patterns may account for age-related variability. We investigated the relation between scanning patterns and recognition of facial emotions. Additionally, as frontal-lobe changes with age may affect scanning patterns and emotion recognition, we examined correlations between scanning parameters and performance on executive function tests. METHODS: We recorded eye movements from 16 OA (mean age 68.9) and 16 YA (mean age 19.2) while they categorized facial expressions and non-face control images (landscapes), and administered standard tests of executive function. RESULTS: OA were less accurate than YA at identifying fear (p < .05, r = .44) and more accurate at identifying disgust (p < .05, r = .39). OA fixated less than YA on the top half of the face for disgust, fearful, happy, neutral, and sad faces (p values < .05, r values ≥ .38), whereas there was no group difference for landscapes. For OA, executive function was correlated with recognition of sad expressions and with scanning patterns for fearful, sad, and surprised expressions. CONCLUSION: We report significant age-related differences in visual scanning that are specific to faces. The observed relation between scanning patterns and executive function supports the hypothesis that frontal-lobe changes with age may underlie some changes in emotion recognition.Accepted manuscrip

Eye-tracking the emergence of attentional anchors in a mathematics learning tablet activity

Little is known about micro-processes by which sensorimotor interaction gives rise to conceptual development. Per embodiment theory, these micro-processes are mediated by dynamical attentional structures. Accordingly this study investigated eye-gaze behaviors during engagement in solving tablet-based bimanual manipulation tasks designed to foster proportional reasoning. Seventy-six elementary- and vocational-school students (9-15 yo) participated in individual task-based clinical interviews. Data gathered included action-logging, eye-tracking, and videography. Analyses revealed the emergence of stable eye-path gaze patterns contemporaneous with first enactments of effective manipulation and prior to verbal articulations of manipulation strategies. Characteristic gaze patterns included consistent or recurring attention to screen locations that bore non-salient stimuli or no stimuli at all yet bore invariant geometric relations to dynamical salient features. Arguably, this research validates empirically hypothetical constructs from constructivism, particularly reflective abstraction

Attentive monitoring of multiple video streams driven by a Bayesian foraging strategy

In this paper we shall consider the problem of deploying attention to subsets of the video streams for collating the most relevant data and information of interest related to a given task. We formalize this monitoring problem as a foraging problem. We propose a probabilistic framework to model observer's attentive behavior as the behavior of a forager. The forager, moment to moment, focuses its attention on the most informative stream/camera, detects interesting objects or activities, or switches to a more profitable stream. The approach proposed here is suitable to be exploited for multi-stream video summarization. Meanwhile, it can serve as a preliminary step for more sophisticated video surveillance, e.g. activity and behavior analysis. Experimental results achieved on the UCR Videoweb Activities Dataset, a publicly available dataset, are presented to illustrate the utility of the proposed technique.Comment: Accepted to IEEE Transactions on Image Processin

The perceptual and attentive impact of delay and jitter in multimedia delivery

In this paper we present the results of a study that examines the user’s perception—understood as both information assimilation and subjective satisfaction—of multimedia quality, when impacted by varying network-level parameters (delay and jitter). In addition, we integrate eye-tracking assessment to provide a more complete understanding of user perception of multimedia quality. Results show that delay and jitter significantly affect user satisfaction; variation in video eye path when either no single/obvious point of focus exists or when the point of attention changes dramatically. Lastly, results showed that content variation significantly affected user satisfaction, as well as user information assimilation

ARTSCENE: A Neural System for Natural Scene Classification

How do humans rapidly recognize a scene? How can neural models capture this biological competence to achieve state-of-the-art scene classification? The ARTSCENE neural system classifies natural scene photographs by using multiple spatial scales to efficiently accumulate evidence for gist and texture. ARTSCENE embodies a coarse-to-fine Texture Size Ranking Principle whereby spatial attention processes multiple scales of scenic information, ranging from global gist to local properties of textures. The model can incrementally learn and predict scene identity by gist information alone and can improve performance through selective attention to scenic textures of progressively smaller size. ARTSCENE discriminates 4 landscape scene categories (coast, forest, mountain and countryside) with up to 91.58% correct on a test set, outperforms alternative models in the literature which use biologically implausible computations, and outperforms component systems that use either gist or texture information alone. Model simulations also show that adjacent textures form higher-order features that are also informative for scene recognition.National Science Foundation (NSF SBE-0354378); Office of Naval Research (N00014-01-1-0624