Scan path visualization and comparison using visual aggregation techniques

We demonstrate the use of different visual aggregation techniques to obtain non-cluttered visual representations of scanpaths. First, fixation points are clustered using the mean-shift algorithm. Second, saccades are aggregated using the Attribute-Driven Edge Bundling (ADEB) algorithm that handles a saccades direction, onset timestamp, magnitude or their combination for the edge compatibility criterion. Flow direction maps, computed during bundling, can be visualized separately (vertical or horizontal components) or as a single image using the Oriented Line Integral Convolution (OLIC) algorithm. Furthermore, cosine similarity between two flow direction maps provides a similarity map to compare two scanpaths. Last, we provide examples of basic patterns, visual search task, and art perception. Used together, these techniques provide valuable insights about scanpath exploration and informative illustrations of the eye movement data

Mobile Consumer Behavior in Fashion m-Retail: An Eye Tracking Study to Understand Gender Differences

© 2020 ACM. With exponential adoption of mobile devices, consumers increasingly use them for shopping. There is a need to understand the gender differences in mobile consumer behavior. This study used mobile eye tracking technology and mixed-method approach to analyze and compare how male and female mobile fashion consumers browse and shop on smartphones. Mobile eye tracking glasses recorded fashion consumers' shopping experiences using smartphones for browsing and shopping on the actual fashion retailer's website. 14 participants successfully completed this study, half of them were males and half females. Two different data analysis approaches were employed, namely a novel framework of the shopping journey, and semantic gaze mapping with 31 Areas of Interest (AOI) representing the elements of the shopping journey. The results showed that male and female users exhibited significantly different behavior patterns, which have implications for mobile website design and fashion m-retail. The shopping journey map framework proves useful for further application in market research

Development of salience-driven and visually-guided eye movement responses

Development of visuospatial attention can be quantified from infancy onward using visually-guided eye movement responses. We investigated the interaction between eye movement response times and salience in target areas of visual stimuli over age in a cohort of typically developing children. A preferential looking (PL) paradigm consisting of stimuli with six different visual modalities (cartoons, contrast, form, local motion, color, global motion) was combined with the automated measurement of reflexive eye movements. Effective salience was defined as visual salience of each target area relative to its background. Three classes of PL stimuli were used: with high- (cartoon, contrast), intermediate- (local motion, form), and low-effective salience (global motion, color). Eye movement response times to the target areas of the six PL stimuli were nonverbally assessed in 220 children aged 1-12 years. The development

Analysis of Eye-Tracking Data in Visualization and Data Space

Eye-tracking devices can tell us where on the screen a person is looking. Researchers frequently analyze eye-tracking data manually, by examining every frame of a visual stimulus used in an eye-tracking experiment so as to match 2D screen-coordinates provided by the eye-tracker to related objects and content within the stimulus. Such task requires significant manual effort and is not feasible for analyzing data collected from many users, long experimental sessions, and heavily interactive and dynamic visual stimuli. In this dissertation, we present a novel analysis method. We would instrument visualizations that have open source code, and leverage real-time information about the layout of the rendered visual content, to automatically relate gaze-samples to visual objects drawn on the screen. Since such visual objects are shown in a visualization stand for data, the method would allow us to necessarily detect data that users focus on or Data of Interest (DOI). This dissertation has two contributions. First, we demonstrated the feasibility of collecting DOI data for real life visualization in a reliable way which is not self-evident. Second, we formalized the process of collecting and interpreting DOI data and test whether the automated DOI detection can lead to research workflows, and insights not possible with traditional, manual approaches

Reinforcement learning approaches to the analysis of the emergence of goal-directed behaviour

Over recent decades, theoretical neuroscience, helped by computational methods such as Reinforcement Learning (RL), has provided detailed descriptions of the psychology and neurobiology of decision-making. RL has provided many insights into the mechanisms underlying decision-making processes from neuronal to behavioral levels. In this work, we attempt to demonstrate the effectiveness of RL methods in explaining behavior in a normative setting through three main case studies. Evidence from literature shows that, apart from the commonly discussed cognitive search process, that governs the solution procedure of a planning task, there is an online perceptual process that directs the action selection towards moves that appear more ‘natural’ at a given configuration of a task. These two processes can be partially dissociated through developmental studies, with perceptual processes apparently more dominant in the planning of younger children, prior to the maturation of executive functions required for the control of search. Therefore, we present a formalization of planning processes to account for perceptual features of the task, and relate it to human data. Although young children are able to demonstrate their preferences by using physical actions, infants are restricted because of their as-yet-undeveloped motor skills. Eye-tracking methods have been employed to tackle this difficulty. Exploring different model-free RL algorithms and their possible cognitive realizations in decision making, in a second case study, we demonstrate behavioral signatures of decision making processes in eye-movement data and provide a potential framework for integrating eye-movement patterns with behavioral patterns. Finally, in a third project we examine how uncertainty in choices might guide exploration in 10-year-olds, using an abstract RL-based mathematical model. Throughout, aspects of action selection are seen as emerging from the RL computational framework. We, thus, conclude that computational descriptions of the developing decision making functions provide one plausible avenue by which to normatively characterize and define the functions that control action selection

The medical pause in simulation training

The medical pause, stopping current performance for a short time for additional cognitive activities, can potentially advance patient safety and learning in medicine. Yet, to date, we do not have a theoretical understanding of why pausing skills should be taught as a professional skill, nor empirical evidence of how pausing affects performance and learning. To address this gap, this thesis investigates the effects of pausing in medical training theoretically and empirically. For the empirical investigation, a computer-based simulation was used for the task environment, and eye-tracking and log data to assess performance

Infant and Child Multisensory Attention Skills: Methods, Measures, and Language Outcomes

Intersensory processing (e.g., matching sights and sounds based on audiovisual synchrony) is thought to be a foundation for more complex developmental outcomes including language. However, the body of research on intersensory processing is characterized by different measures, paradigms, and research questions, making comparisons across studies difficult. Therefore, Manuscript 1 provides a systematic review and synthesis of research on intersensory processing, integrating findings across multiple methods, along with recommendations for future research. This includes a call for a shift in the focus of intersensory processing research from that of assessing average performance of groups of infants, to one assessing individual differences in intersensory processing. Individual difference measures allow researchers to assess developmental trajectories and understand developmental pathways from basic skills to later outcomes. Bahrick and colleagues introduced the first two new individual difference measures of intersensory processing: The Multisensory Attention Assessment Protocol (MAAP) and The Intersensory Processing Efficiency Protocol (IPEP). My prior research using the MAAP has shown that accuracy of intersensory processing at 12 months of age predicted 18- and 24-month child language outcomes. Moreover, it predicted child language to a greater extent than well-established predictors, including parent language input and SES (Edgar et al., under review)! Manuscript 2 extends this research to examine both speed and accuracy of intersensory processing using the IPEP. A longitudinal sample of 103 infants were tested with the IPEP to assess relations between intersensory processing at 6 months of age and language outcomes at 18, 24, and 36 months, while controlling for traditional predictors, parent language input and SES. Results demonstrate that even at 6 months, intersensory processing predicts 18-, 24-, and 36-month child language skills, over and above the traditional predictors. This novel finding reveals the powerful role of intersensory processing in shaping language development and highlights the importance of incorporating individual differences in intersensory processing as a predictor in models of developmental pathways to language. In turn, these findings can inform interventions where intersensory processing can be used as an early screener for children at risk for language delays

Reinforcement learning approaches to the analysis of the emergence of goal-directed behaviour

Over recent decades, theoretical neuroscience, helped by computational methods such as Reinforcement Learning (RL), has provided detailed descriptions of the psychology and neurobiology of decision-making. RL has provided many insights into the mechanisms underlying decision-making processes from neuronal to behavioral levels. In this work, we attempt to demonstrate the effectiveness of RL methods in explaining behavior in a normative setting through three main case studies. Evidence from literature shows that, apart from the commonly discussed cognitive search process, that governs the solution procedure of a planning task, there is an online perceptual process that directs the action selection towards moves that appear more ‘natural’ at a given configuration of a task. These two processes can be partially dissociated through developmental studies, with perceptual processes apparently more dominant in the planning of younger children, prior to the maturation of executive functions required for the control of search. Therefore, we present a formalization of planning processes to account for perceptual features of the task, and relate it to human data. Although young children are able to demonstrate their preferences by using physical actions, infants are restricted because of their as-yet-undeveloped motor skills. Eye-tracking methods have been employed to tackle this difficulty. Exploring different model-free RL algorithms and their possible cognitive realizations in decision making, in a second case study, we demonstrate behavioral signatures of decision making processes in eye-movement data and provide a potential framework for integrating eye-movement patterns with behavioral patterns. Finally, in a third project we examine how uncertainty in choices might guide exploration in 10-year-olds, using an abstract RL-based mathematical model. Throughout, aspects of action selection are seen as emerging from the RL computational framework. We, thus, conclude that computational descriptions of the developing decision making functions provide one plausible avenue by which to normatively characterize and define the functions that control action selection