The platformer experience dataset

Player modeling and estimation of player experience have become very active research fields within affective computing, human computer interaction, and game artificial intelligence in recent years. For advancing our knowledge and understanding on player experience this paper introduces the Platformer Experience Dataset (PED) - the first open-access game experience corpus - that contains multiple modalities of user data of Super Mario Bros players. The open-access database aims to be used for player experience capture through context-based (i.e. game content), behavioral and visual recordings of platform game players. In addition, the database contains demographical data of the players and self-reported annotations of experience in two forms: ratings and ranks. PED opens up the way to desktop and console games that use video from webcameras and visual sensors and offer possibilities for holistic player experience modeling approaches that can, in turn, yield richer game personalization.peer-reviewe

Automatic Gaze Classification for Aviators: Using Multi-task Convolutional Networks as a Proxy for Flight Instructor Observation

In this work, we investigate how flight instructors observe aviator scan patterns and assign quality to an aviator\u27s gaze. We first establish the reliability of instructors to assign similar quality to an aviator\u27s scan patterns, and then investigate methods to automate this quality using machine learning. In particular, we focus on the classification of gaze for aviators in a mixed-reality flight simulation. We create and evaluate two machine learning models for classifying gaze quality of aviators: a task-agnostic model and a multi-task model. Both models use deep convolutional neural networks to classify the quality of pilot gaze patterns for 40 pilots, operators, and novices, as compared to visual inspection by three experienced flight instructors. Our multi-task model can automate the process of gaze inspection with an average accuracy of over 93.0% for three separate flight tasks. Our approach could assist existing flight instructors to provide feedback to learners, or it could open the door to more automated feedback for pilots learning to carry out different maneuvers

3차원 의료 영상 판독 시선 정보의 대화형 시각적 분석 프레임워크

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2016. 2. 서진욱.We propose an interactive visual analytics framework for diagnostic gaze data on volumetric medical images. The framework is designed to compare gaze data from multiple readers with effective visualizations, which are tailored for volumetric gaze data with additional contextual information. Gaze pattern comparison is essential to understand how radiologists examine medical images and to identify factors influencing the examination. However, prior work on diagnostic gaze data using the medical images acquired from volumetric imaging systems (e.g., computed tomography or magnetic resonance imaging) showed a number of limitations in comparative analysis. In the diagnosis, radiologists scroll through a stack of images to get a 3D cognition of organs and lesions that resulting gaze patterns contain additional depth information compared to the gaze tracking study with 2D stimuli. As a result, the additional spatial dimension aggravated the complexity on visual representation of gaze data. A recent work proposed a visualization design based on direct volume rendering (DVR) for gaze patterns in volumetric imageshowever, effective and comprehensive gaze pattern comparison is still challenging due to lack of interactive visualization tools for comparative gaze analysis. In this paper, we first present an effective visual representation, and propose an interactive analytics framework for multiple volumetric gaze data. We also take the challenge integrating crucial contextual information such as pupil size and windowing (i.e., adjusting brightness and contrast of image) into the analysis process for more in-depth and ecologically valid findings. Among the interactive visualization components, a context-embedded interactive scatterplot (CIS) is especially designed to help users to examine abstract gaze data in diverse contexts by embedding medical imaging representations well-known to radiologists in it. We also present the results from case studies with chest and abdominal radiologistsChapter 1 Introduction 1 1.1 Background 1 1.2 Research Components 5 1.3 Radiological Practice 6 1.4 Organization of the Dissertation 8 Chapter 2 Related Work 9 2.1 Visualization Combining 2D and 3D 9 2.2 Eye Tracking Data Visualization 14 2.3 Comparative Data Analysis 16 2.4 Gaze Analysis in the Medical field 18 Chapter 3 GazeVis: Volumetric Gaze Data 21 3.1 Visualization of Stimuli and Gaze Data 23 3.1.1 Computation of Gaze Field 26 3.1.2 Visualization of Gaze Field 29 3.1.3 Gaze Field for Interactive Information Seeking 30 3.2 Interactions and Dynamic Queries 32 3.2.1 Interaction Design 32 3.2.2 Spatial Filtering 34 3.2.3 Temporal Filtering 34 3.2.4 Transfer Function Control 36 3.2.5 Gaussian Blur Control 38 3.3 Implementation 38 3.4 Evaluation with Radiologists 38 3.4.1 Case Study Protocol 39 3.4.2 Datasets 41 3.4.3 Apparatus 42 3.4.4 Chest Radiologists 42 3.4.5 Abdominal Radiologists 45 3.5 Discussion 49 3.5.1 Spatial Data Structure and Flexibility 49 3.5.2 Interacting with Contextual Data 51 Chapter 4 GazeDx: Interactive Gaze Analysis Framework 53 4.1 Design Rationale 54 4.2 Overviews for Comparative Gaze Analysis 57 4.2.1 Spatial Similarity 57 4.2.2 Qualitative Similarity Overview 58 4.2.3 Multi-level Temporal Overview 60 4.3 In-depth Comparison of Gaze Patterns 65 4.3.1 Detail Views for Individual Readers 65 4.3.2 Aggregation for Group Comparison 67 4.4 CIS: Context-embedded Interactive Scatterplot 68 4.4.1 Flexible Axis Configuration 68 4.4.2 Focus Attention with Familiar Representations 69 4.4.3 Scatterplot Matrix with CIS 72 4.5 Interactive Selection and Filtering 72 4.5.1 Selection by Freehand Drawing 73 4.5.2 Selection by Human Anatomy 74 4.6 Implementation 76 4.7 Case Studies 77 4.7.1 Case Study Protocol 78 4.7.2 Apparatus 80 4.7.3 Case Study 1: Chest Radiologists 81 4.7.4 Case Study 2: Abdominal Radiologists 85 4.8 Discussion 88 Chapter 5 Conclusion 91 Bibliography 94 Abstract in Korean 105Docto

Advanced Gaze Visualizations for Three-dimensional Virtual Environments

Gaze visualizations represent an effective way for gaining fast insights into eye tracking data. Current approaches do not adequately support eye tracking studies for three-dimensional (3D) virtual environments. Hence, we propose a set of advanced gaze visualization techniques for supporting gaze behavior analysis in such environments. Similar to commonly used gaze visualizations for two-dimensional stimuli (e.g., images and websites), we contribute advanced 3D scan paths and 3D attentional maps. In addition, we introduce a models-of-interest timeline depicting viewed models, which can be used for displaying scan paths in a selected time segment. A prototype toolkit is also discussed which combines an implementation of our proposed techniques. Their potential for facilitating eye tracking studies in virtual environments was supported by a user study among eye tracking and visualization experts

3D Design Review Systems in Immersive Environments

Design reviews play a crucial role in the development process, ensuring the quality and effectiveness of designs in various industries. However, traditional design review methods face challenges in effectively understanding and communicating complex 3D models. Immersive technologies, particularly Head-Mounted Displays (HMDs), offer new opportunities to enhance the design review process. In this thesis, we investigate using immersive environments, specifically HMDs, for 3D design reviews. We begin with a systematic literature review to understand the current state of employing HMDs in industry for design reviews. As part of this review, we utilize a detailed taxonomy from the literature to categorize and analyze existing approaches. Additionally, we present four iterations of an immersive design review system developed during my industry experience. Two of these iterations are evaluated through case studies involving domain experts, including engineers, designers, and clients. A formal semi-structured focus group is conducted to gain further insights into traditional design review practices. The outcomes of these evaluations and the focus group discussions are thoroughly discussed. Based on the literature review and the focus group findings, we uncover a new challenge associated with using HMDs in immersive design reviews—asynchronous and remote collaboration. Unlike traditional design reviews, where participants view the same section on a shared screen, HMDs allow independent exploration of areas of interest, leading to a shift from synchronous to asynchronous communication. Consequently, important feedback may be missed as the lead designer disconnects from the users' perspectives. To address this challenge, we collaborate with a domain expert to develop a prototype that utilizes heatmap visualization to display 3D gaze data distribution. This prototype enables lead designers to quickly identify areas of review and missed regions. The study incorporates the Design Critique approach and provides valuable insights into different heatmap visualization variants (top view projection, object-based, and volume-based). Furthermore, a list of well-defined requirements is outlined for future spatio-temporal visualization applications aimed at integrating into existing workflows. Overall, this thesis contributes to the understanding and improvement of immersive design review systems, particularly in the context of utilizing HMDs. It offers insights into the current state of employing HMDs for design reviews, utilizes a taxonomy from the literature to analyze existing approaches, highlights challenges associated with asynchronous collaboration, and proposes a prototype solution with heatmap visualization to address the identified challenge

Promoting Andean children's learning of science through cultural and digital tools

Conference Theme: To see the world and a grain of sand: Learning across levels of space, time, and scaleIn Peru, there is a large achievement gap in rural schools. In order to overcome this problem, the study aims to design environments that enhance science learning through the integration of ICT with cultural artifacts, respecting the Andean culture and empower rural children to pursue lifelong learning. This investigation employs the Cultural-Historical Activity Theory (CHAT) framework, and the Design-Based Research (DBR) methodology using an iterative process of design, implementation and evaluation of the innovative practice.published_or_final_versio