Psychophysiology-based QoE assessment : a survey

We present a survey of psychophysiology-based assessment for quality of experience (QoE) in advanced multimedia technologies. We provide a classification of methods relevant to QoE and describe related psychological processes, experimental design considerations, and signal analysis techniques. We summarize multimodal techniques and discuss several important aspects of psychophysiology-based QoE assessment, including the synergies with psychophysical assessment and the need for standardized experimental design. This survey is not considered to be exhaustive but serves as a guideline for those interested to further explore this emerging field of research

Biosignalų požymių regos diskomfortui vertinti išskyrimas ir tyrimas

Comfortable stereoscopic perception continues to be an essential area of research. The growing interest in virtual reality content and increasing market for head-mounted displays (HMDs) still cause issues of balancing depth perception and comfortable viewing. Stereoscopic views are stimulating binocular cues – one type of several available human visual depth cues which becomes conflicting cues when stereoscopic displays are used. Depth perception by binocular cues is based on matching of image features from one retina with corresponding features from the second retina. It is known that our eyes can tolerate small amounts of retinal defocus, which is also known as Depth of Focus. When magnitudes are larger, a problem of visual discomfort arises. The research object of the doctoral dissertation is a visual discomfort level. This work aimed at the objective evaluation of visual discomfort, based on physiological signals. Different levels of disparity and the number of details in stereoscopic views in some cases make it difficult to find the focus point for comfortable depth perception quickly. During this investigation, a tendency for differences in single sensor-based electroencephalographic EEG signal activity at specific frequencies was found. Additionally, changes in eye tracker collected gaze signals were also found. A dataset of EEG and gaze signal records from 28 control subjects was collected and used for further evaluation. The dissertation consists of an introduction, three chapters and general conclusions. The first chapter reveals the fundamental knowledge ways of measuring visual discomfort based on objective and subjective methods. In the second chapter theoretical research results are presented. This research was aimed to investigate methods which use physiological signals to detect changes on the level of sense of presence. Results of the experimental research are presented in the third chapter. This research aimed to find differences in collected physiological signals when a level of visual discomfort changes. An experiment with 28 control subjects was conducted to collect these signals. The results of the thesis were published in six scientific publications – three in peer-reviewed scientific papers, three in conference proceedings. Additionally, the results of the research were presented in 8 conferences.Dissertatio

An Overview of Self-Adaptive Technologies Within Virtual Reality Training

This overview presents the current state-of-the-art of self-adaptive technologies within virtual reality (VR) training. Virtual reality training and assessment is increasingly used for five key areas: medical, industrial & commercial training, serious games, rehabilitation and remote training such as Massive Open Online Courses (MOOCs). Adaptation can be applied to five core technologies of VR including haptic devices, stereo graphics, adaptive content, assessment and autonomous agents. Automation of VR training can contribute to automation of actual procedures including remote and robotic assisted surgery which reduces injury and improves accuracy of the procedure. Automated haptic interaction can enable tele-presence and virtual artefact tactile interaction from either remote or simulated environments. Automation, machine learning and data driven features play an important role in providing trainee-specific individual adaptive training content. Data from trainee assessment can form an input to autonomous systems for customised training and automated difficulty levels to match individual requirements. Self-adaptive technology has been developed previously within individual technologies of VR training. One of the conclusions of this research is that while it does not exist, an enhanced portable framework is needed and it would be beneficial to combine automation of core technologies, producing a reusable automation framework for VR training

DeepMetricEye: Metric Depth Estimation in Periocular VR Imagery

Despite the enhanced realism and immersion provided by VR headsets, users frequently encounter adverse effects such as digital eye strain (DES), dry eye, and potential long-term visual impairment due to excessive eye stimulation from VR displays and pressure from the mask. Recent VR headsets are increasingly equipped with eye-oriented monocular cameras to segment ocular feature maps. Yet, to compute the incident light stimulus and observe periocular condition alterations, it is imperative to transform these relative measurements into metric dimensions. To bridge this gap, we propose a lightweight framework derived from the U-Net 3+ deep learning backbone that we re-optimised, to estimate measurable periocular depth maps. Compatible with any VR headset equipped with an eye-oriented monocular camera, our method reconstructs three-dimensional periocular regions, providing a metric basis for related light stimulus calculation protocols and medical guidelines. Navigating the complexities of data collection, we introduce a Dynamic Periocular Data Generation (DPDG) environment based on UE MetaHuman, which synthesises thousands of training images from a small quantity of human facial scan data. Evaluated on a sample of 36 participants, our method exhibited notable efficacy in the periocular global precision evaluation experiment, and the pupil diameter measurement

Human-Computer Interaction

In this book the reader will find a collection of 31 papers presenting different facets of Human Computer Interaction, the result of research projects and experiments as well as new approaches to design user interfaces. The book is organized according to the following main topics in a sequential order: new interaction paradigms, multimodality, usability studies on several interaction mechanisms, human factors, universal design and development methodologies and tools

Investigation of Pilots\u27 Visual Entropy and Eye Fixations for Simulated Flights Consisted of Multiple Take-Offs and Landings

Eye movement characteristics might provide insights on pilots\u27 mental fatigue during prolonged flight. The visual entropy, eye fixation numbers, and eye fixation durations of ten novice pilots and ten expert pilots were analyzed for a four-hour simulated flight task consisting of four consecutive flight legs. Each flight leg lasted approximately one hour and contained five flight phases: takeoff, climb, cruise, descend, and landing. The pilots maneuvered the simulated B-52 aircraft following instrument flight rules (IFR) in a moderate-fidelity Microsoft Flight Simulator environment. Our results indicate that experts’ eye movement characteristics were significantly different from those of novices. In detail, novices\u27 eye movements were more random, produced longer eye fixation durations, and had fewer eye fixation numbers on the areas of interest (AOIs) than the experts. In addition, the repetitive task (i.e., four consecutive flights) significantly impacted the eye movement characteristics for both experts and novices. Visual entropy and eye fixation duration increased, while eye fixation numbers decreased for both groups as the repetition index increased. Finally, the flight phases also affected eye movement characteristics. The results show that both experts\u27 and novices\u27 visual entropies were relatively higher during climb, cruise, and descend phases, whereas those were relatively lower during the takeoff and landing phases. The present results provide a foundation for us to better understand the similarities and dissimilarities of eye movement characteristics between the experts and novices for a prolonged flight. Lastly, potential scaffolding training methods and pilot anomaly alerting systems, derived from such eye movements, are introduced

A Comparative Emotions-detection Review for Non-intrusive Vision-Based Facial Expression Recognition

Affective computing advocates for the development of systems and devices that can recognize, interpret, process, and simulate human emotion. In computing, the field seeks to enhance the user experience by finding less intrusive automated solutions. However, initiatives in this area focus on solitary emotions that limit the scalability of the approaches. Further reviews conducted in this area have also focused on solitary emotions, presenting challenges to future researchers when adopting these recommendations. This review aims at highlighting gaps in the application areas of Facial Expression Recognition Techniques by conducting a comparative analysis of various emotion detection datasets, algorithms, and results provided in existing studies. The systematic review adopted the PRISMA model and analyzed eighty-three publications. Findings from the review show that different emotions call for different Facial Expression Recognition techniques, which should be analyzed when conducting Facial Expression Recognition. Keywords: Facial Expression Recognition, Emotion Detection, Image Processing, Computer Visio

Computational Analysis of Eye-Strain for Digital Screens based on Eye Tracking Studies

Computer vision syndrome (CVS) is composed of multiple eye vision problems due to the prolonged use of digital displays, including tablets and smartphones. These problems were shown to affect visual comfort as well as work productivity in both adults and teenagers. CVS causes eye and vision symptoms such as eye-strain, eye burn, dry eyes, double vision, and blurred vision. CVS, which causes severe vision and muscular problems due to repeated eye movements and excessive eye focus on computer screens, is a cause of work-related stress. In this thesis, we address this problem and present three general-purpose mathematical compound models for assessing eye-strain in eye-tracking applications, namely (1) Fixation-based Eye fatigue Load Index (FELiX), (2) Index of Difficulty for Eye-tracking Applications (IDEA), and (3) Eye-Strain Probation Model (ESPiM) based on eye-tracking parameters and subjective ratings to measure, predict, and compare the amount of fatigue or cognitive workload during target selection tasks for different user groups or interaction techniques. The ESPiM model is the outcome of both FELiX and IDEA, which benefit from direct subjective rating and, therefore, can be applied to assess the ESPiM model's efficacy. We present experiments and user studies that show that these models can measure potential eye-strain levels on individuals based on physical circumstances such as screen resolution and target positions per time

User-centered Virtual Environment Assessment And Design For Cognitive Rehabilitation Applications

Virtual environment (VE) design for cognitive rehabilitation necessitates a new methodology to ensure the validity of the resulting rehabilitation assessment. We propose that benchmarking the VE system technology utilizing a user-centered approach should precede the VE construction. Further, user performance baselines should be measured throughout testing as a control for adaptive effects that may confound the metrics chosen to evaluate the rehabilitation treatment. To support these claims we present data obtained from two modules of a user-centered head-mounted display (HMD) assessment battery, specifically resolution visual acuity and stereoacuity. Resolution visual acuity and stereoacuity assessments provide information about the image quality achieved by an HMD based upon its unique system parameters. When applying a user-centered approach, we were able to quantify limitations in the VE system components (e.g., low microdisplay resolution) and separately point to user characteristics (e.g., changes in dark focus) that may introduce error in the evaluation of VE based rehabilitation protocols. Based on these results, we provide guidelines for calibrating and benchmarking HMDs. In addition, we discuss potential extensions of the assessment to address higher level usability issues. We intend to test the proposed framework within the Human Experience Modeler (HEM), a testbed created at the University of Central Florida to evaluate technologies that may enhance cognitive rehabilitation effectiveness. Preliminary results of a feasibility pilot study conducted with a memory impaired participant showed that the HEM provides the control and repeatability needed to conduct such technology comparisons. Further, the HEM affords the opportunity to integrate new brain imaging technologies (i.e., functional Near Infrared Imaging) to evaluate brain plasticity associated with VE based cognitive rehabilitation