96 research outputs found
VR.net: A Real-world Dataset for Virtual Reality Motion Sickness Research
Researchers have used machine learning approaches to identify motion sickness
in VR experience. These approaches demand an accurately-labeled, real-world,
and diverse dataset for high accuracy and generalizability. As a starting point
to address this need, we introduce `VR.net', a dataset offering approximately
12-hour gameplay videos from ten real-world games in 10 diverse genres. For
each video frame, a rich set of motion sickness-related labels, such as
camera/object movement, depth field, and motion flow, are accurately assigned.
Building such a dataset is challenging since manual labeling would require an
infeasible amount of time. Instead, we utilize a tool to automatically and
precisely extract ground truth data from 3D engines' rendering pipelines
without accessing VR games' source code. We illustrate the utility of VR.net
through several applications, such as risk factor detection and sickness level
prediction. We continuously expand VR.net and envision its next version
offering 10X more data than the current form. We believe that the scale,
accuracy, and diversity of VR.net can offer unparalleled opportunities for VR
motion sickness research and beyond
Real-Time Viewport-Aware Optical Flow Estimation in 360-degree Videos for Visually-Induced Motion Sickness Mitigation
Visually-induced motion sickness (VIMS), a side effect of perceived motion
caused by visual stimulation, is a major obstacle to the widespread use of
Virtual Reality (VR). Along with scene object information, visual stimulation
can be primarily indicated by optical flow, which characterizes the motion
pattern, such as the intensity and direction of the moving image. We estimated
the real time optical flow in 360-degree videos targeted at immersive user
interactive visualization based on the user's current viewport. The proposed
method allows the estimation of customized visual flow for each experience of
dynamic 360-degree videos and is an improvement over previous methods that
consider a single optical flow value for the entire equirectangular frame. We
applied our method to modulate the opacity of granulated rest frames (GRFs), a
technique consisting of visual noise-like randomly distributed visual
references that are stable to the user's body during immersive pre-recorded
360-degree video experience. We report the results of a pilot one-session
between-subject study with 18 participants, where users watched a 2-minute
high-intensity 360-degree video. The results show that our proposed method
successfully estimates optical flow, with pilot data showing that GRFs combined
with real-time optical flow estimation may improve user comfort when watching
360-degree videos. However, more data are needed for statistically significant
results
Stereoscopic Omnidirectional Image Quality Assessment Based on Predictive Coding Theory
Objective quality assessment of stereoscopic omnidirectional images is a
challenging problem since it is influenced by multiple aspects such as
projection deformation, field of view (FoV) range, binocular vision, visual
comfort, etc. Existing studies show that classic 2D or 3D image quality
assessment (IQA) metrics are not able to perform well for stereoscopic
omnidirectional images. However, very few research works have focused on
evaluating the perceptual visual quality of omnidirectional images, especially
for stereoscopic omnidirectional images. In this paper, based on the predictive
coding theory of the human vision system (HVS), we propose a stereoscopic
omnidirectional image quality evaluator (SOIQE) to cope with the
characteristics of 3D 360-degree images. Two modules are involved in SOIQE:
predictive coding theory based binocular rivalry module and multi-view fusion
module. In the binocular rivalry module, we introduce predictive coding theory
to simulate the competition between high-level patterns and calculate the
similarity and rivalry dominance to obtain the quality scores of viewport
images. Moreover, we develop the multi-view fusion module to aggregate the
quality scores of viewport images with the help of both content weight and
location weight. The proposed SOIQE is a parametric model without necessary of
regression learning, which ensures its interpretability and generalization
performance. Experimental results on our published stereoscopic omnidirectional
image quality assessment database (SOLID) demonstrate that our proposed SOIQE
method outperforms state-of-the-art metrics. Furthermore, we also verify the
effectiveness of each proposed module on both public stereoscopic image
datasets and panoramic image datasets
MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images
We introduce a method to convert stereo 360{\deg} (omnidirectional stereo)
imagery into a layered, multi-sphere image representation for six
degree-of-freedom (6DoF) rendering. Stereo 360{\deg} imagery can be captured
from multi-camera systems for virtual reality (VR), but lacks motion parallax
and correct-in-all-directions disparity cues. Together, these can quickly lead
to VR sickness when viewing content. One solution is to try and generate a
format suitable for 6DoF rendering, such as by estimating depth. However, this
raises questions as to how to handle disoccluded regions in dynamic scenes. Our
approach is to simultaneously learn depth and disocclusions via a multi-sphere
image representation, which can be rendered with correct 6DoF disparity and
motion parallax in VR. This significantly improves comfort for the viewer, and
can be inferred and rendered in real time on modern GPU hardware. Together,
these move towards making VR video a more comfortable immersive medium.Comment: 25 pages, 13 figures, Published at European Conference on Computer
Vision (ECCV 2020), Project Page: http://visual.cs.brown.edu/matryodshk
VR Sickness Prediction for Navigation in Immersive Virtual Environments using a Deep Long Short Term Memory Model
International audienceThis paper proposes a new objective metric of visually induced motion sickness (VIMS) in the context of navigation in virtual environments (VEs). Similar to motion sickness in physical environments, VIMS can induce many physiological symptoms such as general discomfort, nausea, disorientation, vomiting, dizziness and fatigue. To improve user satisfaction with VR applications, it is of great significance to develop objective metrics for VIMS that can analyze and estimate the level of VR sickness when a user is exposed to VEs. One of the well-known objective metrics is the postural instability. In this paper, we trained a LSTM model for each participant using a normal-state postural signal captured before the exposure, and if the postural sway signal from post-exposure was sufficiently different from the pre-exposure signal, the model would fail at encoding and decoding the signal properly; the jump in the reconstruction error was called loss and was proposed as the proposed objective measure of simulator sickness. The effectiveness of the proposed metric was analyzed and compared with subjective assessment methods based on the simulator sickness questionnaire (SSQ) in a VR environment, achieving a Pearson correlation coefficient of .89. Finally, we showed that the proposed method had the potential to be deployed within a closed-loop system and get real-time performance to predict VR sickness, opening new insights to develop user-centered and customized VR applications based on physiological feedback
Machine learning methods for the study of cybersickness: a systematic review
This systematic review offers a world-first critical analysis of machine learning methods and systems, along with future directions for the study of cybersickness induced by virtual reality (VR). VR is becoming increasingly popular and is an important part of current advances in human training, therapies, entertainment, and access to the metaverse. Usage of this technology is limited by cybersickness, a common debilitating condition experienced upon VR immersion. Cybersickness is accompanied by a mix of symptoms including nausea, dizziness, fatigue and oculomotor disturbances. Machine learning can be used to identify cybersickness and is a step towards overcoming these physiological limitations. Practical implementation of this is possible with optimised data collection from wearable devices and appropriate algorithms that incorporate advanced machine learning approaches. The present systematic review focuses on 26 selected studies. These concern machine learning of biometric and neuro-physiological signals obtained from wearable devices for the automatic identification of cybersickness. The methods, data processing and machine learning architecture, as well as suggestions for future exploration on detection and prediction of cybersickness are explored. A wide range of immersion environments, participant activity, features and machine learning architectures were identified. Although models for cybersickness detection have been developed, literature still lacks a model for the prediction of first-instance events. Future research is pointed towards goal-oriented data selection and labelling, as well as the use of brain-inspired spiking neural network models to achieve better accuracy and understanding of complex spatio-temporal brain processes related to cybersickness
- …