Methods for reducing visual discomfort in stereoscopic 3D: A review

This work was supported by the EPSRC Grant EP/M01469X/1, “Geometric Evaluation of Stereoscopic Video”

Remote Visual Observation of Real Places Through Virtual Reality Headsets

Virtual Reality has always represented a fascinating yet powerful opportunity that has attracted studies and technology developments, especially since the latest release on the market of powerful high-resolution and wide field-of-view VR headsets. While the great potential of such VR systems is common and accepted knowledge, issues remain related to how to design systems and setups capable of fully exploiting the latest hardware advances. The aim of the proposed research is to study and understand how to increase the perceived level of realism and sense of presence when remotely observing real places through VR headset displays. Hence, to produce a set of guidelines that give directions to system designers about how to optimize the display-camera setup to enhance performance, focusing on remote visual observation of real places. The outcome of this investigation represents unique knowledge that is believed to be very beneficial for better VR headset designs towards improved remote observation systems. To achieve the proposed goal, this thesis presents a thorough investigation of existing literature and previous researches, which is carried out systematically to identify the most important factors ruling realism, depth perception, comfort, and sense of presence in VR headset observation. Once identified, these factors are further discussed and assessed through a series of experiments and usability studies, based on a predefined set of research questions. More specifically, the role of familiarity with the observed place, the role of the environment characteristics shown to the viewer, and the role of the display used for the remote observation of the virtual environment are further investigated. To gain more insights, two usability studies are proposed with the aim of defining guidelines and best practices. The main outcomes from the two studies demonstrate that test users can experience an enhanced realistic observation when natural features, higher resolution displays, natural illumination, and high image contrast are used in Mobile VR. In terms of comfort, simple scene layouts and relaxing environments are considered ideal to reduce visual fatigue and eye strain. Furthermore, sense of presence increases when observed environments induce strong emotions, and depth perception improves in VR when several monocular cues such as lights and shadows are combined with binocular depth cues. Based on these results, this investigation then presents a focused evaluation on the outcomes and introduces an innovative eye-adapted High Dynamic Range (HDR) approach, which the author believes to be of great improvement in the context of remote observation when combined with eye-tracked VR headsets. Within this purpose, a third user study is proposed to compare static HDR and eye-adapted HDR observation in VR, to assess that the latter can improve realism, depth perception, sense of presence, and in certain cases even comfort. Results from this last study confirmed the author expectations, proving that eye-adapted HDR and eye tracking should be used to achieve best visual performances for remote observation in modern VR systems

Visual discomfort whilst viewing 3D stereoscopic stimuli

3D stereoscopic technology intensifies and heightens the viewer s experience by adding an extra dimension to the viewing of visual content. However, with expansion of this technology to the commercial market concerns have been expressed about the potential negative effects on the visual system, producing viewer discomfort. The visual stimulus provided by a 3D stereoscopic display differs from that of the real world, and so it is important to understand whether these differences may pose a health hazard. The aim of this thesis is to investigate the effect of 3D stereoscopic stimulation on visual discomfort. To that end, four experimental studies were conducted. In the first study two hypotheses were tested. The first hypothesis was that the viewing of 3D stereoscopic stimuli, which are located geometrically beyond the screen on which the images are displayed, would induce adaptation changes in the resting position of the eyes (exophoric heterophoria changes). The second hypothesis was that participants whose heterophoria changed as a consequence of adaptation during the viewing of the stereoscopic stimuli would experience less visual discomfort than those people whose heterophoria did not adapt. In the experiment an increase of visual discomfort change in the 3D condition in comparison with the 2D condition was found. Also, there were statistically significant changes in heterophoria under 3D conditions as compared with 2D conditions. However, there was appreciable variability in the magnitude of this adaptation among individuals, and no correlation between the amount of heterophoria change and visual discomfort change was observed. In the second experiment the two hypotheses tested were based on the vergence-accommodation mismatch theory, and the visual-vestibular mismatch theory. The vergence-accommodation mismatch theory predicts that a greater mismatch between the stimuli to accommodation and to vergence would produce greater symptoms in visual discomfort when viewing in 3D conditions than when viewing in 2D conditions. An increase of visual discomfort change in the 3D condition in comparison with the 2D condition was indeed found; however the magnitude of visual discomfort reported did not correlate with the mismatch present during the watching of 3D stereoscopic stimuli. The visual-vestibular mismatch theory predicts that viewing a stimulus stereoscopically will produce a greater sense of vection than viewing it in 2D. This will increase the conflict between the signals from the visual and vestibular systems, producing greater VIMS (Visually- Induced Motion Sickness) symptoms. Participants did indeed report an increase in motion sickness symptoms in the 3D condition. Furthermore, participants with closer seating positions reported more VIMS than participants sitting farther away whilst viewing 3D stimuli. This suggests that the amount of visual field stimulated during 3D presentation affects VIMS, and is an important factor in terms of viewing comfort. In the study more younger viewers (21 to 39 years old) than older viewers (40 years old and older) reported a greater change in visual discomfort during the 3D condition than the 2D condition. This suggests that the visual system s response to a stimulus, rather than the stimulus itself, is a reason for discomfort. No influence of gender on viewing comfort was found. In the next experiment participants fusion capability, as measured by their fusional reserves, was examined to determine whether this component has an impact on reported discomfort during the watching of movies in the 3D condition versus the 2D condition. It was hypothesised that participants with limited fusional range would experience more visual discomfort than participants with a wide fusion range. The hypothesis was confirmed but only in the case of convergent and not divergent eye movement. This observation illustrates that participants capability to convergence has a significant impact on visual comfort. The aim of the last experiment was to examine responses of the accommodation system to changes in 3D stimulus position and to determine whether discrepancies in these responses (i.e. accommodation overshoot, accommodation undershoot) could account for visual discomfort experienced during 3D stereoscopic viewing. It was found that accommodation discrepancy was larger for perceived forwards movement than for perceived backwards movement. The discrepancy was slightly higher in the group susceptible to visual discomfort than in the group not susceptible to visual discomfort, but this difference was not statistically significant. When considering the research findings as a whole it was apparent that not all participants experienced more discomfort whilst watching 3D stereoscopic stimuli than whilst watching 2D stimuli. More visual discomfort in the 3D condition than in the 2D condition was reported by 35% of the participants, whilst 24% of the participants reported more headaches and 17% of the participants reported more VIMS. The research indicates that multiple causative factors have an impact on reported symptoms. The analysis of the data suggests that discomfort experienced by people during 3D stereoscopic stimulation may reveal binocular vision problems. This observation suggests that 3D technology could be used as a screening method to diagnose un-treated binocular vision disorder. Additionally, this work shows that 3D stereoscopic technology can be easily adopted to binocular vision measurement. The conclusion of this thesis is that many people do not suffer adverse symptoms when viewing 3D stereoscopic displays, but that if adverse symptoms are present they can be caused either by the conflict in the stimulus, or by the heightened experience of self-motion which leads to Visually-Induced Motion Sickness (VIMS)

Stereoscopic Medical Data Video Quality Issues

Stereoscopic medical videos are recorded, e.g., in stereo endoscopy or during video recording medical/dental operations. This paper examines quality issues in the recorded stereoscopic medical videos, as insufficient quality may induce visual fatigue to doctors. No attention has been paid to stereo quality and ensuing fatigue issues in the scientific literature so far. Two of the most commonly encountered quality issues in stereoscopic data, namely stereoscopic window violations and bent windows, were searched for in stereo endoscopic medical videos. Furthermore, an additional stereo quality issue encountered in dental operation videos, namely excessive disparity, was detected and fixed. The conducted experiments prove the existence of such quality issues in stereoscopic medical data and highlight the need for their detection and correction

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Human factors in the perception of stereoscopic images

Research into stereoscopic displays is largely divided into how stereo 3D content looks, a field concerned with distortion, and how such content feels to the viewer, that is, comfort. However, seldom are these measures presented simultaneously. Both comfortable displays with unacceptable 3D and uncomfortable displays with great 3D are undesirable. These two scenarios can render conclusions based on research into these measures both moot and impractical. Furthermore, there is a consensus that more disparity correlates directly with greater viewer discomfort. These experiments, and the dissertation thereof, challenge this notion and argue for a more nuanced argument related to acquisition factors such as interaxial distance (IA) and post processing in the form of horizontal image translation (HIT). Indeed, this research seeks to measure tolerance limits for viewing comfort and perceptual distortions across different camera separations. In the experiments, HIT and IA were altered together. Following Banks et al. (2009), our stimuli were simple stereoscopic hinges, and we measured the perceived angle as a function of camera separation. We compared the predictions based on a ray-tracing model with the perceived 3D shape obtained psychophysically. Participants were asked to judge the angles of 250 hinges at different camera separations (IA and HIT remained linked across a 20 to 100mm range, but the angles ranged between 50° and 130°). In turn, comfort data was obtained using a five-point Likert scale for each trial. Stimuli were presented in orthoscopic conditions with screen and observer field of view (FOV) matched at 45°. The 3D hinge and experimental parameters were run across three distinct series of experiments. The first series involved replicating a typical laboratory scenario where screen position was unchanged (Experiment I), the other presenting scenarios representative of real-world applications for a single viewer (Experiments II, III, and IV), and the last presenting real-world applications for multiple viewers (Experiment V). While the laboratory scenario revealed greatest viewer comfort occurred when a virtual hinge was placed on the screen plane, the single-viewer experiment revealed into-the-screen stereo stimuli was judged flatter while out-of-screen content was perceived more veridically. The multi-viewer scenario revealed a marked decline in comfort for off-axis viewing, but no commensurate effect on distortion; importantly, hinge angles were judged as being the same regardless of off-axis viewing for angles of up to 45. More specifically, the main results are as follows. 1) Increased viewing distance enhances viewer comfort for stereoscopic perception. 2) The amount of disparity present was not correlated with comfort. Comfort is not correlated with angular distortion. 3) Distortion is affected by hinge placement on-screen. There is only a significant effect on comfort when the Camera Separation is at 60mm. 4) A perceptual bias between into the depth orientation of the screen stimuli, in to the screen stimuli were judged as flatter than out of the screen stimuli. 5) Perceived distortion not being affected by oblique viewing. Oblique viewing does not affect perceived comfort. In conclusion, the laboratory experiment highlights the limitations of extrapolating a controlled empirical stimulus into a less controlled “real world” environment. The typical usage scenarios consistently reveal no correlation between the amount of screen disparity (parallax) in the stimulus and the comfort rating. The final usage scenario reveals a perceptual constancy in off-axis viewer conditions for angles of up to 45, which, as reported, is not reflected by a typical ray-tracing model. Stereoscopic presentation with non-orthoscopic HIT may give comfortable 3D. However, there is good reason to believe that this 3D is not being perceived veridically. Comfortable 3D is often incorrectly converged due to the differences between distances specified by disparity and monocular cues. This conflict between monocular and stereo cues in the presentation of S3D content leads to loss of veridicality i.e. a perception of flatness. Therefore, correct HIT is recommended as the starting point for creating realistic and comfortable 3D, and this factor is shown by data to be far more important than limiting screen disparity (i.e. parallax). Based on these findings, this study proposes a predictive model of stereoscopic space for 3D content generators who require flexibility in acquisition parameters. This is important as there is no data for viewing conditions where the acquisition parameters are changed

Perceived Depth Control in Stereoscopic Cinematography

Despite the recent explosion of interest in the stereoscopic 3D (S3D) technology, the ultimate prevailing of the S3D medium is still significantly hindered by adverse effects regarding the S3D viewing discomfort. This thesis attempts to improve the S3D viewing experience by investigating perceived depth control methods in stereoscopic cinematography on desktop 3D displays. The main contributions of this work are: (1) A new method was developed to carry out human factors studies on identifying the practical limits of the 3D Comfort Zone on a given 3D display. Our results suggest that it is necessary for cinematographers to identify the specific limits of 3D Comfort Zone on the target 3D display as different 3D systems have different ranges for the 3D Comfort Zone. (2) A new dynamic depth mapping approach was proposed to improve the depth perception in stereoscopic cinematography. The results of a human-based experiment confirmed its advantages in controlling the perceived depth in viewing 3D motion pictures over the existing depth mapping methods. (3) The practicability of employing the Depth of Field (DoF) blur technique in S3D was also investigated. Our results indicate that applying the DoF blur simulation on stereoscopic content may not improve the S3D viewing experience without the real time information about what the viewer is looking at. Finally, a basic guideline for stereoscopic cinematography was introduced to summarise the new findings of this thesis alongside several well-known key factors in 3D cinematography. It is our assumption that this guideline will be of particular interest not only to 3D filmmaking but also to 3D gaming, sports broadcasting, and TV production

Une méthode pour l'évaluation de la qualité des images 3D stéréoscopiques.

Dans le contexte d'un intérêt grandissant pour les systèmes stéréoscopiques, mais sans méthodes reproductible pour estimer leur qualité, notre travail propose une contribution à la meilleure compréhension des mécanismes de perception et de jugement humains relatifs au concept multi-dimensionnel de qualité d'image stéréoscopique. Dans cette optique, notre démarche s'est basée sur un certain nombre d'outils : nous avons proposé un cadre adapté afin de structurer le processus d'analyse de la qualité des images stéréoscopiques, nous avons implémenté dans notre laboratoire un système expérimental afin de conduire plusieurs tests, nous avons crée trois bases de données d'images stéréoscopiques contenant des configurations précises et enfin nous avons conduit plusieurs expériences basées sur ces collections d'images. La grande quantité d'information obtenue par l'intermédiaire de ces expérimentations a été utilisée afin de construire un premier modèle mathématique permettant d'expliquer la perception globale de la qualité de la stéréoscopie en fonction des paramètres physiques des images étudiée.In a context of ever-growing interest in stereoscopic systems, but where no standardized algorithmic methods of stereoscopic quality assessment exist, our work stands as a step forward in the understanding of the human perception and judgment mechanisms related to the multidimensional concept of stereoscopic image quality. We used a series of tools in order to perform in-depth investigations in this direction: we proposed an adapted framework to structure the process of stereoscopic quality assessment, we implemented a stereoscopic system in our laboratory for performing various tests, we created three stereoscopic datasets with precise structures, and we performed several experimental studies using these datasets. The numerous experimental data obtained were used in order to propose a first mathematical framework for explaining the overall percept of stereoscopic quality in function of the physical parameters of the stereoscopic images under study.SAVOIE-SCD - Bib.électronique (730659901) / SudocGRENOBLE1/INP-Bib.électronique (384210012) / SudocGRENOBLE2/3-Bib.électronique (384219901) / SudocSudocFranceF