Methods for reducing visual discomfort in stereoscopic 3D: A review

This work was supported by the EPSRC Grant EP/M01469X/1, “Geometric Evaluation of Stereoscopic Video”

Crosstalk in stereoscopic displays

Crosstalk is an important image quality attribute of stereoscopic 3D displays. The research presented in this thesis examines the presence, mechanisms, simulation, and reduction of crosstalk for a selection of stereoscopic display technologies. High levels of crosstalk degrade the perceived quality of stereoscopic displays hence it is important to minimise crosstalk. This thesis provides new insights which are critical to a detailed understanding of crosstalk and consequently to the development of effective crosstalk reduction techniques

Adaptive micro-optical phase modulators based on liquid crystal technology

Mención Internacional en el título de doctorThis thesis began with the project “Advanced Devices of Liquid Crystal and Electroluminescent Organic Diodes. Hybrid Applications for 3D Vision” funded by the Spanish government. The goal of this project was the development of optical devices to achieve 3D vision in portable devices without glasses or external elements. In order to achieve the goals of this project, solutions based on liquid crystal are considered. Specifically, adaptive micro-optical phase modulators based on liquid crystal technology are researched in depth. The gradient of the refractive index varies spatially the phase delay experienced by an impinging wavefront of a light beam. By using this effect, any refractive optical element may be reproduced with the proper voltage gradient applied to the sample. This is the main operating principle of the micro-optical phase modulators proposed in this thesis. As original contribution of this thesis, a novel algorithm to solve the position of a nematic liquid crystal molecular director is proposed. Once the liquid crystal is completely characterized, the developing of a specific model to know the electro-optic response of the micro-optical phase modulators is also relevant. Another original contribution is a novel equivalent electric circuit for modeling liquid crystal microlenses. An interesting feature of the model is that it provides an analytical solution for microlenses with modal and hole-patterned electrode schemes, by using a simple software tool. The required driving scheme (modal or hole-patterned) can be predicted. These theories have been validated by experimental results. For more complex devices, the equations are solved by Finite Element Method. A new manufacturing protocol is proposed to make the first set of modal microlens arrays. As a first step simple devices (monopixel cells) are fabricated in order to do a complete study of the liquid crystal electro-optical behavior. The characterization of the liquid crystal electro-optical parameters is determinant in order to design more complex devices. Refractive index and permittivity are the most important features considered. These parameters have been characterized to validate the proposed theoretical modelling of the liquid crystal molecular position. These devices have required special fabrication processes as well as a special characterization set-up especially in terms of size resolution or arrangement complexity. A custom micropositioner is developed and control software is programmed in relation to these tasks. The software automates the characterization process giving directly measured results of: phase modulation, focal distance, thickness or aberrations. These results have made it possible to validate experimentally the proposed electrical modeling for micro-optical devices. Demonstration of the viability of the liquid crystal lenticular technology has been carried out for an autostereoscopic application. This scheme provides the observer with the option of changing between horizontal and vertical views through his portable autostereoscopic display. Finally, last research contributions of this work of thesis have taken advantage of the deep knowledge of the electro-optical properties of lenticular devices for autostereoscopic applications, to guide the design of refined micro-optical phase modulators. Adaptive axicons and optical vortices are specially emphasized because their relevance from both, the scientific and technological point of view.Esta tesis se inició con el proyecto de investigación “Dispositivos avanzados de cristal líquido y diodos orgánicos electroluminiscentes. Aplicaciones híbridas para visión 3D”, financiado por el gobierno español. El objetivo de este proyecto consistía en el desarrollo de dispositivos ópticos para lograr visión 3D en dispositivos portátiles sin necesidad de gafas o elementos externos. Con el fin de alcanzar los objetivos de este proyecto, se consideran soluciones basadas en cristal líquido. En concreto, moduladores adaptativos de fase micro-ópticos basados en tecnología de cristal líquido. El gradiente del índice de refracción varía espacialmente el retardo de fase experimentado por un frente de onda incidente. Mediante el uso de este efecto, cualquier elemento óptico refractivo puede ser reproducido mediante un gradiente de tensión adecuado aplicado a la muestra. Este es el principio de funcionamiento de los moduladores de fase micro-ópticos propuestos en esta tesis. Como aportación original de esta tesis, se propone un nuevo algoritmo para resolver el director molecular de un cristal líquido nemático. Una vez que el cristal líquido está completamente caracterizado, es necesario el desarrollo de un modelo específico para saber la respuesta electro-óptica de los moduladores de fase micro-ópticos. Otra contribución original, consiste en un circuito eléctrico equivalente para el modelado de microlentes de cristal líquido. Una característica interesante del modelo es que proporciona una solución analítica para microlentes con esquemas de electrodos modales y “hole patterned”. Se puede predecir la topología necesaria en función de los parámetros de construcción. Estas teorías han sido validadas por resultados experimentales. Para los dispositivos más complejos, las ecuaciones se resuelven por el método de elementos finitos. Se propone un nuevo protocolo de fabricación para hacer microlentes modales. Como primer paso se fabrican dispositivos sencillos (células monopixel) con el fin de hacer un estudio completo del comportamiento electro-óptico del cristal líquido. La caracterización de los parámetros electro-ópticos de cristal líquido es determinante para diseñar dispositivos más complejos. El índice de refracción y la permitividad son las características más importantes. Estos parámetros se han caracterizado para validar el modelo teórico de la posición molecular de cristal líquido. Estos dispositivos han requerido procesos de fabricación complejos, así como montajes de caracterización determinados. Se ha desarrollado un microposicionador y un software de control. El software automatiza el proceso de caracterización dando resultados de: modulación de fase, distancia focal, grosor o aberraciones. Estos resultados han permitido validar experimentalmente el modelado eléctrico propuesto para dispositivos micro-ópticos. La demostración de la viabilidad de la tecnología propuesta se ha llevado a cabo mediante un dispositivo autoestereoscópico. Este dispositivo ofrece al observador la opción de cambiar entre vistas horizontal y vertical a través de su pantalla autoestereoscópica portátil. Finalmente, los últimos aportes de investigación de este trabajo de tesis se han aprovechado del profundo conocimiento de las propiedades electro-ópticas de los dispositivos lenticulares para aplicaciones autoestereoscópicas. Se pueden destacar los axicones adaptativos y vórtices ópticos por su relevancia tanto desde el punto de vista científico como tecnológico.Este trabajo ha sido desarrollado en el marco de los proyectos TEC2009-13991-C02-01 financiado por el Ministerio de Ciencia e Innovación y FACTOTEM2 S2009/ESP-1781 financiado por la Comunidad de Madrid.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Ignacio Raúl Matías Maestro.- Secretario: Antonia Isabel Pérez Garcilópez.- Vocal: Dimitrios C. Zografopoulo

Remote Visual Observation of Real Places Through Virtual Reality Headsets

Virtual Reality has always represented a fascinating yet powerful opportunity that has attracted studies and technology developments, especially since the latest release on the market of powerful high-resolution and wide field-of-view VR headsets. While the great potential of such VR systems is common and accepted knowledge, issues remain related to how to design systems and setups capable of fully exploiting the latest hardware advances. The aim of the proposed research is to study and understand how to increase the perceived level of realism and sense of presence when remotely observing real places through VR headset displays. Hence, to produce a set of guidelines that give directions to system designers about how to optimize the display-camera setup to enhance performance, focusing on remote visual observation of real places. The outcome of this investigation represents unique knowledge that is believed to be very beneficial for better VR headset designs towards improved remote observation systems. To achieve the proposed goal, this thesis presents a thorough investigation of existing literature and previous researches, which is carried out systematically to identify the most important factors ruling realism, depth perception, comfort, and sense of presence in VR headset observation. Once identified, these factors are further discussed and assessed through a series of experiments and usability studies, based on a predefined set of research questions. More specifically, the role of familiarity with the observed place, the role of the environment characteristics shown to the viewer, and the role of the display used for the remote observation of the virtual environment are further investigated. To gain more insights, two usability studies are proposed with the aim of defining guidelines and best practices. The main outcomes from the two studies demonstrate that test users can experience an enhanced realistic observation when natural features, higher resolution displays, natural illumination, and high image contrast are used in Mobile VR. In terms of comfort, simple scene layouts and relaxing environments are considered ideal to reduce visual fatigue and eye strain. Furthermore, sense of presence increases when observed environments induce strong emotions, and depth perception improves in VR when several monocular cues such as lights and shadows are combined with binocular depth cues. Based on these results, this investigation then presents a focused evaluation on the outcomes and introduces an innovative eye-adapted High Dynamic Range (HDR) approach, which the author believes to be of great improvement in the context of remote observation when combined with eye-tracked VR headsets. Within this purpose, a third user study is proposed to compare static HDR and eye-adapted HDR observation in VR, to assess that the latter can improve realism, depth perception, sense of presence, and in certain cases even comfort. Results from this last study confirmed the author expectations, proving that eye-adapted HDR and eye tracking should be used to achieve best visual performances for remote observation in modern VR systems

Une méthode pour l'évaluation de la qualité des images 3D stéréoscopiques.

Dans le contexte d'un intérêt grandissant pour les systèmes stéréoscopiques, mais sans méthodes reproductible pour estimer leur qualité, notre travail propose une contribution à la meilleure compréhension des mécanismes de perception et de jugement humains relatifs au concept multi-dimensionnel de qualité d'image stéréoscopique. Dans cette optique, notre démarche s'est basée sur un certain nombre d'outils : nous avons proposé un cadre adapté afin de structurer le processus d'analyse de la qualité des images stéréoscopiques, nous avons implémenté dans notre laboratoire un système expérimental afin de conduire plusieurs tests, nous avons crée trois bases de données d'images stéréoscopiques contenant des configurations précises et enfin nous avons conduit plusieurs expériences basées sur ces collections d'images. La grande quantité d'information obtenue par l'intermédiaire de ces expérimentations a été utilisée afin de construire un premier modèle mathématique permettant d'expliquer la perception globale de la qualité de la stéréoscopie en fonction des paramètres physiques des images étudiée.In a context of ever-growing interest in stereoscopic systems, but where no standardized algorithmic methods of stereoscopic quality assessment exist, our work stands as a step forward in the understanding of the human perception and judgment mechanisms related to the multidimensional concept of stereoscopic image quality. We used a series of tools in order to perform in-depth investigations in this direction: we proposed an adapted framework to structure the process of stereoscopic quality assessment, we implemented a stereoscopic system in our laboratory for performing various tests, we created three stereoscopic datasets with precise structures, and we performed several experimental studies using these datasets. The numerous experimental data obtained were used in order to propose a first mathematical framework for explaining the overall percept of stereoscopic quality in function of the physical parameters of the stereoscopic images under study.SAVOIE-SCD - Bib.électronique (730659901) / SudocGRENOBLE1/INP-Bib.électronique (384210012) / SudocGRENOBLE2/3-Bib.électronique (384219901) / SudocSudocFranceF

Methods for Light Field Display Profiling and Scalable Super-Multiview Video Coding

Light field 3D displays reproduce the light field of real or synthetic scenes, as observed by multiple viewers, without the necessity of wearing 3D glasses. Reproducing light fields is a technically challenging task in terms of optical setup, content creation, distributed rendering, among others; however, the impressive visual quality of hologramlike scenes, in full color, with real-time frame rates, and over a very wide field of view justifies the complexity involved. Seeing objects popping far out from the screen plane without glasses impresses even those viewers who have experienced other 3D displays before.Content for these displays can either be synthetic or real. The creation of synthetic (rendered) content is relatively well understood and used in practice. Depending on the technique used, rendering has its own complexities, quite similar to the complexity of rendering techniques for 2D displays. While rendering can be used in many use-cases, the holy grail of all 3D display technologies is to become the future 3DTVs, ending up in each living room and showing realistic 3D content without glasses. Capturing, transmitting, and rendering live scenes as light fields is extremely challenging, and it is necessary if we are about to experience light field 3D television showing real people and natural scenes, or realistic 3D video conferencing with real eye-contact.In order to provide the required realism, light field displays aim to provide a wide field of view (up to 180°), while reproducing up to ~80 MPixels nowadays. Building gigapixel light field displays is realistic in the next few years. Likewise, capturing live light fields involves using many synchronized cameras that cover the same display wide field of view and provide the same high pixel count. Therefore, light field capture and content creation has to be well optimized with respect to the targeted display technologies. Two major challenges in this process are addressed in this dissertation.The first challenge is how to characterize the display in terms of its capabilities to create light fields, that is how to profile the display in question. In clearer terms this boils down to finding the equivalent spatial resolution, which is similar to the screen resolution of 2D displays, and angular resolution, which describes the smallest angle, the color of which the display can control individually. Light field is formalized as 4D approximation of the plenoptic function in terms of geometrical optics through spatiallylocalized and angularly-directed light rays in the so-called ray space. Plenoptic Sampling Theory provides the required conditions to sample and reconstruct light fields. Subsequently, light field displays can be characterized in the Fourier domain by the effective display bandwidth they support. In the thesis, a methodology for displayspecific light field analysis is proposed. It regards the display as a signal processing channel and analyses it as such in spectral domain. As a result, one is able to derive the display throughput (i.e. the display bandwidth) and, subsequently, the optimal camera configuration to efficiently capture and filter light fields before displaying them.While the geometrical topology of optical light sources in projection-based light field displays can be used to theoretically derive display bandwidth, and its spatial and angular resolution, in many cases this topology is not available to the user. Furthermore, there are many implementation details which cause the display to deviate from its theoretical model. In such cases, profiling light field displays in terms of spatial and angular resolution has to be done by measurements. Measurement methods that involve the display showing specific test patterns, which are then captured by a single static or moving camera, are proposed in the thesis. Determining the effective spatial and angular resolution of a light field display is then based on an automated analysis of the captured images, as they are reproduced by the display, in the frequency domain. The analysis reveals the empirical limits of the display in terms of pass-band both in the spatial and angular dimension. Furthermore, the spatial resolution measurements are validated by subjective tests confirming that the results are in line with the smallest features human observers can perceive on the same display. The resolution values obtained can be used to design the optimal capture setup for the display in question.The second challenge is related with the massive number of views and pixels captured that have to be transmitted to the display. It clearly requires effective and efficient compression techniques to fit in the bandwidth available, as an uncompressed representation of such a super-multiview video could easily consume ~20 gigabits per second with today’s displays. Due to the high number of light rays to be captured, transmitted and rendered, distributed systems are necessary for both capturing and rendering the light field. During the first attempts to implement real-time light field capturing, transmission and rendering using a brute force approach, limitations became apparent. Still, due to the best possible image quality achievable with dense multi-camera light field capturing and light ray interpolation, this approach was chosen as the basis of further work, despite the massive amount of bandwidth needed. Decompression of all camera images in all rendering nodes, however, is prohibitively time consuming and is not scalable. After analyzing the light field interpolation process and the data-access patterns typical in a distributed light field rendering system, an approach to reduce the amount of data required in the rendering nodes has been proposed. This approach, on the other hand, requires rectangular parts (typically vertical bars in case of a Horizontal Parallax Only light field display) of the captured images to be available in the rendering nodes, which might be exploited to reduce the time spent with decompression of video streams. However, partial decoding is not readily supported by common image / video codecs. In the thesis, approaches aimed at achieving partial decoding are proposed for H.264, HEVC, JPEG and JPEG2000 and the results are compared.The results of the thesis on display profiling facilitate the design of optimal camera setups for capturing scenes to be reproduced on 3D light field displays. The developed super-multiview content encoding also facilitates light field rendering in real-time. This makes live light field transmission and real-time teleconferencing possible in a scalable way, using any number of cameras, and at the spatial and angular resolution the display actually needs for achieving a compelling visual experience

Quality of Experience in Immersive Video Technologies

Over the last decades, several technological revolutions have impacted the television industry, such as the shifts from black & white to color and from standard to high-definition. Nevertheless, further considerable improvements can still be achieved to provide a better multimedia experience, for example with ultra-high-definition, high dynamic range & wide color gamut, or 3D. These so-called immersive technologies aim at providing better, more realistic, and emotionally stronger experiences. To measure quality of experience (QoE), subjective evaluation is the ultimate means since it relies on a pool of human subjects. However, reliable and meaningful results can only be obtained if experiments are properly designed and conducted following a strict methodology. In this thesis, we build a rigorous framework for subjective evaluation of new types of image and video content. We propose different procedures and analysis tools for measuring QoE in immersive technologies. As immersive technologies capture more information than conventional technologies, they have the ability to provide more details, enhanced depth perception, as well as better color, contrast, and brightness. To measure the impact of immersive technologies on the viewersâ QoE, we apply the proposed framework for designing experiments and analyzing collected subjectsâ ratings. We also analyze eye movements to study human visual attention during immersive content playback. Since immersive content carries more information than conventional content, efficient compression algorithms are needed for storage and transmission using existing infrastructures. To determine the required bandwidth for high-quality transmission of immersive content, we use the proposed framework to conduct meticulous evaluations of recent image and video codecs in the context of immersive technologies. Subjective evaluation is time consuming, expensive, and is not always feasible. Consequently, researchers have developed objective metrics to automatically predict quality. To measure the performance of objective metrics in assessing immersive content quality, we perform several in-depth benchmarks of state-of-the-art and commonly used objective metrics. For this aim, we use ground truth quality scores, which are collected under our subjective evaluation framework. To improve QoE, we propose different systems for stereoscopic and autostereoscopic 3D displays in particular. The proposed systems can help reducing the artifacts generated at the visualization stage, which impact picture quality, depth quality, and visual comfort. To demonstrate the effectiveness of these systems, we use the proposed framework to measure viewersâ preference between these systems and standard 2D & 3D modes. In summary, this thesis tackles the problems of measuring, predicting, and improving QoE in immersive technologies. To address these problems, we build a rigorous framework and we apply it through several in-depth investigations. We put essential concepts of multimedia QoE under this framework. These concepts not only are of fundamental nature, but also have shown their impact in very practical applications. In particular, the JPEG, MPEG, and VCEG standardization bodies have adopted these concepts to select technologies that were proposed for standardization and to validate the resulting standards in terms of compression efficiency

The quality of experience of emerging display technologies

As new display technologies emerge and become part of everyday life, the understanding of the visual experience they provide becomes more relevant. The cognition of perception is the most vital component of visual experience; however, it is not the only cognition that contributes to the complex overall experience of the end-user. Expectations can create significant cognitive bias that may even override what the user genuinely perceives. Even if a visualization technology is somewhat novel, expectations can be fuelled by prior experiences gained from using similar displays and, more importantly, even a single word or an acronym may induce serious preconceptions, especially if such word suggests excellence in quality. In this interdisciplinary Ph.D. thesis, the effect of minimal, one-word labels on the Quality of Experience (QoE) is investigated in a series of subjective tests. In the studies carried out on an ultra-high-definition (UHD) display, UHD video contents were directly compared to their HD counterparts, with and without labels explicitly informing the test participants about the resolution of each stimulus. The experiments on High Dynamic Range (HDR) visualization addressed the effect of the word “premium” on the quality aspects of HDR video, and also how this may affect the perceived duration of stalling events. In order to support the findings, additional tests were carried out comparing the stalling detection thresholds of HDR video with conventional Low Dynamic Range (LDR) video. The third emerging technology addressed by this thesis is light field visualization. Due to its novel nature and the lack of comprehensive, exhaustive research on the QoE of light field displays and content parameters at the time of this thesis, instead of investigating the labeling effect, four phases of subjective studies were performed on light field QoE. The first phases started with fundamental research, and the experiments progressed towards the concept and evaluation of the dynamic adaptive streaming of light field video, introduced in the final phase

NASA Tech Briefs, February 1993

Topics include: Communication Technology; Electronic Components and Circuits; Electronic Systems; Physical Sciences; Materials; Computer Programs; Mechanics; Machinery; Fabrication Technology; Mathematics and Information Sciences; Life Sciences