Deep Eyes: Binocular Depth-from-Focus on Focal Stack Pairs

Human visual system relies on both binocular stereo cues and monocular focusness cues to gain effective 3D perception. In computer vision, the two problems are traditionally solved in separate tracks. In this paper, we present a unified learning-based technique that simultaneously uses both types of cues for depth inference. Specifically, we use a pair of focal stacks as input to emulate human perception. We first construct a comprehensive focal stack training dataset synthesized by depth-guided light field rendering. We then construct three individual networks: a Focus-Net to extract depth from a single focal stack, a EDoF-Net to obtain the extended depth of field (EDoF) image from the focal stack, and a Stereo-Net to conduct stereo matching. We show how to integrate them into a unified BDfF-Net to obtain high-quality depth maps. Comprehensive experiments show that our approach outperforms the state-of-the-art in both accuracy and speed and effectively emulates human vision systems

Depth estimation based on thick oriented edges in images

International audienceThe field of contact free 3D measure is particularly open since current measuring devices use lines or points. In this domain, cameras are the preferential sensors to develop global 3D measurement system. Many methods have been developed to obtain the 3D coordinates of objects using images and all exploit the variations of acquisition parameters. Acquisition parameters of the system or of the luminous environment, controlled or not are the essential information to establish a relationship between the image and the real scene. This article deals with a spatial approach to depth estimation by analysis of edges in images. A depth from defocus method is explained and the physical process is described. Theoretical developments are made to apply it on thin or thick edges and for height orientations of the edge profile. Some results on images are presented to illustrate the efficiency and the influence of the conditions of use

A helmet mounted display to adapt the telerobotic environment to human vision

A Helmet Mounted Display system has been developed. It provides the capability to display stereo images with the viewpoint tied to subjects' head orientation. The type of display might be useful in a telerobotic environment provided the correct operating parameters are known. The effects of update frequency were tested using a 3D tracking task. The effects of blur were tested using both tracking and pick-and-place tasks. For both, researchers found that operator performance can be degraded if the correct parameters are not used. Researchers are also using the display to explore the use of head movements as part of gaze as subjects search their visual field for target objects

Influence of mathematic models used on the quality of estimation of the depth in images

International audienceThe problem of mathematical problem chosen for the precision of depth perception in a shape from shading method is studied

Effects of monocular blur on clinical measurements of stereopsis and binocular contrast sensitivity

Visual conditions such as anisometropia, monovision and monocular undercorrection affect the combination of visual input from both eyes. This study investigated the effects of monocular blur, in binocularly normal participants, on stereoacuity and binocular contrast sensitivity. Fifteen young adults (age range between 19 and 23 years old) with normal visual acuity and binocular vision participated in this study. Stereopsis was measured using the TNO test with a series of positive spherical lenses placed before the dominant eye. The procedure was repeated using the Titmus Stereotest on five participants as a control experiment. Monocular and binocular contrast sensitivities were also measured using the Pelli-Robson Contrast Sensitivity Chart. Blur was induced monocularly with a series of positive spherical lenses placed before the dominant eye and binocular contrast sensitivity was re-measured. Stereopsis scores decreased significantly when monocular blur was imposed. Across blur levels, absolute stereopsis scores measured with TNO test were worse than those measured with Titmus stereotest (all p < 0.05). However, the ratio of scores obtained without blur and under monocular blur appeared to be similar for both tests. Stereopsis without blur was between 6.82× to 8× better than that obtained with the highest level of imposed monocular blur. Binocular contrast sensitivity score decreased significantly with increasing level of monocular blur (p < 0.01). Binocular contrast sensitivity score without blur was 1.62× better than that obtained under binocular viewing with highest level of imposed blur. Stereopsis tests are more sensitive than measurements of binocular contrast sensitivity as an indicator of interocular acuity discrepancies which could occur in anisometropic or monovision patients. However, the choice of stereopsis test is crucial, as the TNO test appears to be more sensitive to monocular blur than the Titmus stereostest

Coded aperture imaging

This thesis studies the coded aperture camera, a device consisting of a conventional camera with a modified aperture mask, that enables the recovery of both depth map and all-in-focus image from a single 2D input image. Key contributions of this work are the modeling of the statistics of natural images and the design of efficient blur identification methods in a Bayesian framework. Two cases are distinguished: 1) when the aperture can be decomposed in a small set of identical holes, and 2) when the aperture has a more general configuration. In the first case, the formulation of the problem incorporates priors about the statistical variation of the texture to avoid ambiguities in the solution. This allows to bypass the recovery of the sharp image and concentrate only on estimating depth. In the second case, the depth reconstruction is addressed via convolutions with a bank of linear filters. Key advantages over competing methods are the higher numerical stability and the ability to deal with large blur. The all-in-focus image can then be recovered by using a deconvolution step with the estimated depth map. Furthermore, for the purpose of depth estimation alone, the proposed algorithm does not require information about the mask in use. The comparison with existing algorithms in the literature shows that the proposed methods achieve state-of-the-art performance. This solution is also extended for the first time to images affected by both defocus and motion blur and, finally, to video sequences with moving and deformable objects

Apparent sharpness of 3D video when one eye's view is more blurry.

When the images presented to each eye differ in sharpness, the fused percept remains relatively sharp. Here, we measure this effect by showing stereoscopic videos that have been blurred for one eye, or both eyes, and psychophysically determining when they appear equally sharp. For a range of blur magnitudes, the fused percept always appeared significantly sharper than the blurrier view. From these data, we investigate to what extent discarding high spatial frequencies from just one eye's view reduces the bandwidth necessary to transmit perceptually sharp 3D content. We conclude that relatively high-resolution video transmission has the most potential benefit from this method

Correcciones para la presbicia : implicaciones ópticas, perceptuales y adaptativas

Tesis de la Universidad Complutense de Madrid, Facultad de Óptica y Optometría, leída el 18-05-2016Presbyopia is the physiological inability of the crystalline lens to accommodate for objects at near distance. While accommodative lenses are the ideal solutions for presbyopia, current optical solutions rely on providing an acceptable quality of vision at near and far distances. Optimization of the optical solutions rely on better understanding of how the visual system copes with the visual quality produced by the various optical solutions. The aim of this thesis is to study optical, visual and perceptual performance of different presbyopic corrections such as alternating vision, monovision and simultaneous vision, and to study the effect of adaptation on perceptual performances. Methods: We measured and corrected ocular aberrations using custom developed adaptive optics setup, used images blurred by real aberrations of different orientation and/or magnitude and measured the internal code for blur in eyes with long term differences in blur magnitude or orientation using a classification-image like technique. We later used numerically convolved images of different far/near energy and different near additions to study the short term adaptation to pure simultaneous vision using single stimulus detection and scoring tasks...La presbicia es la incapacidad del cristalino para enfocar objetos cercanos. Mientras que las lentes acomodativas son una buena solución para la presbicia, las soluciones más actuales se basan en una corrección aceptable de la visión cercana y lejana simultáneamente. La optimización de estas soluciones pasa por comprender cómo reacciona el sistema a las diferentes correcciones ópticas. El objetivo de esta tesis es el estudio óptico, visual y perceptual de diferentes correcciones a la presbicia como la visión alternante, la mono visión y la visión simultánea, y el estudio del efecto dela adaptación desde el punto de vista perceptual. MétodosSe han medido y corregido las aberraciones oculares mediante un sistema de óptica adaptativa de construcción propia y se han usado imágenes desenfocadas con aberraciones reales con diferentes magnitudes y/u orientaciones para medir el código interno de emborronamiento en los ojos para los diferentes desenfoques y orientaciones mediante métodos de clasificación de imágenes. Posteriormente se han usado imágenes convolucionadas numéricamente con diferentes proporciones en las energías del enfoque cercano o lejano y con diferentes adiciones para estudiar laadaptación a corto plazo en la visión simultánea pura a través de la detección y valoración de estímulos individuales...Fac. de Óptica y OptometríaTRUEunpu

Augmented reality fonts with enhanced out-of-focus text legibility

In augmented reality, information is often distributed between real and virtual contexts, and often appears at different distances from the viewer. This raises the issues of (1) context switching, when attention is switched between real and virtual contexts, (2) focal distance switching, when the eye accommodates to see information in sharp focus at a new distance, and (3) transient focal blur, when information is seen out of focus, during the time interval of focal distance switching. This dissertation research has quantified the impact of context switching, focal distance switching, and transient focal blur on human performance and eye fatigue in both monocular and binocular viewing conditions. Further, this research has developed a novel font that when seen out-of-focus looks sharper than standard fonts. This SharpView font promises to mitigate the effect of transient focal blur. Developing this font has required (1) mathematically modeling out-of-focus blur with Zernike polynomials, which model focal deficiencies of human vision, (2) developing a focus correction algorithm based on total variation optimization, which corrects out-of-focus blur, and (3) developing a novel algorithm for measuring font sharpness. Finally, this research has validated these fonts through simulation and optical camera-based measurement. This validation has shown that, when seen out of focus, SharpView fonts are as much as 40 to 50% sharper than standard fonts. This promises to improve font legibility in many applications of augmented reality