Deep Eyes: Binocular Depth-from-Focus on Focal Stack Pairs

Human visual system relies on both binocular stereo cues and monocular focusness cues to gain effective 3D perception. In computer vision, the two problems are traditionally solved in separate tracks. In this paper, we present a unified learning-based technique that simultaneously uses both types of cues for depth inference. Specifically, we use a pair of focal stacks as input to emulate human perception. We first construct a comprehensive focal stack training dataset synthesized by depth-guided light field rendering. We then construct three individual networks: a Focus-Net to extract depth from a single focal stack, a EDoF-Net to obtain the extended depth of field (EDoF) image from the focal stack, and a Stereo-Net to conduct stereo matching. We show how to integrate them into a unified BDfF-Net to obtain high-quality depth maps. Comprehensive experiments show that our approach outperforms the state-of-the-art in both accuracy and speed and effectively emulates human vision systems

Velocity-Based LOD Reduction in Virtual Reality: A Psychometric Approach

Virtual Reality headsets enable users to explore the environment by performing self-induced movements. The retinal velocity produced by such motion reduces the visual system's ability to resolve fine detail. We measured the impact of self-induced head rotations on the ability to detect quality changes of a realistic 3D model in an immersive virtual reality environment. We varied the Level-of-Detail (LOD) as a function of rotational head velocity with different degrees of severity. Using a psychophysical method, we asked 17 participants to identify which of the two presented intervals contained the higher quality model under two different maximum velocity conditions. After fitting psychometric functions to data relating the percentage of correct responses to the aggressiveness of LOD manipulations, we identified the threshold severity for which participants could reliably (75\%) detect the lower LOD model. Participants accepted an approximately four-fold LOD reduction even in the low maximum velocity condition without a significant impact on perceived quality, which suggests that there is considerable potential for optimisation when users are moving (increased range of perceptual uncertainty). Moreover, LOD could be degraded significantly more in the maximum head velocity condition, suggesting these effects are indeed speed dependent

A perceptual model of motion quality for rendering with adaptive refresh-rate and resolution

Limited GPU performance budgets and transmission bandwidths mean that real-time rendering often has to compromise on the spatial resolution or temporal resolution (refresh rate). A common practice is to keep either the resolution or the refresh rate constant and dynamically control the other variable. But this strategy is non-optimal when the velocity of displayed content varies. To find the best trade-off between the spatial resolution and refresh rate, we propose a perceptual visual model that predicts the quality of motion given an object velocity and predictability of motion. The model considers two motion artifacts to establish an overall quality score: non-smooth (juddery) motion, and blur. Blur is modeled as a combined effect of eye motion, finite refresh rate and display resolution. To fit the free parameters of the proposed visual model, we measured eye movement for predictable and unpredictable motion, and conducted psychophysical experiments to measure the quality of motion from 50 Hz to 165 Hz. We demonstrate the utility of the model with our on-the-fly motion-adaptive rendering algorithm that adjusts the refresh rate of a G-Sync-capable monitor based on a given rendering budget and observed object motion. Our psychophysical validation experiments demonstrate that the proposed algorithm performs better than constant-refresh-rate solutions, showing that motion-adaptive rendering is an attractive technique for driving variable-refresh-rate displays.</jats:p

Photorealistic physically based render engines: a comparative study

Pérez Roig, F. (2012). Photorealistic physically based render engines: a comparative study. http://hdl.handle.net/10251/14797.Archivo delegad

Disparity map generation based on trapezoidal camera architecture for multiview video

Visual content acquisition is a strategic functional block of any visual system. Despite its wide possibilities, the arrangement of cameras for the acquisition of good quality visual content for use in multi-view video remains a huge challenge. This paper presents the mathematical description of trapezoidal camera architecture and relationships which facilitate the determination of camera position for visual content acquisition in multi-view video, and depth map generation. The strong point of Trapezoidal Camera Architecture is that it allows for adaptive camera topology by which points within the scene, especially the occluded ones can be optically and geometrically viewed from several different viewpoints either on the edge of the trapezoid or inside it. The concept of maximum independent set, trapezoid characteristics, and the fact that the positions of cameras (with the exception of few) differ in their vertical coordinate description could very well be used to address the issue of occlusion which continues to be a major problem in computer vision with regards to the generation of depth map

Real-time impluse-based rigid body simulation and rendering

The purpose of this thesis is to develop and demonstrate a physically based rigid body simulation with a focus on simplifications to achieve real-time performance. This thesis aims to demonstrate that by improving the efficiency with simplified calculations of possible bottlenecks of a real-time rigid body simulation, the accuracy can be improved. A prototype simulation framework is implemented to evaluate the simplifications. Finally, various real-time rendering features are implemented to achieve a realistic look, and also to imitate the game-like environment where real-time rigid body simulations are mostly utilized. A series of demonstration experiments are used to show that our simulator does, in fact, achieve real-time performance, while maintaining satisfactory accuracy. A direct comparison of this prototype with a commercially available simulator verifies that the simplified approach improves the efficiency and does not damage the accuracy under our test conditions. Integration of rendering elements like advanced shading, shadowing, depth of field and motion blur into our real-time framework also enhanced the perception of simulation outcomes

Measuring the Discernability of Virtual Objects in Conventional and Stylized Augmented Reality

In augmented reality, virtual graphical objects are overlaid over the real environment of the observer. Conventional augmented reality systems normally use standard real-time rendering methods for generating the graphical representations of virtual objects. These renderings contain the typical artifacts of computer generated graphics, e.g., aliasing caused by the rasterization process and unrealistic, manually configured illumination models. Due to these artifacts, virtual objects look artifical and can easily be distinguished from the real environment. A different approach to generating augmented reality images is the basis of stylized augmented reality [FBS05c]. Here, similar types of artistic or illustrative stylization are applied to the virtual objects and the camera image of the real enviroment. Therefore, real and virtual image elements look significantly more similar and are less distinguishable from each other. In this paper, we present the results of a psychophysical study on the effectiveness of stylized augmented reality. In this study, a number of participants were asked to decide whether objects shown in images of augmented reality scenes are virtual or real. Conventionally rendered as well as stylized augmented reality images and short video clips were presented to the participants. The correctness of the participants' responses and their reaction times were recorded. The results of our study show that an equalized level of realism is achieved by using stylized augmented reality, i.e., that it is significantly more difficult to distinguish virtual objects from real objects