25 research outputs found
Improving visual quality of view transitions in automultiscopic displays
Automultiscopic screens present different images depending on the viewing direction. This enables glasses-free 3D and provides motion parallax effect. However, due to the limited angular resolution of such displays, they suffer from hot-spotting, i. e., image quality is highly affected by the viewing position. In this paper, we analyze light fields produced by lenticular and parallax-barrier displays, and show that, unlike in real world, the light fields produced by such screens have a repetitive structure. This induces visual artifacts in the form of view discontinuities, depth reversals, and excessive disparities when viewing position is not optimal. Although the problem has been always considered as inherent to the technology, we demonstrate that light fields reproduced on automultiscopic displays have enough degrees of freedom to improve the visual quality. We propose a new technique that modifies light fields using global and local shears followed by stitching to improve their continuity when displayed on a screen. We show that this enhances visual quality significantly, which is demonstrated in a series of user experiments with an automultiscopic display as well as lenticular prints.National Science Foundation (U.S.) (IIS-1111415)National Science Foundation (U.S.) (IIS-1116296)Quanta Computer (Firm)National Basic Research Program of China (973 Program) (Project 2011CB302205)National Natural Science Foundation (China) (Project 61272226/61120106007)National High-Tech R&D (863) Plan of China (Project 2013AA013903)Beijing Higher Institution Engineering Research Center (Research Grant
GazeStereo3D: seamless disparity manipulations
Producing a high quality stereoscopic impression on current displays is a challenging task. The content has to be carefully prepared in order to maintain visual comfort, which typically affects the quality of depth reproduction. In this work, we show that this problem can be significantly alleviated when the eye fixation regions can be roughly estimated. We propose a new method for stereoscopic depth adjustment that utilizes eye tracking or other gaze prediction information. The key idea that distinguishes our approach from the previous work is to apply gradual depth adjustments at the eye fixation stage, so that they remain unnoticeable. To this end, we measure the limits imposed on the speed of disparity changes in various depth adjustment scenarios, and formulate a new model that can guide such seamless stereoscopic content processing. Based on this model, we propose a real-time controller that applies local manipulations to stereoscopic content to find the optimum between depth reproduction and visual comfort. We show that the controller is mostly immune to the limitations of low-cost eye tracking solutions. We also demonstrate benefits of our model in off-line applications, such as stereoscopic movie production, where skillful directors can reliably guide and predict viewers' attention or where attended image regions are identified during eye tracking sessions. We validate both our model and the controller in a series of user experiments. They show significant improvements in depth perception without sacrificing the visual quality when our techniques are applied
Light field image processing: an overview
Light field imaging has emerged as a technology allowing to capture richer visual information from our world. As opposed to traditional photography, which captures a 2D projection of the light in the scene integrating the angular domain, light fields collect radiance from rays in all directions, demultiplexing the angular information lost in conventional photography. On the one hand, this higher dimensional representation of visual data offers powerful capabilities for scene understanding, and substantially improves the performance of traditional computer vision problems such as depth sensing, post-capture refocusing, segmentation, video stabilization, material classification, etc. On the other hand, the high-dimensionality of light fields also brings up new challenges in terms of data capture, data compression, content editing, and display. Taking these two elements together, research in light field image processing has become increasingly popular in the computer vision, computer graphics, and signal processing communities. In this paper, we present a comprehensive overview and discussion of research in this field over the past 20 years. We focus on all aspects of light field image processing, including basic light field representation and theory, acquisition, super-resolution, depth estimation, compression, editing, processing algorithms for light field display, and computer vision applications of light field data
Neural View-Interpolation for Sparse Light Field Video
We suggest representing light field (LF) videos as "one-off" neural networks (NN), i.e., a learned mapping from view-plus-time coordinates to high-resolution color values, trained on sparse views. Initially, this sounds like a bad idea for three main reasons: First, a NN LF will likely have less quality than a same-sized pixel basis representation. Second, only few training data, e.g., 9 exemplars per frame are available for sparse LF videos. Third, there is no generalization across LFs, but across view and time instead. Consequently, a network needs to be trained for each LF video. Surprisingly, these problems can turn into substantial advantages: Other than the linear pixel basis, a NN has to come up with a compact, non-linear i.e., more intelligent, explanation of color, conditioned on the sparse view and time coordinates. As observed for many NN however, this representation now is interpolatable: if the image output for sparse view coordinates is plausible, it is for all intermediate, continuous coordinates as well. Our specific network architecture involves a differentiable occlusion-aware warping step, which leads to a compact set of trainable parameters and consequently fast learning and fast execution
Depth, shading, and stylization in stereoscopic cinematography
Due to the constantly increasing focus of the entertainment industry on stereoscopic imaging, techniques and tools that enable precise control over the depth impression and help to overcome limitations of the current stereoscopic hardware are gaining in importance. In this dissertation, we address selected problems encountered during stereoscopic content production, with a particular focus on stereoscopic cinema. First, we consider abrupt changes of depth, such as those induced by cuts in films. We derive a model predicting the time the visual system needs to adapt to such changes and propose how to employ this model for film cut optimization. Second, we tackle the issue of discrepancies between the two views of a stereoscopic image due to view-dependent shading of glossy materials. The suggested solution eliminates discomfort caused by non-matching specular highlights while preserving the perception of gloss. Last, we deal with the problem of filmgrainmanagement in stereoscopic productions and propose a new method for film grain application that reconciles visual comfort with the idea of medium-scene separation.Aufgrund der stĂ€ndig steigenden Beachtung der stereoskopische Abbildung durch die Unterhaltungsindustrie, gewinnen Techniken und Werkzeuge an Bedeutung, die eine prĂ€zise Steuerung der Tiefenwahrnehmung ermöglichen und EinschrĂ€nkungen der gegenwĂ€rtigen stereoskopischen GerĂ€te ĂŒberwinden. In dieser Dissertation adressieren wir ausgewĂ€hlte Probleme, die wĂ€hrend der Erzeugung von stereoskopischen Inhalten auftreten, mit besonderem Schwerpunkt auf der stereoskopischen Kinematographie. Zuerst betrachten wir abrupte TiefenĂ€nderungen, wie sie durch Filmschnitte hervergerufen werden. Wir leiten ein Modell her, das die Zeit vorhersagt, die fĂŒr das menschliche Sehsystem notwendig ist, um sich an solche Ănderungen der Tiefe zu adaptieren, und schlagen vor wie dieses Modell fĂŒr Schnittoptimierung angewendet werden kann. Danach gehen wir das Problem der Unstimmigkeiten zwischen den zwei Ansichten eines stereoskopischen Bildes, infolge der sichtabhĂ€ngigen Schattierung von glĂ€nzenden Materialien, an. Die vorgeschlagene Lösung eliminiert das visuelle Unbehagen, welches von nicht zusammenpassenden Glanzlichtern verursacht wird, indessen bewahrt sie die Glanzwahrnehmung. Zuletzt behandeln wir das Problem des Filmkornsmanagements in stereoskopischen Produktionen und schlagen eine neue Methode fĂŒr das HinzufĂŒgen vom Filmkorn vor, die die visuelle Behaglichkeit mit der Idee der Medium-Szenen-Trennung in Einklang bringt
Widening the view angle of auto-multiscopic display, denoising low brightness light field data and 3D reconstruction with delicate details
This doctoral thesis will present the results of my work into widening the viewing angle
of the auto-multiscopic display, denoising light filed data the enhancement of captured
light filed data captured in low light circumstance, and the attempts on reconstructing
the subject surface with delicate details from microscopy image sets.
The automultiscopic displays carefully control the distribution of emitted light over
space, direction (angle) and time so that even a static image displayed can encode
parallax across viewing directions (light field). This allows simultaneous observation by
multiple viewers, each perceiving 3D from their own (correct) perspective. Currently,
the illusion can only be effectively maintained over a narrow range of viewing angles.
We propose and analyze a simple solution to widen the range of viewing angles for
automultiscopic displays that use parallax barriers. We insert a refractive medium, with
a high refractive index, between the display and parallax barriers. The inserted medium
warps the exitant lightfield in a way that increases the potential viewing angle. We
analyze the consequences of this warp and build a prototype with a 93% increase in
the effective viewing angle. Additionally, we developed an integral images synthesis
method that can address the refraction introduced by the inserted medium efficiently
without the use of ray tracing.
Capturing light field image with a short exposure time is preferable for eliminating
the motion blur but it also leads to low brightness in a low light environment, which
results in a low signal noise ratio. Most light field denoising methods apply regular 2D
image denoising method to the sub-aperture images of a 4D light field directly, but it
is not suitable for focused light field data whose sub-aperture image resolution is too
low to be applied regular denoising methods. Therefore, we propose a deep learning
denoising method based on micro lens images of focused light field to denoise the depth
map and the original micro lens image set simultaneously, and achieved high quality
total focused images from the low focused light field data.
In areas like digital museum, remote researching, 3D reconstruction with delicate
details of subjects is desired and technology like 3D reconstruction based on macro
photography has been used successfully for various purposes. We intend to push it
further by using microscope rather than macro lens, which is supposed to be able to
capture the microscopy level details of the subject. We design and implement a scanning
method which is able to capture microscopy image set from a curve surface based on
robotic arm, and the 3D reconstruction method suitable for the microscopy image set
Methods for reducing visual discomfort in stereoscopic 3D: A review
This work was supported by the EPSRC Grant EP/M01469X/1, âGeometric Evaluation of Stereoscopic Videoâ