304 research outputs found
Recommended from our members
Defocus Video Matting
Video matting is the process of pulling a high-quality alpha matte and foreground from a video sequence. Current techniques require either a known background (e.g., a blue screen) or extensive user interaction (e.g., to specify known foreground and background elements). The matting problem is generally under-constrained, since not enough information has been collected at capture time. We propose a novel, fully autonomous method for pulling a matte using multiple synchronized video streams that share a point of view but differ in their plane of focus. The solution is obtained by directly minimizing the error in filter-based image formation equations, which are over-constrained by our rich data stream. Our system solves the fully dynamic video matting problem without user assistance: both the foreground and background may be high frequency and have dynamic content, the foreground may resemble the background, and the scene is lit by natural (as opposed to polarized or collimated) illumination.Engineering and Applied Science
Light field image processing: an overview
Light field imaging has emerged as a technology allowing to capture richer visual information from our world. As opposed to traditional photography, which captures a 2D projection of the light in the scene integrating the angular domain, light fields collect radiance from rays in all directions, demultiplexing the angular information lost in conventional photography. On the one hand, this higher dimensional representation of visual data offers powerful capabilities for scene understanding, and substantially improves the performance of traditional computer vision problems such as depth sensing, post-capture refocusing, segmentation, video stabilization, material classification, etc. On the other hand, the high-dimensionality of light fields also brings up new challenges in terms of data capture, data compression, content editing, and display. Taking these two elements together, research in light field image processing has become increasingly popular in the computer vision, computer graphics, and signal processing communities. In this paper, we present a comprehensive overview and discussion of research in this field over the past 20 years. We focus on all aspects of light field image processing, including basic light field representation and theory, acquisition, super-resolution, depth estimation, compression, editing, processing algorithms for light field display, and computer vision applications of light field data
Highlighted depth-of-field photography: Shining light on focus
We present a photographic method to enhance intensity differences between
objects at varying distances from the focal plane. By combining a unique
capture procedure with simple image processing techniques, the detected
brightness of an object is decreased proportional to its degree of defocus. A
camera-projector system casts distinct grid patterns onto a scene to generate
a spatial distribution of point reflections. These point reflections relay a
relative measure of defocus that is utilized in postprocessing to generate a
highlighted DOF photograph. Trade-offs between three different projectorprocessing
pairs are analyzed, and a model is developed to help describe a
new intensity-dependent depth of field that is controlled by the pattern of
illumination. Results are presented for a primary single snapshot design as
well as a scanning method and a comparison method. As an application,
automatic matting results are presented.Alfred P. Sloan Foundatio
DeOccNet: Learning to See Through Foreground Occlusions in Light Fields
Background objects occluded in some views of a light field (LF) camera can be
seen by other views. Consequently, occluded surfaces are possible to be
reconstructed from LF images. In this paper, we handle the LF de-occlusion
(LF-DeOcc) problem using a deep encoder-decoder network (namely, DeOccNet). In
our method, sub-aperture images (SAIs) are first given to the encoder to
incorporate both spatial and angular information. The encoded representations
are then used by the decoder to render an occlusionfree center-view SAI. To the
best of our knowledge, DeOccNet is the first deep learning-based LF-DeOcc
method. To handle the insufficiency of training data, we propose an LF
synthesis approach to embed selected occlusion masks into existing LF images.
Besides, several synthetic and realworld LFs are developed for performance
evaluation. Experimental results show that, after training on the generated
data, our DeOccNet can effectively remove foreground occlusions and achieves
superior performance as compared to other state-of-the-art methods. Source
codes are available at: https://github.com/YingqianWang/DeOccNet.Comment: 10 pages, 8 figure
Joint Blind Motion Deblurring and Depth Estimation of Light Field
Removing camera motion blur from a single light field is a challenging task
since it is highly ill-posed inverse problem. The problem becomes even worse
when blur kernel varies spatially due to scene depth variation and high-order
camera motion. In this paper, we propose a novel algorithm to estimate all blur
model variables jointly, including latent sub-aperture image, camera motion,
and scene depth from the blurred 4D light field. Exploiting multi-view nature
of a light field relieves the inverse property of the optimization by utilizing
strong depth cues and multi-view blur observation. The proposed joint
estimation achieves high quality light field deblurring and depth estimation
simultaneously under arbitrary 6-DOF camera motion and unconstrained scene
depth. Intensive experiment on real and synthetic blurred light field confirms
that the proposed algorithm outperforms the state-of-the-art light field
deblurring and depth estimation methods
Recommended from our members
Depth and Deblurring from a Spectrally-varying Depth-of-Field
We propose modifying the aperture of a conventional color camera so that the effective aperture size for one color channel is smaller than that for the other two. This produces an image where different color channels have different depths-of-field, and from this we can computationally recover scene depth, reconstruct an all-focus image and achieve synthetic re-focusing, all from a single shot. These capabilities are enabled by a spatio-spectral image model that encodes the statistical relationship between gradient profiles across color channels. This approach substantially improves depth accuracy over alternative single-shot coded-aperture designs, and since it avoids introducing additional spatial distortions and is light efficient, it allows high-quality deblurring and lower exposure times. We demonstrate these benefits with comparisons on synthetic data, as well as results on images captured with a prototype lens.Engineering and Applied Science
- …