640 research outputs found
Occlusion Handling using Semantic Segmentation and Visibility-Based Rendering for Mixed Reality
Real-time occlusion handling is a major problem in outdoor mixed reality
system because it requires great computational cost mainly due to the
complexity of the scene. Using only segmentation, it is difficult to accurately
render a virtual object occluded by complex objects such as trees, bushes etc.
In this paper, we propose a novel occlusion handling method for real-time,
outdoor, and omni-directional mixed reality system using only the information
from a monocular image sequence. We first present a semantic segmentation
scheme for predicting the amount of visibility for different type of objects in
the scene. We also simultaneously calculate a foreground probability map using
depth estimation derived from optical flow. Finally, we combine the
segmentation result and the probability map to render the computer generated
object and the real scene using a visibility-based rendering method. Our
results show great improvement in handling occlusions compared to existing
blending based methods
Shadow Generation in Augmented Reality: A Complete Survey
This paper provides an overview of the issues and techniques involved in shadow generation in mixed reality environments. Shadow generation techniques in virtual environments are explained briefly. The key factors characterizing the well-known techniques are described in detail and the pros and cons of each technique are discussed. The conceptual perspective, the improvements, and future techniques are also investigated, sum- marized, and analysed in depth. This paper aims to provide researchers with a solid background on the state- of-the-art implementation of shadows in mixed reality. Thus, this could make it easier to choose the most appropriate method to achieve the aims. It is also hoped that this analysis will help researchers find solutions to the problems facing each technique
Static scene illumination estimation from video with applications
We present a system that automatically recovers scene geometry and illumination from a video, providing a basis for various applications. Previous image based illumination estimation methods require either user interaction or external information in the form of a database. We adopt structure-from-motion and multi-view stereo for initial scene reconstruction, and then estimate an environment map represented by spherical harmonics (as these perform better than other bases). We also demonstrate several video editing applications that exploit the recovered geometry and illumination, including object insertion (e.g., for augmented reality), shadow detection, and video relighting
Shadow Generation in Mixed Reality: A Comprehensive Survey
This paper provides an overview of the issues and techniques involved in shadow generation in mixed reality environments. Shadow generation techniques in virtual environments are explained briefly. The key factors characterizing the well-known techniques are described in detail and the pros and cons of each technique are discussed. The conceptual perspective, the improvements, and future techniques are also investigated, summarized, and analysed in depth. This paper aims to provide researchers with a solid background on the state-of-the-art implementation of shadows in mixed reality. Thus, this could make it easier to choose the most appropriate method to achieve the aims. It is also hoped that this analysis will help researchers find solutions to the problems facing each technique
A Dataset of Multi-Illumination Images in the Wild
Collections of images under a single, uncontrolled illumination have enabled
the rapid advancement of core computer vision tasks like classification,
detection, and segmentation. But even with modern learning techniques, many
inverse problems involving lighting and material understanding remain too
severely ill-posed to be solved with single-illumination datasets. To fill this
gap, we introduce a new multi-illumination dataset of more than 1000 real
scenes, each captured under 25 lighting conditions. We demonstrate the richness
of this dataset by training state-of-the-art models for three challenging
applications: single-image illumination estimation, image relighting, and
mixed-illuminant white balance.Comment: ICCV 201
- …