Occlusion Handling using Semantic Segmentation and Visibility-Based Rendering for Mixed Reality

Real-time occlusion handling is a major problem in outdoor mixed reality system because it requires great computational cost mainly due to the complexity of the scene. Using only segmentation, it is difficult to accurately render a virtual object occluded by complex objects such as trees, bushes etc. In this paper, we propose a novel occlusion handling method for real-time, outdoor, and omni-directional mixed reality system using only the information from a monocular image sequence. We first present a semantic segmentation scheme for predicting the amount of visibility for different type of objects in the scene. We also simultaneously calculate a foreground probability map using depth estimation derived from optical flow. Finally, we combine the segmentation result and the probability map to render the computer generated object and the real scene using a visibility-based rendering method. Our results show great improvement in handling occlusions compared to existing blending based methods

CAPT: Category-level Articulation Estimation from a Single Point Cloud Using Transformer

The ability to estimate joint parameters is essential for various applications in robotics and computer vision. In this paper, we propose CAPT: category-level articulation estimation from a point cloud using Transformer. CAPT uses an end-to-end transformer-based architecture for joint parameter and state estimation of articulated objects from a single point cloud. The proposed CAPT methods accurately estimate joint parameters and states for various articulated objects with high precision and robustness. The paper also introduces a motion loss approach, which improves articulation estimation performance by emphasizing the dynamic features of articulated objects. Additionally, the paper presents a double voting strategy to provide the framework with coarse-to-fine parameter estimation. Experimental results on several category datasets demonstrate that our methods outperform existing alternatives for articulation estimation. Our research provides a promising solution for applying Transformer-based architectures in articulated object analysis.Comment: Accepted to ICRA 202

Non-learning Stereo-aided Depth Completion under Mis-projection via Selective Stereo Matching

We propose a non-learning depth completion method for a sparse depth map captured using a light detection and ranging (LiDAR) sensor guided by a pair of stereo images. Generally, conventional stereo-aided depth completion methods have two limiations. (i) They assume the given sparse depth map is accurately aligned to the input image, whereas the alignment is difficult to achieve in practice. (ii) They have limited accuracy in the long range because the depth is estimated by pixel disparity. To solve the abovementioned limitations, we propose selective stereo matching (SSM) that searches the most appropriate depth value for each image pixel from its neighborly projected LiDAR points based on an energy minimization framework. This depth selection approach can handle any type of mis-projection. Moreover, SSM has an advantage in terms of long-range depth accuracy because it directly uses the LiDAR measurement rather than the depth acquired from the stereo. SSM is a discrete process; thus, we apply variational smoothing with binary anisotropic diffusion tensor (B-ADT) to generate a continuous depth map while preserving depth discontinuity across object boundaries. Experimentally, compared with the previous state-of-the-art stereo-aided depth completion, the proposed method reduced the mean absolute error (MAE) of the depth estimation to 0.65 times and demonstrated approximately twice more accurate estimation in the long range. Moreover, under various LiDAR-camera calibration errors, the proposed method reduced the depth estimation MAE to 0.34-0.93 times from previous depth completion methods.Comment: 15 pages, 13 figure

Combined Influences of Gradual Changes in Room Temperature and Light around Dusk and Dawn on Circadian Rhythms of Core Temperature, Urinary 6-Hydroxymelatonin Sulfate and Waking Sensation Just after Rising

The present experiment aimed at knowing how a gradual changes of room temperature (Ta) and light in the evening and early morning could influence circadian rhythms of core temperature (Tcore), skin temperatures, urinary 6-hydroxymelatonin sulfate and waking sensation just after rising in humans. Two kinds of room environment were provided for each participant: 1) Constant room temperature (Ta) of 27 °C over the 24 h and LD-rectangular light change with abrupt decreasing from 3,000 lx to100 lx at 1800,abrupt increasing from 0 lx to 3,000 lx at 0700. 2) Cyclic changes of Ta and with gradual decrease from 3,000 lx to 100 lx onset at 1700 (twilight period about 2 h), with gradual increasing from 0 lx to 3,000 lx onset at 0500 (about 2 h). Main results are summarized as follows: 1) Circadian rhythms of nadir in the core temperature (Tcore) significantly advanced earlier under the influence of gradual changes of Ta and light than no gradual changes of Ta and light. 2) Nocturnal fall of Tcore and morning rise of Tcore were greater and quicker, respectively, under the influence of gradual changes of Ta and light than no gradual changes of Ta and light. 3) Urinary 6-hydroxymelatonin sulfate during nocturnal sleep was significantly greater under the influence of gradual changes of Ta and light. 4) Waking sensation just after rising was significantly better under the influence of gradual changes of Ta and light. We discussed these findings in terms of circadian and thermoregulatory physiology