What Does Stable Diffusion Know about the 3D Scene?

Recent advances in generative models like Stable Diffusion enable the generation of highly photo-realistic images. Our objective in this paper is to probe the diffusion network to determine to what extent it 'understands' different properties of the 3D scene depicted in an image. To this end, we make the following contributions: (i) We introduce a protocol to evaluate whether a network models a number of physical 'properties' of the 3D scene by probing for explicit features that represent these properties. The probes are applied on datasets of real images with annotations for the property. (ii) We apply this protocol to properties covering scene geometry, scene material, support relations, lighting, and view dependent measures. (iii) We find that Stable Diffusion is good at a number of properties including scene geometry, support relations, shadows and depth, but less performant for occlusion. (iv) We also apply the probes to other models trained at large-scale, including DINO and CLIP, and find their performance inferior to that of Stable Diffusion

Score-PA: Score-based 3D Part Assembly

Autonomous 3D part assembly is a challenging task in the areas of robotics and 3D computer vision. This task aims to assemble individual components into a complete shape without relying on predefined instructions. In this paper, we formulate this task from a novel generative perspective, introducing the Score-based 3D Part Assembly framework (Score-PA) for 3D part assembly. Knowing that score-based methods are typically time-consuming during the inference stage. To address this issue, we introduce a novel algorithm called the Fast Predictor-Corrector Sampler (FPC) that accelerates the sampling process within the framework. We employ various metrics to assess assembly quality and diversity, and our evaluation results demonstrate that our algorithm outperforms existing state-of-the-art approaches. We release our code at https://github.com/J-F-Cheng/Score-PA_Score-based-3D-Part-Assembly.Comment: BMVC 202

Learning Environment-Aware Affordance for 3D Articulated Object Manipulation under Occlusions

Perceiving and manipulating 3D articulated objects in diverse environments is essential for home-assistant robots. Recent studies have shown that point-level affordance provides actionable priors for downstream manipulation tasks. However, existing works primarily focus on single-object scenarios with homogeneous agents, overlooking the realistic constraints imposed by the environment and the agent's morphology, e.g., occlusions and physical limitations. In this paper, we propose an environment-aware affordance framework that incorporates both object-level actionable priors and environment constraints. Unlike object-centric affordance approaches, learning environment-aware affordance faces the challenge of combinatorial explosion due to the complexity of various occlusions, characterized by their quantities, geometries, positions and poses. To address this and enhance data efficiency, we introduce a novel contrastive affordance learning framework capable of training on scenes containing a single occluder and generalizing to scenes with complex occluder combinations. Experiments demonstrate the effectiveness of our proposed approach in learning affordance considering environment constraints. Project page at https://chengkaiacademycity.github.io/EnvAwareAfford/Comment: In 37th Conference on Neural Information Processing Systems (NeurIPS 2023). Website at https://chengkaiacademycity.github.io/EnvAwareAfford

A cost-effective method to fabricate VO2 (M) nanoparticles and films with excellent thermochromic properties

In this paper, high crystallinity and pure phase VO2 (M) powder is synthesized by a novel and facile method. Aiding by additional manual grinding and etching process, 22 nm high-quality VO2 (M) nanoparticles can be obtained. The structure and properties of the VO2 (M) particles were characterized by X-ray diffraction analysis, transmission electron microscopy, differential scanning calorimetry and UV-vis-NIR spectrophotometer. After mixing VO2 (M) nanoparticles with transparent polymer, thin films prepared by grinded VO2 nanoparticles show excellent thermochromic properties. The solar modulation ability is up to 12.4% with luminous transmittance of 62.7%. Moreover, The haze of films prepared by grinded VO2 (M) nanoparticles is down to 1.9%, which is far less than that of films prepared by original VO2 (Haze = 8.5%) and etched VO2 particles (Haze = 4.6%). Dramatical improvement of thermochromic property and definition indicate that it is a promising method to prepare large-scale VO2 nanoparticles and cost-effective smart window. (C) 2015 Elsevier B.V. All rights reserved

Anisotropic vanadium dioxide sculptured thin films with superior thermochromic properties

VO2 (M) STF through reduction of V2O5 STF was prepared. The results illustrate that V2O5 STF can be successfully obtained by oblique angle thermal evaporation technique. After annealing at 550 degrees C/3 min, the V2O5 STF deposited at 856 can be easily transformed into VO2 STF with slanted columnar structure and superior thermochromic properties. After deposition SiO2 antireflective layer, T-lum of VO2 STF is enhanced 26% and Delta T-sol increases 60% compared with that of normal VO2 thin films. Due to the anisotropic microstructure of VO2 STF, angular selectivity transmission of VO2 STF is observed and the solar modulation ability is further improved from 7.2% to 8.7% when light is along columnar direction. Moreover, the phase transition temperature of VO2 STF can be depressed into 54.5 degrees C without doping. Considering the oblique incidence of sunlight on windows, VO2 STF is more beneficial for practical application as smart windows compared with normal homogenous VO2 thin films