ZeroMesh: Zero-shot Single-view 3D Mesh Reconstruction

Single-view 3D object reconstruction is a fundamental and challenging computer vision task that aims at recovering 3D shapes from single-view RGB images. Most existing deep learning based reconstruction methods are trained and evaluated on the same categories, and they cannot work well when handling objects from novel categories that are not seen during training. Focusing on this issue, this paper tackles Zero-shot Single-view 3D Mesh Reconstruction, to study the model generalization on unseen categories and encourage models to reconstruct objects literally. Specifically, we propose an end-to-end two-stage network, ZeroMesh, to break the category boundaries in reconstruction. Firstly, we factorize the complicated image-to-mesh mapping into two simpler mappings, i.e., image-to-point mapping and point-to-mesh mapping, while the latter is mainly a geometric problem and less dependent on object categories. Secondly, we devise a local feature sampling strategy in 2D and 3D feature spaces to capture the local geometry shared across objects to enhance model generalization. Thirdly, apart from the traditional point-to-point supervision, we introduce a multi-view silhouette loss to supervise the surface generation process, which provides additional regularization and further relieves the overfitting problem. The experimental results show that our method significantly outperforms the existing works on the ShapeNet and Pix3D under different scenarios and various metrics, especially for novel objects

Neural Vector Fields: Generalizing Distance Vector Fields by Codebooks and Zero-Curl Regularization

Recent neural networks based surface reconstruction can be roughly divided into two categories, one warping templates explicitly and the other representing 3D surfaces implicitly. To enjoy the advantages of both, we propose a novel 3D representation, Neural Vector Fields (NVF), which adopts the explicit learning process to manipulate meshes and implicit unsigned distance function (UDF) representation to break the barriers in resolution and topology. This is achieved by directly predicting the displacements from surface queries and modeling shapes as Vector Fields, rather than relying on network differentiation to obtain direction fields as most existing UDF-based methods do. In this way, our approach is capable of encoding both the distance and the direction fields so that the calculation of direction fields is differentiation-free, circumventing the non-trivial surface extraction step. Furthermore, building upon NVFs, we propose to incorporate two types of shape codebooks, \ie, NVFs (Lite or Ultra), to promote cross-category reconstruction through encoding cross-object priors. Moreover, we propose a new regularization based on analyzing the zero-curl property of NVFs, and implement this through the fully differentiable framework of our NVF (ultra). We evaluate both NVFs on four surface reconstruction scenarios, including watertight vs non-watertight shapes, category-agnostic reconstruction vs category-unseen reconstruction, category-specific, and cross-domain reconstruction

Neural Vector Fields: Implicit Representation by Explicit Learning

Deep neural networks (DNNs) are widely applied for nowadays 3D surface reconstruction tasks and such methods can be further divided into two categories, which respectively warp templates explicitly by moving vertices or represent 3D surfaces implicitly as signed or unsigned distance functions. Taking advantage of both advanced explicit learning process and powerful representation ability of implicit functions, we propose a novel 3D representation method, Neural Vector Fields (NVF). It not only adopts the explicit learning process to manipulate meshes directly, but also leverages the implicit representation of unsigned distance functions (UDFs) to break the barriers in resolution and topology. Specifically, our method first predicts the displacements from queries towards the surface and models the shapes as \textit{Vector Fields}. Rather than relying on network differentiation to obtain direction fields as most existing UDF-based methods, the produced vector fields encode the distance and direction fields both and mitigate the ambiguity at "ridge" points, such that the calculation of direction fields is straightforward and differentiation-free. The differentiation-free characteristic enables us to further learn a shape codebook via Vector Quantization, which encodes the cross-object priors, accelerates the training procedure, and boosts model generalization on cross-category reconstruction. The extensive experiments on surface reconstruction benchmarks indicate that our method outperforms those state-of-the-art methods in different evaluation scenarios including watertight vs non-watertight shapes, category-specific vs category-agnostic reconstruction, category-unseen reconstruction, and cross-domain reconstruction. Our code is released at https://github.com/Wi-sc/NVF.Comment: Accepted by CVPR2023. Video: https://www.youtube.com/watch?v=GMXKoJfmHr

Unsupervised Discovery of Interpretable Directions in h-space of Pre-trained Diffusion Models

We propose the first unsupervised and learning-based method to identify interpretable directions in h-space of pre-trained diffusion models. Our method is derived from an existing technique that operates on the GAN latent space. Specifically, we employ a shift control module that works on h-space of pre-trained diffusion models to manipulate a sample into a shifted version of itself, followed by a reconstructor to reproduce both the type and the strength of the manipulation. By jointly optimizing them, the model will spontaneously discover disentangled and interpretable directions. To prevent the discovery of meaningless and destructive directions, we employ a discriminator to maintain the fidelity of shifted sample. Due to the iterative generative process of diffusion models, our training requires a substantial amount of GPU VRAM to store numerous intermediate tensors for back-propagating gradient. To address this issue, we propose a general VRAM-efficient training algorithm based on gradient checkpointing technique to back-propagate any gradient through the whole generative process, with acceptable occupancy of VRAM and sacrifice of training efficiency. Compared with existing related works on diffusion models, our method inherently identifies global and scalable directions, without necessitating any other complicated procedures. Extensive experiments on various datasets demonstrate the effectiveness of our method

The hidden spin-momentum locking and topological defects in unpolarized light fields

Electromagnetic waves characterized by intensity, phase, and polarization degrees of freedom are widely applied in data storage, encryption, and communications. However, these properties can be substantially affected by phase disorders and disturbances, whereas high-dimensional degrees of freedom including momentum and angular momentum of electromagnetic waves can offer new insights into their features and phenomena, for example topological characteristics and structures that are robust to these disturbances. Here, we discover and demonstrate theoretically and experimentally spin-momentum locking and topological defects in unpolarized light. The coherent spin is locked to the kinetic momentum except for a small coupling spin term, due to the simultaneous presence of transverse magnetic and electric components in unpolarized light. To cancel the coupling term, we employ a metal film acting as a polarizer to form some skyrmion-like spin textures at the metal/air interface. Using an in-house scanning optical microscopic system to image the out-of-plane spin density of the focused unpolarized vortex light, we obtained experimental results that coincide well with our theoretical predictions. The theory and technique promote the applications of topological defects in optical data storage, encryption, and decryption, and communications.Comment: 9 pages, 3 figures, 47 reference

Slag Blended Cement Paste Carbonation under Different CO(2)Concentrations: Controls on Mineralogy and Morphology of Products

To investigate the effect of different CO(2)concentrations on the carbonation results of slag blended cement pastes, carbonation experiments under natural (0.03% CO2) and accelerated conditions (3, 20, and 100% CO2) were investigated with various microscopic testing methods, including X-ray diffraction (XRD),Si-29 magic angle spinning nuclear magnetic resonance (Si-29 MAS NMR) and scanning electron microscopy (SEM). The XRD results indicated that the major polymorphs of CaCO(3)after carbonation were calcite and vaterite. The values of the calcite/(aragonite + vaterite) (c/(a + v)) ratios were almost the same in all carbonation conditions. Additionally, NMR results showed that the decalcification degree of C-S-H gel exposed to 0.03% CO(2)was less than that exposed to accelerated carbonation; under accelerated conditions, it increased from 83.1 to 84.2% when the CO(2)concentration improved from 3% to 100%. In SEM observations, the microstructures after accelerated carbonation were denser than those under natural carbonation but showed minor differences between different CO(2)concentrations. In conclusion, for cement pastes blended with 20% slag, a higher CO(2)concentration (above 3%) led to products different from those produced under natural carbonation. A further increase in CO(2)concentration showed limited variation in generated carbonation products

Mode-matching metasurfaces: coherent reconstruction and multiplexing of surface waves

Metasurfaces are promising two-dimensional metamaterials that are engineered to provide unique properties or functionalities absent in naturally occurring homogeneous surfaces. Here, we report a type of metasurface for tailored reconstruction of surface plasmon waves from light. The design is generic in a way that one can selectively generate different surface plasmon waves through simple variation of the wavelength or the polarization state of incident light. The ultra-thin metasurface demonstrated in this paper provides a versatile interface between the conventional free-space optics and a two-dimensional platform such as surface plasmonics.Comment: 7 figures, supplementary information at the end of the documen

Carbonation of the synthetic calcium silicate hydrate (C-S-H) under different concentrations of CO2: Chemical phases analysis and kinetics

In this study, the chemical phases analysis and the kinetics of synthetic calcium silicate hydrate (C-S-H) under differentCO2concentrations (natural (0.03%), 3%, 10%, 20%, 50%, 100%) were investigated. For this aim, the scanning electron microscope (SEM) and transmission electron microscope (TEM) were employed for microstructure characterisation. The 29Si magic angle spinning nuclear magnetic resonance (29Si MAS NMR), X-ray diffraction (XRD) and thermogravimetric analysis (TGA) coupled with mass spectrometer (MS) were used for characterising the chemical phases before and after carbonation. From the NMR results, it was found that C-S-H would be partly decalcified under the natural condition but completely under the accelerated conditions. Two equations related to the carbonation kinetics under natural and accelerated conditions were proposed respectively. The compositions in decalcified C-S-H were not affected by the CO2 concentration. The XRD analysis showed that vaterite, aragonite and calcite were coexistent after carbonation, which would be transformed to aragonite and calcite with further carbonation. The preferential formation of the allotropic calcium carbonate was not impacted by the concentration of CO2 either. Based on the TGA-MS test, the stoichiometric formula of synthetic C-S-H was determined with CaO\ue2\u27™SiO2\ue2\u27™0.87H2O or C\ue2 S\ue2 H0.87. In addition, a carbonation kinetics model was proposed to learn the carbonation kinetics of C-S-H carbonated in different CO2 concentrations. The experimental data fitted well with the model. The carbonation kinetics between 3% and 20% CO2 are similar, but different from that under 50% and 100% CO