In-Domain GAN Inversion for Faithful Reconstruction and Editability

Generative Adversarial Networks (GANs) have significantly advanced image synthesis through mapping randomly sampled latent codes to high-fidelity synthesized images. However, applying well-trained GANs to real image editing remains challenging. A common solution is to find an approximate latent code that can adequately recover the input image to edit, which is also known as GAN inversion. To invert a GAN model, prior works typically focus on reconstructing the target image at the pixel level, yet few studies are conducted on whether the inverted result can well support manipulation at the semantic level. This work fills in this gap by proposing in-domain GAN inversion, which consists of a domain-guided encoder and a domain-regularized optimizer, to regularize the inverted code in the native latent space of the pre-trained GAN model. In this way, we manage to sufficiently reuse the knowledge learned by GANs for image reconstruction, facilitating a wide range of editing applications without any retraining. We further make comprehensive analyses on the effects of the encoder structure, the starting inversion point, as well as the inversion parameter space, and observe the trade-off between the reconstruction quality and the editing property. Such a trade-off sheds light on how a GAN model represents an image with various semantics encoded in the learned latent distribution. Code, models, and demo are available at the project page: https://genforce.github.io/idinvert/

Improving GANs with A Dynamic Discriminator

Discriminator plays a vital role in training generative adversarial networks (GANs) via distinguishing real and synthesized samples. While the real data distribution remains the same, the synthesis distribution keeps varying because of the evolving generator, and thus effects a corresponding change to the bi-classification task for the discriminator. We argue that a discriminator with an on-the-fly adjustment on its capacity can better accommodate such a time-varying task. A comprehensive empirical study confirms that the proposed training strategy, termed as DynamicD, improves the synthesis performance without incurring any additional computation cost or training objectives. Two capacity adjusting schemes are developed for training GANs under different data regimes: i) given a sufficient amount of training data, the discriminator benefits from a progressively increased learning capacity, and ii) when the training data is limited, gradually decreasing the layer width mitigates the over-fitting issue of the discriminator. Experiments on both 2D and 3D-aware image synthesis tasks conducted on a range of datasets substantiate the generalizability of our DynamicD as well as its substantial improvement over the baselines. Furthermore, DynamicD is synergistic to other discriminator-improving approaches (including data augmentation, regularizers, and pre-training), and brings continuous performance gain when combined for learning GANs.Comment: To appear in NeurIPS 202

Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator

3D-aware image synthesis aims at learning a generative model that can render photo-realistic 2D images while capturing decent underlying 3D shapes. A popular solution is to adopt the generative adversarial network (GAN) and replace the generator with a 3D renderer, where volume rendering with neural radiance field (NeRF) is commonly used. Despite the advancement of synthesis quality, existing methods fail to obtain moderate 3D shapes. We argue that, considering the two-player game in the formulation of GANs, only making the generator 3D-aware is not enough. In other words, displacing the generative mechanism only offers the capability, but not the guarantee, of producing 3D-aware images, because the supervision of the generator primarily comes from the discriminator. To address this issue, we propose GeoD through learning a geometry-aware discriminator to improve 3D-aware GANs. Concretely, besides differentiating real and fake samples from the 2D image space, the discriminator is additionally asked to derive the geometry information from the inputs, which is then applied as the guidance of the generator. Such a simple yet effective design facilitates learning substantially more accurate 3D shapes. Extensive experiments on various generator architectures and training datasets verify the superiority of GeoD over state-of-the-art alternatives. Moreover, our approach is registered as a general framework such that a more capable discriminator (i.e., with a third task of novel view synthesis beyond domain classification and geometry extraction) can further assist the generator with a better multi-view consistency.Comment: Accepted by NeurIPS 2022. Project page: https://vivianszf.github.io/geo

Temperature-dependent structure and magnetization of YCrO3_3 compound

We have grown a YCrO$_3$ single crystal by the floating-zone method and studied its temperature-dependent crystalline structure and magnetization by X-ray powder diffraction and PPMS DynaCool measurements. All diffraction patterns were well indexed by an orthorhombic structure with space group of $Pbnm$ (No. 62). From 36 to 300 K, no structural phase transition occurs in the pulverized YCrO$_3$ single crystal. The antiferromagnetic phase transition temperature was determined as $T_\textrm{N} =$ 141.58(5) K by the magnetization versus temperature measurements. We found weak ferromagnetic behavior in the magnetic hysteresis loops below $T_\textrm{N}$. Especially, we demonstrated that the antiferromagnetism and weak ferromagnetism appear simutaniously upon cooling. The lattice parameters ($a$, $b$, $c$, and $V$) deviate downward from the Gr$\ddot{\textrm{u}}$neisen law, displaying an anisotropic magnetostriction effect. We extracted temperature variation of the local distortion parameter $\Delta$. Compared to the $\Delta$ value of Cr ions, Y, O1, and O2 ions show one order of magnitude larger $\Delta$ values indicative of much stronger local lattice distortions. Moreover, the calculated bond valence states of Y and O2 ions have obvious subduction charges.Comment: 8 pages, 9 figures, submitted to Chinese Physics

PolyIE: A Dataset of Information Extraction from Polymer Material Scientific Literature

Scientific information extraction (SciIE), which aims to automatically extract information from scientific literature, is becoming more important than ever. However, there are no existing SciIE datasets for polymer materials, which is an important class of materials used ubiquitously in our daily lives. To bridge this gap, we introduce POLYIE, a new SciIE dataset for polymer materials. POLYIE is curated from 146 full-length polymer scholarly articles, which are annotated with different named entities (i.e., materials, properties, values, conditions) as well as their N-ary relations by domain experts. POLYIE presents several unique challenges due to diverse lexical formats of entities, ambiguity between entities, and variable-length relations. We evaluate state-of-the-art named entity extraction and relation extraction models on POLYIE, analyze their strengths and weaknesses, and highlight some difficult cases for these models. To the best of our knowledge, POLYIE is the first SciIE benchmark for polymer materials, and we hope it will lead to more research efforts from the community on this challenging task. Our code and data are available on: https://github.com/jerry3027/PolyIE.Comment: Work in progres

Temperature-dependent structure of an intermetallic ErPd2_2Si2_2 single crystal: A combined synchrotron and in-house X-ray diffraction study

We have grown intermetallic ErPd$_2$Si$_2$ single crystals employing laser-diodes with the floating-zone method. The temperature-dependent crystallography was determined using synchrotron and in-house X-ray powder diffraction measurements from 20 to 500 K. The diffraction patterns fit well with the tetragonal $I$4/$mmm$ space group (No. 139) with two chemical formulas within one unit cell. Our synchrotron X-ray powder diffraction study shows that the refined lattice constants are $a$ = 4.10320(2) {\AA}, $c$ = 9.88393(5) {\AA} at 298 K and $a$ = 4.11737(2) {\AA}, $c$ = 9.88143(5) {\AA} at 500 K, resulting in the unit-cell volume $V$ = 166.408(1) {\AA}$^3$ (298 K) and 167.517(2) {\AA}$^3$ (500 K). In the whole studied temperature range, we did not find any structural phase transition. Upon cooling, the lattice constants a and c are shortened and elongated, respectively.Comment: 5 Figures, 4 Table