In-Domain GAN Inversion for Faithful Reconstruction and Editability

Generative Adversarial Networks (GANs) have significantly advanced image synthesis through mapping randomly sampled latent codes to high-fidelity synthesized images. However, applying well-trained GANs to real image editing remains challenging. A common solution is to find an approximate latent code that can adequately recover the input image to edit, which is also known as GAN inversion. To invert a GAN model, prior works typically focus on reconstructing the target image at the pixel level, yet few studies are conducted on whether the inverted result can well support manipulation at the semantic level. This work fills in this gap by proposing in-domain GAN inversion, which consists of a domain-guided encoder and a domain-regularized optimizer, to regularize the inverted code in the native latent space of the pre-trained GAN model. In this way, we manage to sufficiently reuse the knowledge learned by GANs for image reconstruction, facilitating a wide range of editing applications without any retraining. We further make comprehensive analyses on the effects of the encoder structure, the starting inversion point, as well as the inversion parameter space, and observe the trade-off between the reconstruction quality and the editing property. Such a trade-off sheds light on how a GAN model represents an image with various semantics encoded in the learned latent distribution. Code, models, and demo are available at the project page: https://genforce.github.io/idinvert/

LinkGAN: Linking GAN Latents to Pixels for Controllable Image Synthesis

This work presents an easy-to-use regularizer for GAN training, which helps explicitly link some axes of the latent space to a set of pixels in the synthesized image. Establishing such a connection facilitates a more convenient local control of GAN generation, where users can alter the image content only within a spatial area simply by partially resampling the latent code. Experimental results confirm four appealing properties of our regularizer, which we call LinkGAN. (1) The latent-pixel linkage is applicable to either a fixed region (\textit{i.e.}, same for all instances) or a particular semantic category (i.e., varying across instances), like the sky. (2) Two or multiple regions can be independently linked to different latent axes, which further supports joint control. (3) Our regularizer can improve the spatial controllability of both 2D and 3D-aware GAN models, barely sacrificing the synthesis performance. (4) The models trained with our regularizer are compatible with GAN inversion techniques and maintain editability on real images

Reciprocal facilitation between large herbivores and ants in a semi-arid grassland

While positive interactions have been well documented in plant and sessile benthic marine communities, their role in structuring mobile animal communities and underlying mechanisms has been less explored. Using field removal experiments, we demonstrated that a large vertebrate herbivore (cattle; Bos taurus) and a much smaller invertebrate (ants; Lasius spp.), the two dominant animal taxa in a semi-arid grassland in Northeast China, facilitate each other. Cattle grazing led to higher ant mound abundance compared with ungrazed sites, while the presence of ant mounds increased the foraging of cattle during the peak of the growing season. Mechanistically, these reciprocal positive effects were driven by habitat amelioration and resource (food) enhancement by cattle and ants (respectively). Cattle facilitated ants, probably by decreasing plant litter accumulation by herbivory and trampling, allowing more light to reach the soil surface leading to microclimatic conditions that favour ants. Ants facilitated cattle probably by increasing soil nutrients via bioturbation, increasing food (plant) biomass and quality (nitrogen content) for cattle. Our study demonstrates reciprocal facilitative interactions between two animal species from phylogenetically very distant taxa. Such reciprocal positive interactions may be more common in animal communities than so far assumed, and they should receive more attention to improve our understanding of species coexistence and animal community assembly.</p

Effects of grazing on C : N:P stoichiometry attenuate from soils to plants and insect herbivores in a semi-arid grassland

Understanding the processing of limiting nutrients among organisms is an important goal of community ecology. Less known is how human disturbances may alter the stoichiometric patterns among organisms from different trophic levels within communities. Here, we investigated how livestock grazing affects the C:N:P ecological stoichiometry of soils, plants (Leymus chinensis), and grasshoppers (Euchorthippus spp.) in a semi-arid grassland in northeastern China. We found that grazing significantly enhanced soil available N and leaf N content of the dominant L. chinensis grass by 15% and 20%, respectively. Grazing also reduced (soluble) C:N of L. chinensis leaves by 22%. However, grazing did not affect total C, N, or P contents nor their ratios in Euchorthippus grasshoppers. Our results reveal that the effects of grazing disturbances on elemental composition attenuated from lower to higher trophic levels. These findings support the theory that organisms from higher trophic levels have relatively stronger stoichiometric homeostasis compared to those from lower trophic levels. Moreover, grasshopper abundance dropped by 66% in the grazed areas, and they reduced the feeding time on their host L. chinensis grass by 43%, presumably to limit the intake of excess nitrogen from host plants. The energetic costs associated with the maintenance of elemental homeostasis likely reduced grasshopper individual performance and population abundance in the grazed areas. A comprehensive investigation of stoichiometric properties of organisms across trophic levels may enable a better understanding of the nature of species interactions, and facilitate predictions of the consequences of future environmental changes for a community organization.Peer reviewe

Neural Dependencies Emerging from Learning Massive Categories

This work presents two astonishing findings on neural networks learned for large-scale image classification. 1) Given a well-trained model, the logits predicted for some category can be directly obtained by linearly combining the predictions of a few other categories, which we call \textbf{neural dependency}. 2) Neural dependencies exist not only within a single model, but even between two independently learned models, regardless of their architectures. Towards a theoretical analysis of such phenomena, we demonstrate that identifying neural dependencies is equivalent to solving the Covariance Lasso (CovLasso) regression problem proposed in this paper. Through investigating the properties of the problem solution, we confirm that neural dependency is guaranteed by a redundant logit covariance matrix, which condition is easily met given massive categories, and that neural dependency is highly sparse, implying that one category correlates to only a few others. We further empirically show the potential of neural dependencies in understanding internal data correlations, generalizing models to unseen categories, and improving model robustness with a dependency-derived regularizer. Code for this work will be made publicly available