AgentAvatar: Disentangling Planning, Driving and Rendering for Photorealistic Avatar Agents

In this study, our goal is to create interactive avatar agents that can autonomously plan and animate nuanced facial movements realistically, from both visual and behavioral perspectives. Given high-level inputs about the environment and agent profile, our framework harnesses LLMs to produce a series of detailed text descriptions of the avatar agents' facial motions. These descriptions are then processed by our task-agnostic driving engine into motion token sequences, which are subsequently converted into continuous motion embeddings that are further consumed by our standalone neural-based renderer to generate the final photorealistic avatar animations. These streamlined processes allow our framework to adapt to a variety of non-verbal avatar interactions, both monadic and dyadic. Our extensive study, which includes experiments on both newly compiled and existing datasets featuring two types of agents -- one capable of monadic interaction with the environment, and the other designed for dyadic conversation -- validates the effectiveness and versatility of our approach. To our knowledge, we advanced a leap step by combining LLMs and neural rendering for generalized non-verbal prediction and photo-realistic rendering of avatar agents.Comment: Project page: https://dorniwang.github.io/AgentAvatar_project

Learning One-Shot 4D Head Avatar Synthesis using Synthetic Data

Existing one-shot 4D head synthesis methods usually learn from monocular videos with the aid of 3DMM reconstruction, yet the latter is evenly challenging which restricts them from reasonable 4D head synthesis. We present a method to learn one-shot 4D head synthesis via large-scale synthetic data. The key is to first learn a part-wise 4D generative model from monocular images via adversarial learning, to synthesize multi-view images of diverse identities and full motions as training data; then leverage a transformer-based animatable triplane reconstructor to learn 4D head reconstruction using the synthetic data. A novel learning strategy is enforced to enhance the generalizability to real images by disentangling the learning process of 3D reconstruction and reenactment. Experiments demonstrate our superiority over the prior art.Comment: Project page: https://yudeng.github.io/Portrait4D

PICTURE: PhotorealistIC virtual Try-on from UnconstRained dEsigns

In this paper, we propose a novel virtual try-on from unconstrained designs (ucVTON) task to enable photorealistic synthesis of personalized composite clothing on input human images. Unlike prior arts constrained by specific input types, our method allows flexible specification of style (text or image) and texture (full garment, cropped sections, or texture patches) conditions. To address the entanglement challenge when using full garment images as conditions, we develop a two-stage pipeline with explicit disentanglement of style and texture. In the first stage, we generate a human parsing map reflecting the desired style conditioned on the input. In the second stage, we composite textures onto the parsing map areas based on the texture input. To represent complex and non-stationary textures that have never been achieved in previous fashion editing works, we first propose extracting hierarchical and balanced CLIP features and applying position encoding in VTON. Experiments demonstrate superior synthesis quality and personalization enabled by our method. The flexible control over style and texture mixing brings virtual try-on to a new level of user experience for online shopping and fashion design.Comment: Project page: https://ningshuliang.github.io/2023/Arxiv/index.htm

Research Hotspots and Evolution Trends of Carbon Neutrality—Visual Analysis of Bibliometrics Based on CiteSpace

Climate change is one of the most urgent challenges facing the world. All countries should take joint actions to achieve the goal of carbon neutrality, which include controlling global warming to within a 1.5 °C temperature rise, to mitigate the extreme harm caused by climate change. However, ways in which to achieve economically and environmentally sustainable carbon neutrality are yet to be established. Carbon neutrality appears frequently in international policy and the scientific literature, but there is little detailed literature. It is necessary to conduct an in-depth analysis of the development context of its research. This paper analyzed the literature on carbon neutrality using bibliometric methods. A total of 1383 research papers were collected from the “Web of Science core database” from 1995 to 2021. Descriptive statistical analysis and keyword co-occurrence and literature co-citation network analyses were utilized to sort the research hotspots, and the detected bursts, the top 30 keywords in terms of word frequency, and 12 clusters were selected. It was found that the existing carbon neutrality research literature mainly focuses on carbon neutrality energy transformation, carbon neutrality technology development, carbon neutrality effect evaluation, and carbon neutrality industry examples. The analysis process involved comprehensively reading the key articles and considering the co-citation, burstiness, centrality, and other indicators under clustering; the carbon neutrality research was then divided into three stages, and evolving themes were observed. Based on the burst detection, this paper holds that with the energy structure transformation, energy consumption assessment and carbon neutrality schemes of various industries, carbon dioxide capture technology, and biogas resource utilization, urban carbon neutrality policy will become a research hotspot in the future. This paper helps to provide a reference for scholars’ theoretical research and has important reference value for policymakers to formulate relevant policy measures. It is helpful for enterprises to make strategic decisions and determine the direction of technology, for R&D and investment, and it is of considerable significance to promote the research of carbon neutrality technology

Research Hotspots and Evolution Trends of Carbon Neutrality—Visual Analysis of Bibliometrics Based on CiteSpace

Climate change is one of the most urgent challenges facing the world. All countries should take joint actions to achieve the goal of carbon neutrality, which include controlling global warming to within a 1.5 °C temperature rise, to mitigate the extreme harm caused by climate change. However, ways in which to achieve economically and environmentally sustainable carbon neutrality are yet to be established. Carbon neutrality appears frequently in international policy and the scientific literature, but there is little detailed literature. It is necessary to conduct an in-depth analysis of the development context of its research. This paper analyzed the literature on carbon neutrality using bibliometric methods. A total of 1383 research papers were collected from the “Web of Science core database” from 1995 to 2021. Descriptive statistical analysis and keyword co-occurrence and literature co-citation network analyses were utilized to sort the research hotspots, and the detected bursts, the top 30 keywords in terms of word frequency, and 12 clusters were selected. It was found that the existing carbon neutrality research literature mainly focuses on carbon neutrality energy transformation, carbon neutrality technology development, carbon neutrality effect evaluation, and carbon neutrality industry examples. The analysis process involved comprehensively reading the key articles and considering the co-citation, burstiness, centrality, and other indicators under clustering; the carbon neutrality research was then divided into three stages, and evolving themes were observed. Based on the burst detection, this paper holds that with the energy structure transformation, energy consumption assessment and carbon neutrality schemes of various industries, carbon dioxide capture technology, and biogas resource utilization, urban carbon neutrality policy will become a research hotspot in the future. This paper helps to provide a reference for scholars’ theoretical research and has important reference value for policymakers to formulate relevant policy measures. It is helpful for enterprises to make strategic decisions and determine the direction of technology, for R&D and investment, and it is of considerable significance to promote the research of carbon neutrality technology