6 research outputs found
AgentAvatar: Disentangling Planning, Driving and Rendering for Photorealistic Avatar Agents
In this study, our goal is to create interactive avatar agents that can
autonomously plan and animate nuanced facial movements realistically, from both
visual and behavioral perspectives. Given high-level inputs about the
environment and agent profile, our framework harnesses LLMs to produce a series
of detailed text descriptions of the avatar agents' facial motions. These
descriptions are then processed by our task-agnostic driving engine into motion
token sequences, which are subsequently converted into continuous motion
embeddings that are further consumed by our standalone neural-based renderer to
generate the final photorealistic avatar animations. These streamlined
processes allow our framework to adapt to a variety of non-verbal avatar
interactions, both monadic and dyadic. Our extensive study, which includes
experiments on both newly compiled and existing datasets featuring two types of
agents -- one capable of monadic interaction with the environment, and the
other designed for dyadic conversation -- validates the effectiveness and
versatility of our approach. To our knowledge, we advanced a leap step by
combining LLMs and neural rendering for generalized non-verbal prediction and
photo-realistic rendering of avatar agents.Comment: Project page: https://dorniwang.github.io/AgentAvatar_project
Learning One-Shot 4D Head Avatar Synthesis using Synthetic Data
Existing one-shot 4D head synthesis methods usually learn from monocular
videos with the aid of 3DMM reconstruction, yet the latter is evenly
challenging which restricts them from reasonable 4D head synthesis. We present
a method to learn one-shot 4D head synthesis via large-scale synthetic data.
The key is to first learn a part-wise 4D generative model from monocular images
via adversarial learning, to synthesize multi-view images of diverse identities
and full motions as training data; then leverage a transformer-based animatable
triplane reconstructor to learn 4D head reconstruction using the synthetic
data. A novel learning strategy is enforced to enhance the generalizability to
real images by disentangling the learning process of 3D reconstruction and
reenactment. Experiments demonstrate our superiority over the prior art.Comment: Project page: https://yudeng.github.io/Portrait4D
PICTURE: PhotorealistIC virtual Try-on from UnconstRained dEsigns
In this paper, we propose a novel virtual try-on from unconstrained designs
(ucVTON) task to enable photorealistic synthesis of personalized composite
clothing on input human images. Unlike prior arts constrained by specific input
types, our method allows flexible specification of style (text or image) and
texture (full garment, cropped sections, or texture patches) conditions. To
address the entanglement challenge when using full garment images as
conditions, we develop a two-stage pipeline with explicit disentanglement of
style and texture. In the first stage, we generate a human parsing map
reflecting the desired style conditioned on the input. In the second stage, we
composite textures onto the parsing map areas based on the texture input. To
represent complex and non-stationary textures that have never been achieved in
previous fashion editing works, we first propose extracting hierarchical and
balanced CLIP features and applying position encoding in VTON. Experiments
demonstrate superior synthesis quality and personalization enabled by our
method. The flexible control over style and texture mixing brings virtual
try-on to a new level of user experience for online shopping and fashion
design.Comment: Project page: https://ningshuliang.github.io/2023/Arxiv/index.htm
Research Hotspots and Evolution Trends of Carbon Neutrality—Visual Analysis of Bibliometrics Based on CiteSpace
Climate change is one of the most urgent challenges facing the world. All countries should take joint actions to achieve the goal of carbon neutrality, which include controlling global warming to within a 1.5 °C temperature rise, to mitigate the extreme harm caused by climate change. However, ways in which to achieve economically and environmentally sustainable carbon neutrality are yet to be established. Carbon neutrality appears frequently in international policy and the scientific literature, but there is little detailed literature. It is necessary to conduct an in-depth analysis of the development context of its research. This paper analyzed the literature on carbon neutrality using bibliometric methods. A total of 1383 research papers were collected from the “Web of Science core database” from 1995 to 2021. Descriptive statistical analysis and keyword co-occurrence and literature co-citation network analyses were utilized to sort the research hotspots, and the detected bursts, the top 30 keywords in terms of word frequency, and 12 clusters were selected. It was found that the existing carbon neutrality research literature mainly focuses on carbon neutrality energy transformation, carbon neutrality technology development, carbon neutrality effect evaluation, and carbon neutrality industry examples. The analysis process involved comprehensively reading the key articles and considering the co-citation, burstiness, centrality, and other indicators under clustering; the carbon neutrality research was then divided into three stages, and evolving themes were observed. Based on the burst detection, this paper holds that with the energy structure transformation, energy consumption assessment and carbon neutrality schemes of various industries, carbon dioxide capture technology, and biogas resource utilization, urban carbon neutrality policy will become a research hotspot in the future. This paper helps to provide a reference for scholars’ theoretical research and has important reference value for policymakers to formulate relevant policy measures. It is helpful for enterprises to make strategic decisions and determine the direction of technology, for R&D and investment, and it is of considerable significance to promote the research of carbon neutrality technology
Research Hotspots and Evolution Trends of Carbon Neutrality—Visual Analysis of Bibliometrics Based on CiteSpace
Climate change is one of the most urgent challenges facing the world. All countries should take joint actions to achieve the goal of carbon neutrality, which include controlling global warming to within a 1.5 °C temperature rise, to mitigate the extreme harm caused by climate change. However, ways in which to achieve economically and environmentally sustainable carbon neutrality are yet to be established. Carbon neutrality appears frequently in international policy and the scientific literature, but there is little detailed literature. It is necessary to conduct an in-depth analysis of the development context of its research. This paper analyzed the literature on carbon neutrality using bibliometric methods. A total of 1383 research papers were collected from the “Web of Science core database” from 1995 to 2021. Descriptive statistical analysis and keyword co-occurrence and literature co-citation network analyses were utilized to sort the research hotspots, and the detected bursts, the top 30 keywords in terms of word frequency, and 12 clusters were selected. It was found that the existing carbon neutrality research literature mainly focuses on carbon neutrality energy transformation, carbon neutrality technology development, carbon neutrality effect evaluation, and carbon neutrality industry examples. The analysis process involved comprehensively reading the key articles and considering the co-citation, burstiness, centrality, and other indicators under clustering; the carbon neutrality research was then divided into three stages, and evolving themes were observed. Based on the burst detection, this paper holds that with the energy structure transformation, energy consumption assessment and carbon neutrality schemes of various industries, carbon dioxide capture technology, and biogas resource utilization, urban carbon neutrality policy will become a research hotspot in the future. This paper helps to provide a reference for scholars’ theoretical research and has important reference value for policymakers to formulate relevant policy measures. It is helpful for enterprises to make strategic decisions and determine the direction of technology, for R&D and investment, and it is of considerable significance to promote the research of carbon neutrality technology