Story-to-Motion: Synthesizing Infinite and Controllable Character Animation from Long Text

Generating natural human motion from a story has the potential to transform the landscape of animation, gaming, and film industries. A new and challenging task, Story-to-Motion, arises when characters are required to move to various locations and perform specific motions based on a long text description. This task demands a fusion of low-level control (trajectories) and high-level control (motion semantics). Previous works in character control and text-to-motion have addressed related aspects, yet a comprehensive solution remains elusive: character control methods do not handle text description, whereas text-to-motion methods lack position constraints and often produce unstable motions. In light of these limitations, we propose a novel system that generates controllable, infinitely long motions and trajectories aligned with the input text. (1) We leverage contemporary Large Language Models to act as a text-driven motion scheduler to extract a series of (text, position, duration) pairs from long text. (2) We develop a text-driven motion retrieval scheme that incorporates motion matching with motion semantic and trajectory constraints. (3) We design a progressive mask transformer that addresses common artifacts in the transition motion such as unnatural pose and foot sliding. Beyond its pioneering role as the first comprehensive solution for Story-to-Motion, our system undergoes evaluation across three distinct sub-tasks: trajectory following, temporal action composition, and motion blending, where it outperforms previous state-of-the-art motion synthesis methods across the board. Homepage: https://story2motion.github.io/.Comment: 8 pages, 6 figure

SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modeling

Synthetic data has emerged as a promising source for 3D human research as it offers low-cost access to large-scale human datasets. To advance the diversity and annotation quality of human models, we introduce a new synthetic dataset, SynBody, with three appealing features: 1) a clothed parametric human model that can generate a diverse range of subjects; 2) the layered human representation that naturally offers high-quality 3D annotations to support multiple tasks; 3) a scalable system for producing realistic data to facilitate real-world tasks. The dataset comprises 1.2M images with corresponding accurate 3D annotations, covering 10,000 human body models, 1,187 actions, and various viewpoints. The dataset includes two subsets for human pose and shape estimation as well as human neural rendering. Extensive experiments on SynBody indicate that it substantially enhances both SMPL and SMPL-X estimation. Furthermore, the incorporation of layered annotations offers a valuable training resource for investigating the Human Neural Radiance Fields (NeRF).Comment: Accepted by ICCV 2023. Project webpage: https://synbody.github.io

Digital Life Project: Autonomous 3D Characters with Social Intelligence

In this work, we present Digital Life Project, a framework utilizing language as the universal medium to build autonomous 3D characters, who are capable of engaging in social interactions and expressing with articulated body motions, thereby simulating life in a digital environment. Our framework comprises two primary components: 1) SocioMind: a meticulously crafted digital brain that models personalities with systematic few-shot exemplars, incorporates a reflection process based on psychology principles, and emulates autonomy by initiating dialogue topics; 2) MoMat-MoGen: a text-driven motion synthesis paradigm for controlling the character's digital body. It integrates motion matching, a proven industry technique to ensure motion quality, with cutting-edge advancements in motion generation for diversity. Extensive experiments demonstrate that each module achieves state-of-the-art performance in its respective domain. Collectively, they enable virtual characters to initiate and sustain dialogues autonomously, while evolving their socio-psychological states. Concurrently, these characters can perform contextually relevant bodily movements. Additionally, a motion captioning module further allows the virtual character to recognize and appropriately respond to human players' actions. Homepage: https://digital-life-project.com/Comment: Homepage: https://digital-life-project.com

Study on Galloping Oscillation of Iced Twin Bundle Conductors considering the Effects of Variation of Aerodynamic and Electromagnetic Forces

Due to the occurrence of whole-span and subspan vibration at the same time during the galloping of the iced bundle conductors, the distance between the subconductors changes, resulting in the variation of the aerodynamic parameters for the leeward subconductors. The existing conventional studies usually only consider the whole-span galloping, ignoring the relative motion and the electromagnetic force between the subconductors. Based on considering the above two factors at the same time, a new numerical simulation method to study the galloping behaviors of the iced conductors is presented. Then, the galloping behaviors of the iced twin bundle conductors’ transmission line with different current intensity, initial wind attack angle, and spacer layout are studied. The galloping oscillation behaviors include time histories of vibration displacements, mode, amplitude, frequency, motion traces, and the distance between two subconductors. The simulation results showed that the whole-span and subspan vibration appear at the same time during galloping oscillation and two subconductors may collide with each other when affected by the varying electromagnetic and aerodynamic forces. The effects of varying electromagnetic and aerodynamic forces on galloping behavior cannot be ignored. The new method presented in this work can contribute to the galloping study of the iced bundle conductors

3D printed grafts with gradient structures for organized vascular regeneration

Synthetic vascular grafts suitable for small-diameter arteries (<6 mm) are in great need. However, there are still no commercially available small-diameter vascular grafts (SDVGs) in clinical practice due to thrombosis and stenosis after in vivo implantation. When designing SDVGs, many studies emphasized reendothelization but ignored the importance of reconstruction of the smooth muscle layer (SML). To facilitate rapid SML regeneration, a high-resolution 3D printing method was used to create a novel bilayer SDVG with structures and mechanical properties mimicking natural arteries. Bioinspired by the collagen alignment of SML, the inner layer of the grafts had larger pore sizes and high porosity to accelerate the infiltration of cells and their circumferential alignment, which could facilitate SML reconstruction for compliance restoration and spontaneous endothelialization. The outer layer was designed to induce fibroblast recruitment by low porosity and minor pore size and provide SDVG with sufficient mechanical strength. One month after implantation, the arteries regenerated by 3D-printed grafts exhibited better pulsatility than electrospun grafts, with a compliance (8.9%) approaching that of natural arteries (11.36%) and significantly higher than that of electrospun ones (1.9%). The 3D-printed vascular demonstrated a three-layer structure more closely resembling natural arteries while electrospun grafts showed incomplete endothelium and immature SML. Our study shows the importance of SML reconstruction during vascular graft regeneration and provides an effective strategy to reconstruct blood vessels through 3D-printed structures rapidly

Atomic Replacement of PtNi Nanoalloys within Zn-ZIF‑8 for the Fabrication of a Multisite CO₂ Reduction Electrocatalyst

Exploring the transformation/interconversion pathways of catalytic active metal species (single atoms, clusters, nanoparticles) on a support is crucial for the fabrication of high-efficiency catalysts, the investigation of how catalysts are deactivated, and the regeneration of spent catalysts. Sintering and redispersion represent the two main transformation modes for metal active components in heterogeneous catalysts. Herein, we established a novel solid-state atomic replacement transformation for metal catalysts, through which metal atoms exchanged between single atoms and nanoalloys to form a new set of nanoalloys and single atoms. Specifically, we found that the Ni of the PtNi nanoalloy and the Zn of the ZIF-8-derived Zn1 on nitrogen-doped carbon (Zn1-CN) experienced metal interchange to produce PtZn nanocrystals and Ni single atoms (Ni1-CN) at high temperature. The elemental migration and chemical bond evolution during the atomic replacement displayed a Ni and Zn mutual migration feature. Density functional theory calculations revealed that the atomic replacement was realized by endothermically stretching Zn from the CN support into the nanoalloy and exothermically trapping Ni with defects on the CN support. Owing to the synergistic effect of the PtZn nanocrystal and Ni1-CN, the obtained (PtZn)n/Ni1-CN multisite catalyst showed a lower energy barrier of CO2 protonation and CO desorption than that of the reference catalysts in the CO2 reduction reaction (CO2RR), resulting in a much enhanced CO2RR catalytic performance. This unique atomic replacement transformation was also applicable to other metal alloys such as PtPd