7,790 research outputs found
A Unified Framework for Multimodal, Multi-Part Human Motion Synthesis
The field has made significant progress in synthesizing realistic human
motion driven by various modalities. Yet, the need for different methods to
animate various body parts according to different control signals limits the
scalability of these techniques in practical scenarios. In this paper, we
introduce a cohesive and scalable approach that consolidates multimodal (text,
music, speech) and multi-part (hand, torso) human motion generation. Our
methodology unfolds in several steps: We begin by quantizing the motions of
diverse body parts into separate codebooks tailored to their respective
domains. Next, we harness the robust capabilities of pre-trained models to
transcode multimodal signals into a shared latent space. We then translate
these signals into discrete motion tokens by iteratively predicting subsequent
tokens to form a complete sequence. Finally, we reconstruct the continuous
actual motion from this tokenized sequence. Our method frames the multimodal
motion generation challenge as a token prediction task, drawing from
specialized codebooks based on the modality of the control signal. This
approach is inherently scalable, allowing for the easy integration of new
modalities. Extensive experiments demonstrated the effectiveness of our design,
emphasizing its potential for broad application.Comment: 19 pages, 18 figure
AvatarGPT: All-in-One Framework for Motion Understanding, Planning, Generation and Beyond
Large Language Models(LLMs) have shown remarkable emergent abilities in
unifying almost all (if not every) NLP tasks. In the human motion-related
realm, however, researchers still develop siloed models for each task. Inspired
by InstuctGPT, and the generalist concept behind Gato, we introduce AvatarGPT,
an All-in-One framework for motion understanding, planning, generations as well
as other tasks such as motion in-between synthesis. AvatarGPT treats each task
as one type of instruction fine-tuned on the shared LLM. All the tasks are
seamlessly interconnected with language as the universal interface,
constituting a closed-loop within the framework. To achieve this, human motion
sequences are first encoded as discrete tokens, which serve as the extended
vocabulary of LLM. Then, an unsupervised pipeline to generate natural language
descriptions of human action sequences from in-the-wild videos is developed.
Finally, all tasks are jointly trained. Extensive experiments show that
AvatarGPT achieves SOTA on low-level tasks, and promising results on high-level
tasks, demonstrating the effectiveness of our proposed All-in-One framework.
Moreover, for the first time, AvatarGPT enables a principled approach by
iterative traversal of the tasks within the closed-loop for unlimited
long-motion synthesis.Comment: 22 pages, 21 figure
Searching for Dark Matter Signals in the Left-Right Symmetric Gauge Model with CP Symmetry
We investigate singlet scalar dark matter (DM) candidate in a left-right
symmetric gauge model with two Higgs bidoublets (2HBDM) in which the
stabilization of the DM particle is induced by the discrete symmetries P and
CP. According to the observed DM abundance, we predict the DM direct and
indirect detection cross sections for the DM mass range from 10 GeV to 500 GeV.
We show that the DM indirect detection cross section is not sensitive to the
light Higgs mixing and Yukawa couplings except the resonance regions. The
predicted spin-independent DM-nucleon elastic scattering cross section is found
to be significantly dependent on the above two factors. Our results show that
the future DM direct search experiments can cover the most parts of the allowed
parameter space. The PAMELA antiproton data can only exclude two very narrow
regions in the 2HBDM. It is very difficult to detect the DM direct or indirect
signals in the resonance regions due to the Breit-Wigner resonance effect.Comment: 24 pages, 8 figures. minor changes and a reference added, published
in Phys. Rev.
- …