10,219 research outputs found
Music-Driven Group Choreography
Music-driven choreography is a challenging problem with a wide variety of
industrial applications. Recently, many methods have been proposed to
synthesize dance motions from music for a single dancer. However, generating
dance motion for a group remains an open problem. In this paper, we present
, a new large-scale dataset for music-driven group dance
generation. Unlike existing datasets that only support single dance, our new
dataset contains group dance videos, hence supporting the study of group
choreography. We propose a semi-autonomous labeling method with humans in the
loop to obtain the 3D ground truth for our dataset. The proposed dataset
consists of 16.7 hours of paired music and 3D motion from in-the-wild videos,
covering 7 dance styles and 16 music genres. We show that naively applying
single dance generation technique to creating group dance motion may lead to
unsatisfactory results, such as inconsistent movements and collisions between
dancers. Based on our new dataset, we propose a new method that takes an input
music sequence and a set of 3D positions of dancers to efficiently produce
multiple group-coherent choreographies. We propose new evaluation metrics for
measuring group dance quality and perform intensive experiments to demonstrate
the effectiveness of our method. Our project facilitates future research on
group dance generation and is available at:
https://aioz-ai.github.io/AIOZ-GDANCE/Comment: accepted in CVPR 202
Dance Revolution: Long-Term Dance Generation with Music via Curriculum Learning
Dancing to music is one of human's innate abilities since ancient times. In
machine learning research, however, synthesizing dance movements from music is
a challenging problem. Recently, researchers synthesize human motion sequences
through autoregressive models like recurrent neural network (RNN). Such an
approach often generates short sequences due to an accumulation of prediction
errors that are fed back into the neural network. This problem becomes even
more severe in the long motion sequence generation. Besides, the consistency
between dance and music in terms of style, rhythm and beat is yet to be taken
into account during modeling. In this paper, we formalize the music-driven
dance generation as a sequence-to-sequence learning problem and devise a novel
seq2seq architecture to efficiently process long sequences of music features
and capture the fine-grained correspondence between music and dance.
Furthermore, we propose a novel curriculum learning strategy to alleviate error
accumulation of autoregressive models in long motion sequence generation, which
gently changes the training process from a fully guided teacher-forcing scheme
using the previous ground-truth movements, towards a less guided autoregressive
scheme mostly using the generated movements instead. Extensive experiments show
that our approach significantly outperforms the existing state-of-the-arts on
automatic metrics and human evaluation. We also make a demo video in the
supplementary material to demonstrate the superior performance of our proposed
approach.Comment: Accepted by ICLR 202
EDGE: Editable Dance Generation From Music
Dance is an important human art form, but creating new dances can be
difficult and time-consuming. In this work, we introduce Editable Dance
GEneration (EDGE), a state-of-the-art method for editable dance generation that
is capable of creating realistic, physically-plausible dances while remaining
faithful to the input music. EDGE uses a transformer-based diffusion model
paired with Jukebox, a strong music feature extractor, and confers powerful
editing capabilities well-suited to dance, including joint-wise conditioning,
and in-betweening. We introduce a new metric for physical plausibility, and
evaluate dance quality generated by our method extensively through (1) multiple
quantitative metrics on physical plausibility, beat alignment, and diversity
benchmarks, and more importantly, (2) a large-scale user study, demonstrating a
significant improvement over previous state-of-the-art methods. Qualitative
samples from our model can be found at our website.Comment: Project website: https://edge-dance.github.i
Dance Generation by Sound Symbolic Words
This study introduces a novel approach to generate dance motions using
onomatopoeia as input, with the aim of enhancing creativity and diversity in
dance generation. Unlike text and music, onomatopoeia conveys rhythm and
meaning through abstract word expressions without constraints on expression and
without need for specialized knowledge. We adapt the AI Choreographer framework
and employ the Sakamoto system, a feature extraction method for onomatopoeia
focusing on phonemes and syllables. Additionally, we present a new dataset of
40 onomatopoeia-dance motion pairs collected through a user survey. Our results
demonstrate that the proposed method enables more intuitive dance generation
and can create dance motions using sound-symbolic words from a variety of
languages, including those without onomatopoeia. This highlights the potential
for diverse dance creation across different languages and cultures, accessible
to a wider audience. Qualitative samples from our model can be found at:
https://sites.google.com/view/onomatopoeia-dance/home/
Beating-time gestures imitation learning for humanoid robots
Beating-time gestures are movement patterns of the hand swaying along with music, thereby indicating accented musical pulses. The spatiotemporal configuration of these patterns makes it diÿcult to analyse and model them. In this paper we present an innovative modelling approach that is based upon imitation learning or Programming by Demonstration (PbD). Our approach - based on Dirichlet Process Mixture Models, Hidden Markov Models, Dynamic Time Warping, and non-uniform cubic spline regression - is particularly innovative as it handles spatial and temporal variability by the generation of a generalised trajectory from a set of periodically repeated movements. Although not within the scope of our study, our procedures may be implemented for the sake of controlling movement behaviour of robots and avatar animations in response to music
A view of computer music from New Zealand: Auckland, Waikato and the Asia/Pacific connection
Dealing predominantly with ‘art music’ aspects of electroacoustic music practice, this paper looks at cultural, aesthetic, environmental and technical influences on current and emerging practices from the upper half of the North Island of New Zealand. It also discusses the influences of Asian and Pacific cultures on the idiom locally. Rather than dwell on the similarities with current international styles, the focus is largely on some of the differences
- …