MDSC: Towards Evaluating the Style Consistency Between Music and

We propose MDSC(Music-Dance-Style Consistency), the first evaluation metric which assesses to what degree the dance moves and music match. Existing metrics can only evaluate the fidelity and diversity of motion and the degree of rhythmic matching between music and motion. MDSC measures how stylistically correlated the generated dance motion sequences and the conditioning music sequences are. We found that directly measuring the embedding distance between motion and music is not an optimal solution. We instead tackle this through modelling it as a clustering problem. Specifically, 1) we pre-train a music encoder and a motion encoder, then 2) we learn to map and align the motion and music embedding in joint space by jointly minimizing the intra-cluster distance and maximizing the inter-cluster distance, and 3) for evaluation purpose, we encode the dance moves into embedding and measure the intra-cluster and inter-cluster distances, as well as the ratio between them. We evaluate our metric on the results of several music-conditioned motion generation methods, combined with user study, we found that our proposed metric is a robust evaluation metric in measuring the music-dance style correlation. The code is available at: https://github.com/zixiangzhou916/MDSC.Comment: 17 pages, 17 figur

A Unified Framework for Multimodal, Multi-Part Human Motion Synthesis

The field has made significant progress in synthesizing realistic human motion driven by various modalities. Yet, the need for different methods to animate various body parts according to different control signals limits the scalability of these techniques in practical scenarios. In this paper, we introduce a cohesive and scalable approach that consolidates multimodal (text, music, speech) and multi-part (hand, torso) human motion generation. Our methodology unfolds in several steps: We begin by quantizing the motions of diverse body parts into separate codebooks tailored to their respective domains. Next, we harness the robust capabilities of pre-trained models to transcode multimodal signals into a shared latent space. We then translate these signals into discrete motion tokens by iteratively predicting subsequent tokens to form a complete sequence. Finally, we reconstruct the continuous actual motion from this tokenized sequence. Our method frames the multimodal motion generation challenge as a token prediction task, drawing from specialized codebooks based on the modality of the control signal. This approach is inherently scalable, allowing for the easy integration of new modalities. Extensive experiments demonstrated the effectiveness of our design, emphasizing its potential for broad application.Comment: 19 pages, 18 figure

AvatarGPT: All-in-One Framework for Motion Understanding, Planning, Generation and Beyond

Large Language Models(LLMs) have shown remarkable emergent abilities in unifying almost all (if not every) NLP tasks. In the human motion-related realm, however, researchers still develop siloed models for each task. Inspired by InstuctGPT, and the generalist concept behind Gato, we introduce AvatarGPT, an All-in-One framework for motion understanding, planning, generations as well as other tasks such as motion in-between synthesis. AvatarGPT treats each task as one type of instruction fine-tuned on the shared LLM. All the tasks are seamlessly interconnected with language as the universal interface, constituting a closed-loop within the framework. To achieve this, human motion sequences are first encoded as discrete tokens, which serve as the extended vocabulary of LLM. Then, an unsupervised pipeline to generate natural language descriptions of human action sequences from in-the-wild videos is developed. Finally, all tasks are jointly trained. Extensive experiments show that AvatarGPT achieves SOTA on low-level tasks, and promising results on high-level tasks, demonstrating the effectiveness of our proposed All-in-One framework. Moreover, for the first time, AvatarGPT enables a principled approach by iterative traversal of the tasks within the closed-loop for unlimited long-motion synthesis.Comment: 22 pages, 21 figure

Summary of Dissertation Recitals

The three dissertation recitals explore musical works in Romantic style by major composers including Robert Schumann, Frédéric Chopin, Franz Liszt, Alexander Scriabin, and Sergei Rachmaninoff, with the exception of an early sonata by Franz Joseph Haydn, which was offered for contrast. The first recital featured works by Chopin and Liszt. The first half opened with several of Chopin’s character pieces, including two mazurkas, two etudes, and a nocturne, and closed with one of the composer’s most demanding works, his Barcarolle. The second half concluded with some of Liszt’s less frequently performed pieces, including transcriptions of two songs by Ludwig van Beethoven and one from George Frideric Handel’s opera, Almira, and an original waltz, in addition to his virtuosic Transcendental Étude No. 2. The second recital presented three sonata or sonata-like works by Haydn, Schumann, and Rachmaninoff. It opened with the classical sonata that demonstrated Haydn’s lighthearted and lyrical styles, and proceeded to Schumann’s most emotional work, the Fantasie in C major, Op.17, originally entitled “a sonata for Beethoven” as a contribution to Beethoven’s monument in Bonn. The second half concluded with a substantial, but infrequently performed, work by Rachmaninoff, his Piano Sonata No. 1 in D minor, Op. 28. The lecture-recital showcased Scriabin’s Piano Sonata No.1. The lecture examined the work’s cultural and musical inspiration, considered the musical details of each movement, finally revealing some extra-musical aspects of this sonata. Though this sonata and Rachmaninoff’s first sonata are unfortunately neglected by performers and listeners, their artistic beauty and emotional power deserve to be revealed. Tuesday, November 26, 2019, 8:00 p.m., Walgreen Drama Center, Stamps Auditorium, The University of Michigan. Frederic Chopin Mazurka in B-flat minor, Op. 24 No.4, Mazurka in C minor, Op. 30 No.1, Nocturne in B major, Op. 62 No. 1, Etude in C-sharp minor, Op. 10 No. 4, Etude in E minor, Op. 25 No. 5, Barcarolle in F-sharp major, Op. 60; Franz Liszt Sarabande un Chaconne über Themen aus dem Singspiel “Almira” von G. F. Händel, S. 181, Liszt/Beethoven Wonne der Wehmut, Op. 83 No. 1 (S. 468/5), Mit einem gemalten Bande, Op. 83, No. 3 (S. 468/2), Franz Liszt Transcendental Etude No. 2 in A minor, S. 139/2, Trois Caprices-Valses: Valse de bravoure, S. 214/1. Friday, January 17, 2020, 7:30 p.m., Performance Hall, Wuhan. Joseph Haydn Keyboard Sonata in A-flat Major, Hob. XVI: 46; Robert Schumann Fantasy in C major, Op. 17; Sergei Rachmaninoff Piano Sonata No. 1 in D minor, Op. 28. Wednesday, September 23, 2020, 7:30 p.m., School of Music, Theatre & Dance, Britton Recital Hall, The University of Michigan. Alexander Scriabin Piano Sonata No. 1 in F minor, Op. 6: Its Inspiration, Musical Language and Emotional Power.AMUMusic: PerformanceUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/166124/1/zxw_1.pd

Quantum Discord for Investigating Quantum Correlations without Entanglement in Solids

Quantum systems unfold diversified correlations which have no classical counterparts. These quantum correlations have various different facets. Quantum entanglement, as the most well known measure of quantum correlations, plays essential roles in quantum information processing. However, it has recently been pointed out that quantum entanglement cannot describe all the nonclassicality in the correlations. Thus the study of quantum correlations in separable states attracts widely attentions. Herein, we experimentally investigate the quantum correlations of separable thermal states in terms of quantum discord. The sudden change of quantum discord is observed, which captures ambiguously the critical point associated with the behavior of Hamiltonian. Our results display the potential applications of quantum correlations in studying the fundamental properties of quantum system, such as quantum criticality of non-zero temperature.Comment: 4 pages, 4 figure

Coherence-protected Quantum Gate by Continuous Dynamical Decoupling in Diamond

To implement reliable quantum information processing, quantum gates have to be protected together with the qubits from decoherence. Here we demonstrate experimentally on nitrogen-vacancy system that by using continuous wave dynamical decoupling method, not only the coherence time is prolonged by about 20 times, but also the quantum gates is protected for the duration of controlling time. This protocol shares the merits of retaining the superiority of prolonging the coherence time and at the same time easily combining with quantum logic tasks. It is expected to be useful in task where duration of quantum controlling exceeds far beyond the dephasing time.Comment: 5 pages, 4 figure

Exact values and improved bounds on kk-neighborly families of boxes

A finite family $\mathcal{F}$ of $d$-dimensional convex polytopes is called $k$-neighborly if $d-k\le\textup{dim}(C\cap C')\le d-1$ for any two distinct members $C,C'\in\mathcal{F}$. In 1997, Alon initiated the study of the general function $n(k,d)$, which is defined to be the maximum size of $k$-neighborly families of standard boxes in $\mathbb{R}^{d}$. Based on a weighted count of vectors in $\{0,1\}^{d}$, we improve a recent upper bound on $n(k,d)$ by Alon, Grytczuk, Kisielewicz, and Przes\l awski for any positive integers $d$ and $k$ with $d\ge k+2$. In particular, when $d$ is sufficiently large and $k\ge 0.123d$, our upper bound on $n(k,d)$ improves the bound $\sum_{i=1}^{k}2^{i-1}\binom{d}{i}+1$ shown by Huang and Sudakov exponentially. Furthermore, we determine that $n(2,4)=9$, $n(3,5)=18$, $n(3,6)=27$, $n(4,6)=37$, $n(5,7)=74$, and $n(6,8)=150$. The stability result of Kleitman's isodiametric inequality plays an important role in the proofs.Comment: 17 pages. The main results were further improve