Conversational Co-Speech Gesture Generation via Modeling Dialog Intention, Emotion, and Context with Diffusion Models

Audio-driven co-speech human gesture generation has made remarkable advancements recently. However, most previous works only focus on single person audio-driven gesture generation. We aim at solving the problem of conversational co-speech gesture generation that considers multiple participants in a conversation, which is a novel and challenging task due to the difficulty of simultaneously incorporating semantic information and other relevant features from both the primary speaker and the interlocutor. To this end, we propose CoDiffuseGesture, a diffusion model-based approach for speech-driven interaction gesture generation via modeling bilateral conversational intention, emotion, and semantic context. Our method synthesizes appropriate interactive, speech-matched, high-quality gestures for conversational motions through the intention perception module and emotion reasoning module at the sentence level by a pretrained language model. Experimental results demonstrate the promising performance of the proposed method.Comment: 5 pages,2 figures, Accepted for publication at the 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2024

Effect of Natural Nanostructured Rods and Platelets on Mechanical and Water Resistance Properties of Alginate-Based Nanocomposites

A series of biopolymer-based nanocomposite films were prepared by incorporating natural one-dimensional (1D) palygorskite (PAL) nanorods, and two-dimensional (2D) montmorillonite (MMT) nanoplatelets into sodium alginate (SA) film by a simple solution casting method. The effect of different dimensions of nanoclays on the mechanical, water resistance, and light transmission properties of the SA/PAL or MMT nanocomposite films were studied. The field-emission scanning electron microscopy (FE-SEM) result showed that PAL can disperse better than MMT in the SA matrix in the case of the same addition amount. The incorporation of both PAL and MMT into the SA film can enhance the tensile strength (TS) and water resistance capability of the film. At a high content of nanoclays, the SA/PAL nanocomposite film shows relatively higher TS, and better water resistance than the SA/MMT nanocomposite film. The SA/MMT nanocomposite films have better light transmission than SA/PAL nanocomposite film at the same loading amount of nanoclays. These results demonstrated that 1D PAL nanorods are more suitable candidate of inorganic filler to improve the mechanical and water resistance properties of biopolymers/nanoclays nanocomposites

UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletons

The automatic co-speech gesture generation draws much attention in computer animation. Previous works designed network structures on individual datasets, which resulted in a lack of data volume and generalizability across different motion capture standards. In addition, it is a challenging task due to the weak correlation between speech and gestures. To address these problems, we present UnifiedGesture, a novel diffusion model-based speech-driven gesture synthesis approach, trained on multiple gesture datasets with different skeletons. Specifically, we first present a retargeting network to learn latent homeomorphic graphs for different motion capture standards, unifying the representations of various gestures while extending the dataset. We then capture the correlation between speech and gestures based on a diffusion model architecture using cross-local attention and self-attention to generate better speech-matched and realistic gestures. To further align speech and gesture and increase diversity, we incorporate reinforcement learning on the discrete gesture units with a learned reward function. Extensive experiments show that UnifiedGesture outperforms recent approaches on speech-driven gesture generation in terms of CCA, FGD, and human-likeness. All code, pre-trained models, databases, and demos are available to the public at https://github.com/YoungSeng/UnifiedGesture.Comment: 16 pages, 11 figures, ACM MM 202

Sub-second periodic radio oscillations in a microquasar

Powerful relativistic jets are one of the ubiquitous features of accreting black holes in all scales. GRS 1915+105 is a well-known fast-spinning black-hole X-ray binary with a relativistic jet, termed as a ``microquasar'', as indicated by its superluminal motion of radio emission. It exhibits persistent x-ray activity over the last 30 years, with quasi-periodic oscillations of $\sim 1-10$ Hz and 34 and 67 Hz in the x-ray band. These oscillations likely originate in the inner accretion disk, but other origins have been considered. Radio observations found variable light curves with quasi-periodic flares or oscillations with periods of $\sim 20-50$ minutes. Here we report two instances of $\sim$5 Hz transient periodic oscillation features from the source detected in the 1.05-1.45 GHz radio band that occurred in January 2021 and June 2022, respectively. Circular polarization was also observed during the oscillation phase.Comment: The author version of the article which will appear in Nature on 26 July 2023, 32 pages including the extended data. The online publication version can be found at the following URL: https://www.nature.com/articles/s41586-023-06336-

Transposable elements cause the loss of self-incompatibility in citrus

Self-incompatibility (SI) is a widespread prezygotic mechanism for flowering plants to avoid inbreeding depression and promote genetic diversity. Citrus has an S-RNase-based SI system, which was frequently lost during evolution. We previously identified a single nucleotide mutation in Sm-RNase, which is responsible for the loss of SI in mandarin and its hybrids. However, little is known about other mechanisms responsible for conversion of SI to self-compatibility (SC) and we identify a completely different mechanism widely utilized by citrus. Here, we found a 786-bp miniature inverted-repeat transposable element (MITE) insertion in the promoter region of the FhiS2-RNase in Fortunella hindsii Swingle (a model plant for citrus gene function), which does not contain the Sm-RNase allele but are still SC. We demonstrate that this MITE plays a pivotal role in the loss of SI in citrus, providing evidence that this MITE insertion prevents expression of the S-RNase; moreover, transgenic experiments show that deletion of this 786-bp MITE insertion recovers the expression of FhiS2-RNase and restores SI. This study identifies the first evidence for a role for MITEs at the S-locus affecting the SI phenotype. A family-wide survey of the S-locus revealed that MITE insertions occur frequently adjacent to S-RNase alleles in different citrus genera, but only certain MITEs appear to be responsible for the loss of SI. Our study provides evidence that insertion of MITEs into a promoter region can alter a breeding strategy and suggests that this phenomenon may be broadly responsible for SC in species with the S-RNase system

Yi: Open Foundation Models by 01.AI

We introduce the Yi model family, a series of language and multimodal models that demonstrate strong multi-dimensional capabilities. The Yi model family is based on 6B and 34B pretrained language models, then we extend them to chat models, 200K long context models, depth-upscaled models, and vision-language models. Our base models achieve strong performance on a wide range of benchmarks like MMLU, and our finetuned chat models deliver strong human preference rate on major evaluation platforms like AlpacaEval and Chatbot Arena. Building upon our scalable super-computing infrastructure and the classical transformer architecture, we attribute the performance of Yi models primarily to its data quality resulting from our data-engineering efforts. For pretraining, we construct 3.1 trillion tokens of English and Chinese corpora using a cascaded data deduplication and quality filtering pipeline. For finetuning, we polish a small scale (less than 10K) instruction dataset over multiple iterations such that every single instance has been verified directly by our machine learning engineers. For vision-language, we combine the chat language model with a vision transformer encoder and train the model to align visual representations to the semantic space of the language model. We further extend the context length to 200K through lightweight continual pretraining and demonstrate strong needle-in-a-haystack retrieval performance. We show that extending the depth of the pretrained checkpoint through continual pretraining further improves performance. We believe that given our current results, continuing to scale up model parameters using thoroughly optimized data will lead to even stronger frontier models

A note on Cauchy-Lipschitz-Picard theorem

Abstract In this note, we try to generalize the classical Cauchy-Lipschitz-Picard theorem on the global existence and uniqueness for the Cauchy initial value problem of the ordinary differential equation with global Lipschitz condition, and we try to weaken the global Lipschitz condition. We can also get the global existence and uniqueness

Reducing Water Sensitivity of Chitosan Biocomposite Films Using Gliadin Particles Made by In Situ Method

In order to sustain rapid expansion in the field of biocomposites, it is necessary to develop novel fillers that are biodegradable, and easy to disperse and obtain. In this work, gliadin particles (GPs) fabricated through an in situ method have been reported as fillers for creating chitosan (CS)-based biocomposite films. In general, the particles tend to agglomerate in the polymer matrix at high loading (approximately >10%) in the biopolymer/particles composites prepared by the traditional solution-blending method. However, the micrographs of biocomposites confirmed that the GPs are well dispersed in the CS matrix in all CS/GPs composites even at a high loading of 30% in this study. It was found that the GPs could improve the mechanical properties of the biocomposites. In addition, the results of moisture uptake and solubility in water of biocomposites showed that water resistance of biocomposites was enhanced by the introduction of GPs. These results suggested that GPs fabricated through an in situ method could be a good candidate for use in biopolymer-based composites