SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmark

In this paper, we present SignAvatars, the first large-scale multi-prompt 3D sign language (SL) motion dataset designed to bridge the communication gap for hearing-impaired individuals. While there has been an exponentially growing number of research regarding digital communication, the majority of existing communication technologies primarily cater to spoken or written languages, instead of SL, the essential communication method for hearing-impaired communities. Existing SL datasets, dictionaries, and sign language production (SLP) methods are typically limited to 2D as the annotating 3D models and avatars for SL is usually an entirely manual and labor-intensive process conducted by SL experts, often resulting in unnatural avatars. In response to these challenges, we compile and curate the SignAvatars dataset, which comprises 70,000 videos from 153 signers, totaling 8.34 million frames, covering both isolated signs and continuous, co-articulated signs, with multiple prompts including HamNoSys, spoken language, and words. To yield 3D holistic annotations, including meshes and biomechanically-valid poses of body, hands, and face, as well as 2D and 3D keypoints, we introduce an automated annotation pipeline operating on our large corpus of SL videos. SignAvatars facilitates various tasks such as 3D sign language recognition (SLR) and the novel 3D SL production (SLP) from diverse inputs like text scripts, individual words, and HamNoSys notation. Hence, to evaluate the potential of SignAvatars, we further propose a unified benchmark of 3D SL holistic motion production. We believe that this work is a significant step forward towards bringing the digital world to the hearing-impaired communities. Our project page is at https://signavatars.github.io/Comment: 9 pages; Project page available at https://signavatars.github.io

A Study on Differentiable Logic and LLMs for EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognition 2023

In this technical report, we present our findings from a study conducted on the EPIC-KITCHENS-100 Unsupervised Domain Adaptation task for Action Recognition. Our research focuses on the innovative application of a differentiable logic loss in the training to leverage the co-occurrence relations between verb and noun, as well as the pre-trained Large Language Models (LLMs) to generate the logic rules for the adaptation to unseen action labels. Specifically, the model's predictions are treated as the truth assignment of a co-occurrence logic formula to compute the logic loss, which measures the consistency between the predictions and the logic constraints. By using the verb-noun co-occurrence matrix generated from the dataset, we observe a moderate improvement in model performance compared to our baseline framework. To further enhance the model's adaptability to novel action labels, we experiment with rules generated using GPT-3.5, which leads to a slight decrease in performance. These findings shed light on the potential and challenges of incorporating differentiable logic and LLMs for knowledge extraction in unsupervised domain adaptation for action recognition. Our final submission (entitled `NS-LLM') achieved the first place in terms of top-1 action recognition accuracy.Comment: Technical report submitted to CVPR 2023 EPIC-Kitchens challenge

Synthesis and physical properties of Ce2_2Rh3+δ_{3+\delta}Sb4_4 single crystals

Millimeter-sized Ce$_2$Rh$_{3+\delta}$Sb$_4$ ($\delta\approx 1/8$) single crystals were synthesized by a Bi-flux method and their physical properties were studied by a combination of electrical transport, magnetic and thermodynamic measurements. The resistivity anisotropy $\rho_{a,b}/\rho_{c}\sim2$, manifesting a quasi-one-dimensional electronic character. Magnetic susceptibility measurements confirm $\mathbf{ab}$ as the magnetic easy plane. A long-range antiferromagnetic transition occurs at $T_N=1.4$ K, while clear short-range ordering can be detected well above $T_N$. The low ordering temperature is ascribed to the large Ce-Ce distance as well as the geometric frustration. Kondo scale is estimated to be about 2.4 K, comparable to the strength of magnetic exchange. Ce$_2$Rh$_{3+\delta}$Sb$_4$, therefore, represents a rare example of dense Kondo lattice whose Ruderman-Kittel-Kasuya-Yosida exchange and Kondo coupling are both weak but competing.Comment: 7 pages, 4 figures, 2 table