Laugh Betrays You? Learning Robust Speaker Representation From Speech Containing Non-Verbal Fragments

The success of automatic speaker verification shows that discriminative speaker representations can be extracted from neutral speech. However, as a kind of non-verbal voice, laughter should also carry speaker information intuitively. Thus, this paper focuses on exploring speaker verification about utterances containing non-verbal laughter segments. We collect a set of clips with laughter components by conducting a laughter detection script on VoxCeleb and part of the CN-Celeb dataset. To further filter untrusted clips, probability scores are calculated by our binary laughter detection classifier, which is pre-trained by pure laughter and neutral speech. After that, based on the clips whose scores are over the threshold, we construct trials under two different evaluation scenarios: Laughter-Laughter (LL) and Speech-Laughter (SL). Then a novel method called Laughter-Splicing based Network (LSN) is proposed, which can significantly boost performance in both scenarios and maintain the performance on the neutral speech, such as the VoxCeleb1 test set. Specifically, our system achieves relative 20% and 22% improvement on Laughter-Laughter and Speech-Laughter trials, respectively. The meta-data and sample clips have been released at https://github.com/nevermoreLin/Laugh_LSN.Comment: Submitted to ICASSP202

Tuning Pre-trained Model via Moment Probing

Recently, efficient fine-tuning of large-scale pre-trained models has attracted increasing research interests, where linear probing (LP) as a fundamental module is involved in exploiting the final representations for task-dependent classification. However, most of the existing methods focus on how to effectively introduce a few of learnable parameters, and little work pays attention to the commonly used LP module. In this paper, we propose a novel Moment Probing (MP) method to further explore the potential of LP. Distinguished from LP which builds a linear classification head based on the mean of final features (e.g., word tokens for ViT) or classification tokens, our MP performs a linear classifier on feature distribution, which provides the stronger representation ability by exploiting richer statistical information inherent in features. Specifically, we represent feature distribution by its characteristic function, which is efficiently approximated by using first- and second-order moments of features. Furthermore, we propose a multi-head convolutional cross-covariance (MHC$^3$) to compute second-order moments in an efficient and effective manner. By considering that MP could affect feature learning, we introduce a partially shared module to learn two recalibrating parameters (PSRP) for backbones based on MP, namely MP$_{+}$. Extensive experiments on ten benchmarks using various models show that our MP significantly outperforms LP and is competitive with counterparts at less training cost, while our MP$_{+}$ achieves state-of-the-art performance.Comment: Accepted to ICCV 2023; Project Page: https://github.com/mingzeG/Moment-Probin

New Treatment of Strongly Anisotropic Scattering Phase Functions: The Delta-M+ Method

The treatment of strongly anisotropic scattering phase functions is still a challenge for accurate radiance computations. The new delta-M+ method resolves this problem by introducing a reliable, fast, accurate, and easy-to-use Legendre expansion of the scattering phase function with modified moments. Delta-M+ is an upgrade of the widely used delta-M method that truncates the forward scattering peak with a Dirac delta function, where the + symbol indicates that it essentially matches moments beyond the first M terms. Compared with the original delta-M method, delta-M+ has the same computational efficiency, but for radiance computations, the accuracy and stability have been increased dramatically

Crystal Structure of the Cysteine Desulfurase DndA from Streptomyces lividans Which Is Involved in DNA Phosphorothioation

DNA phosphorothioation is widespread among prokaryotes, and might function to restrict gene transfer among different kinds of bacteria. There has been little investigation into the structural mechanism of the DNA phosphorothioation process. DndA is a cysteine desulfurase which is involved in the first step of DNA phosphorothioation. In this study, we determined the crystal structure of Streptomyces lividans DndA in complex with its covalently bound cofactor PLP, to a resolution of 2.4 Å. Our structure reveals the molecular mechanism that DndA employs to recognize its cofactor PLP, and suggests the potential binding site for the substrate L-cysteine on DndA. In contrast to previously determined structures of cysteine desulfurases, the catalytic cysteine of DndA was found to reside on a β strand. This catalytic cysteine is very far away from the presumable location of the substrate, suggesting that a conformational change of DndA is required during the catalysis process to bring the catalytic cysteine close to the substrate cysteine. Moreover, our in vitro enzymatic assay results suggested that this conformational change is unlikely to be a simple result of random thermal motion, since moving the catalytic cysteine two residues forward or backward in the primary sequence completely disabled the cysteine desulfurase activity of DndA

Notch-Deficient Skin Induces a Lethal Systemic B-Lymphoproliferative Disorder by Secreting TSLP, a Sentinel for Epidermal Integrity

Epidermal keratinocytes form a highly organized stratified epithelium and sustain a competent barrier function together with dermal and hematopoietic cells. The Notch signaling pathway is a critical regulator of epidermal integrity. Here, we show that keratinocyte-specific deletion of total Notch signaling triggered a severe systemic B-lymphoproliferative disorder, causing death. RBP-j is the DNA binding partner of Notch, but both RBP-j–dependent and independent Notch signaling were necessary for proper epidermal differentiation and lipid deposition. Loss of both pathways caused a persistent defect in skin differentiation/barrier formation. In response, high levels of thymic stromal lymphopoietin (TSLP) were released into systemic circulation by Notch-deficient keratinocytes that failed to differentiate, starting in utero. Exposure to high TSLP levels during neonatal hematopoiesis resulted in drastic expansion of peripheral pre- and immature B-lymphocytes, causing B-lymphoproliferative disorder associated with major organ infiltration and subsequent death, a previously unappreciated systemic effect of TSLP. These observations demonstrate that local skin perturbations can drive a lethal systemic disease and have important implications for a wide range of humoral and autoimmune diseases with skin manifestations

Eye-brain connections revealed by multimodal retinal and brain imaging genetics

The retina, an anatomical extension of the brain, forms physiological connections with the visual cortex of the brain. Although retinal structures offer a unique opportunity to assess brain disorders, their relationship to brain structure and function is not well understood. In this study, we conducted a systematic cross-organ genetic architecture analysis of eye-brain connections using retinal and brain imaging endophenotypes. We identified novel phenotypic and genetic links between retinal imaging biomarkers and brain structure and function measures from multimodal magnetic resonance imaging (MRI), with many associations involving the primary visual cortex and visual pathways. Retinal imaging biomarkers shared genetic influences with brain diseases and complex traits in 65 genomic regions, with 18 showing genetic overlap with brain MRI traits. Mendelian randomization suggests bidirectional genetic causal links between retinal structures and neurological and neuropsychiatric disorders, such as Alzheimer’s disease. Overall, our findings reveal the genetic basis for eye-brain connections, suggesting that retinal images can help uncover genetic risk factors for brain disorders and disease-related changes in intracranial structure and function