MelHuBERT: A simplified HuBERT on Mel spectrograms

Self-supervised models have had great success in learning speech representations that can generalize to various downstream tasks. However, most self-supervised models require a large amount of compute and multiple GPUs to train, significantly hampering the development of self-supervised learning. In an attempt to reduce the computation of training, we revisit the training of HuBERT, a highly successful self-supervised model. We improve and simplify several key components, including the loss function, input representation, and training in multiple stages. Our model, MelHuBERT, is able to achieve favorable performance on phone recognition, speaker identification, and automatic speech recognition against HuBERT, while saving 31.2% of the pre-training time, or equivalently 33.5% MACs per one second speech. The code and pre-trained models are available in https://github.com/nervjack2/MelHuBERT.Comment: ASRU 202

Distinct roles for two Caenorhabditis elegans acid-sensing ion channels in an ultradian clock

Biological clocks are fundamental to an organism’s health, controlling periodicity of behaviour and metabolism. Here, we identify two acid-sensing ion channels, with very different proton sensing properties, and describe their role in an ultradian clock, the defecation motor program (DMP) of the nematode Caenorhabditis elegans. An ACD-5-containing channel, on the apical membrane of the intestinal epithelium, is essential for maintenance of luminal acidity, and thus the rhythmic oscillations in lumen pH. In contrast, the second channel, composed of FLR-1, ACD-3 and/or DEL-5, located on the basolateral membrane, controls the intracellular Ca2+ wave and forms a core component of the master oscillator that controls the timing and rhythmicity of the DMP. flr-1 and acd-3/del-5 mutants show severe developmental and metabolic defects. We thus directly link the proton-sensing properties of these channels to their physiological roles in pH regulation and Ca2+ signalling, the generation of an ultradian oscillator, and its metabolic consequences

Compressing Transformer-based self-supervised models for speech processing

Despite the success of Transformers in self- supervised learning with applications to various downstream tasks, the computational cost of training and inference remains a major challenge for applying these models to a wide spectrum of devices. Several isolated attempts have been made to compress Transformers, but the settings and metrics are different across studies. Trade-off at various compression rates are also largely missing in prior work, making it difficult to compare compression techniques. In this work, we aim to provide context for the isolated results, studying several commonly used compression techniques, including weight pruning, head pruning, low-rank approximation, and knowledge distillation. We report trade- off at various compression rate, including wall-clock time, the number of parameters, and the number of multiply-accumulate operations. Our results show that compared to recent approaches, basic compression techniques are strong baselines. We further present several applications of our results, revealing properties of Transformers, such as the significance of diagonal attention heads. In addition, our results lead to a simple combination of compression techniques that improves trade-off over recent approaches. We hope the results would promote more diverse comparisons among model compression techniques and promote the use of model compression as a tool for analyzing models. Our code of compressing speech self-supervised model is available at https://github.com/nervjack2/Speech-SSL-Compression/.Comment: Submitted to IEEE Transactions on Audio, Speech and Language Processing (TASLP

Storage of multiple single-photon pulses emitted from a quantum dot in a solid-state quantum memory

Quantum repeaters are critical components for distributing entanglement over long distances in presence of unavoidable optical losses during transmission. Stimulated by Duan-Lukin-Cirac-Zoller protocol, many improved quantum-repeater protocols based on quantum memories have been proposed, which commonly focus on the entanglement-distribution rate. Among these protocols, the elimination of multi-photons (multi-photon-pairs) and the use of multimode quantum memory are demonstrated to have the ability to greatly improve the entanglement-distribution rate. Here, we demonstrate the storage of deterministic single photons emitted from a quantum dot in a polarization-maintaining solid-state quantum memory; in addition, multi-temporal-mode memory with $1$, $20$ and $100$ narrow single-photon pulses is also demonstrated. Multi-photons are eliminated, and only one photon at most is contained in each pulse. Moreover, the solid-state properties of both sub-systems make this configuration more stable and easier to be scalable. Our work will be helpful in the construction of efficient quantum repeaters based on all-solid-state devicesComment: Published version, including supplementary materia

Anti-HPV16 oncoproteins siRNA therapy for cervical cancer using a novel transdermal peptide PKU12

In this study, an innovative transdermal peptide, #PKU12, was developed based on transdermal peptide TD-1, and the anti-tumor effect of PKU12-based siRNA against HPV was investigated in vivo. Furthermore, transcriptome differences between PKU12 + siRNA treatment and control groups were compared to assess treatment effects. The top five upregulated and downregulated genes identified by RNA sequencing were further subjected to survival analysis. The present study, for the first time, showed that this novel peptide could enhance the transdermal delivery of the siRNA targeting HPV16 L1, E6, and E7. PKU12-based siRNA delivery significantly repressed the mRNA expression levels of HPV16 L1, E6, and E7 in the SiHa xenograft tumors and attenuated tumor growth as well. The RNA-sequencing results showed that a total of 586 DEGs were detected in the PKU12 + siRNA-treated tumor tissues compared to the control tumor tissues. The GSEA analysis revealed that DEGs were inversely associated with the HIF-1 signaling pathway, the TNF signaling pathway, the AGE-RAGE signaling pathway, the NF-kappa B signaling pathway, ferroptosis, the IL-17 signaling pathway, ovarian steroidogenesis, and rheumatoid arthritis. Further functional enrichment analysis revealed that DEGs were significantly enriched in several key pathways, including cytokine–cytokine receptor interaction, the TNF signaling pathway, and the IL-17 signaling pathway. High expression of MYH1, MYH4, FGG, DEPP1, and ZBTB16 was associated with shorter overall survival of patients with cervical cancer; high expression of SULT1E1, RAB3C, CXCR3, and PROX2 was associated with longer overall survival of patients with cervical cancer. In conclusion, the transdermal peptide PKU12 is potentially a good candidate for a siRNA delivery vehicle for the treatment of cervical cancer

Stray dogs as indicators of Toxoplasma gondii distributed in the environment: the first report across an urban-rural gradient in China

<p>Abstract</p> <p>Background</p> <p>Toxoplasmosis is an important parasitic zoonosis caused by the protozoan <it>Toxoplasma gondii </it>that is distributed world-wide and infects a variety of hosts. However, the prevalence of <it>T. gondii </it>in the environment (such as soil, water and food) is largely unknown. Due to the technical difficulty in oocyst counting directly, an alternative assay using the serologic status of <it>T. gondii </it>in free-living animals, such as stray or free-living dogs, as an indicator, can be used to evaluate environmental contamination indirectly, as they are exposed to the same risk of infection as humans and other animals.</p> <p>Results</p> <p>In the present study, 231 stray or free-living dogs across an urban-rural gradient were examined to assess the frequency of <it>T. gondii </it>in the environment. Specific antibodies to <it>T. gondii </it>were found in 93 dogs (40.3%) by enzyme-linked immunosorbent assay (ELISA), and no statistically significant differences were observed in seroprevalences of <it>T. gondii </it>between urban dogs (38.7%) and rural dogs (41%) (<it>p </it>> 0.05).</p> <p>Conclusions</p> <p>A high seroprevalence of <it>T. gondii </it>in stray or free-living dogs in the present study indicates that there would be a wide distribution and a constant infection pressure of <it>T. gondii </it>across an urban-rural gradient, and the oocysts of <it>T. gondii </it>in the environment would be an important source of infection for humans and other animals both in urban and rural areas in China.</p

CIB2 Interacts with TMC1 and TMC2 and is Essential for Mechanotransduction in Auditory Hair Cells

Inner ear hair cells detect sound through deflection of stereocilia, the microvilli-like projections that are arranged in rows of graded heights. Calcium and integrin-binding protein 2 is essential for hearing and localizes to stereocilia, but its exact function is unknown. Here, we have characterized two mutant mouse lines, one lacking calcium and integrin-binding protein 2 and one carrying a human deafness-related Cib2 mutation, and show that both are deaf and exhibit no mechanotransduction in auditory hair cells, despite the presence of tip links that gate the mechanotransducer channels. In addition, mechanotransducing shorter row stereocilia overgrow in hair cell bundles of both Cib2 mutants. Furthermore, we report that calcium and integrin-binding protein 2 binds to the components of the hair cell mechanotransduction complex, TMC1 and TMC2, and these interactions are disrupted by deafness-causing Cib2 mutations. We conclude that calcium and integrin-binding protein 2 is required for normal operation of the mechanotransducer channels and is involved in limiting the growth of transducing stereocilia