Multi-Dialect Speech Recognition With A Single Sequence-To-Sequence Model

Sequence-to-sequence models provide a simple and elegant solution for building speech recognition systems by folding separate components of a typical system, namely acoustic (AM), pronunciation (PM) and language (LM) models into a single neural network. In this work, we look at one such sequence-to-sequence model, namely listen, attend and spell (LAS), and explore the possibility of training a single model to serve different English dialects, which simplifies the process of training multi-dialect systems without the need for separate AM, PM and LMs for each dialect. We show that simply pooling the data from all dialects into one LAS model falls behind the performance of a model fine-tuned on each dialect. We then look at incorporating dialect-specific information into the model, both by modifying the training targets by inserting the dialect symbol at the end of the original grapheme sequence and also feeding a 1-hot representation of the dialect information into all layers of the model. Experimental results on seven English dialects show that our proposed system is effective in modeling dialect variations within a single LAS model, outperforming a LAS model trained individually on each of the seven dialects by 3.1 ~ 16.5% relative.Comment: submitted to ICASSP 201

Inositol Trisphosphate 3-Kinase B (InsP3KB) as a Physiological Modulator of Myelopoiesis

Inositol trisphosphate 3-kinase B (InsP3KB) belongs to a family of kinases that convert inositol 1,4,5-trisphosphate (Ins(1,4,5)P3 or IP3) to inositol 1,3,4,5-tetrakisphosphate (Ins(1,3,4,5)P4). Previous studies have shown that disruption of InsP3KB leads to impaired T cell and B cell development as well as hyperactivation of neutrophils. Here, we demonstrate that InsP3KB is also a physiological modulator of myelopoiesis. The InsP3KB gene is expressed in all hematopoietic stem/progenitor cell populations. In InsP3KB null mice, the bone marrow granulocyte monocyte progenitor (GMP) population was expanded, and GMP cells proliferated significantly faster. Consequently, neutrophil production in the bone marrow was enhanced, and the peripheral blood neutrophil count was also substantially elevated in these mice. These effects might be due to enhancement of PtdIns(3,4,5)P3/Akt signaling in the InsP3KB null cells. Phosphorylation of cell cycle-inhibitory protein $p21^{cip1}$, one of the downstream targets of Akt, was augmented, which can lead to the suppression of the cell cycle-inhibitory effect of p21

Pre-Training on Large-Scale Generated Docking Conformations with HelixDock to Unlock the Potential of Protein-ligand Structure Prediction Models

Protein-ligand structure prediction is an essential task in drug discovery, predicting the binding interactions between small molecules (ligands) and target proteins (receptors). Although conventional physics-based docking tools are widely utilized, their accuracy is compromised by limited conformational sampling and imprecise scoring functions. Recent advances have incorporated deep learning techniques to improve the accuracy of structure prediction. Nevertheless, the experimental validation of docking conformations remains costly, it raises concerns regarding the generalizability of these deep learning-based methods due to the limited training data. In this work, we show that by pre-training a geometry-aware SE(3)-Equivariant neural network on a large-scale docking conformation generated by traditional physics-based docking tools and then fine-tuning with a limited set of experimentally validated receptor-ligand complexes, we can achieve outstanding performance. This process involved the generation of 100 million docking conformations, consuming roughly 1 million CPU core days. The proposed model, HelixDock, aims to acquire the physical knowledge encapsulated by the physics-based docking tools during the pre-training phase. HelixDock has been benchmarked against both physics-based and deep learning-based baselines, showing that it outperforms its closest competitor by over 40% for RMSD. HelixDock also exhibits enhanced performance on a dataset that poses a greater challenge, thereby highlighting its robustness. Moreover, our investigation reveals the scaling laws governing pre-trained structure prediction models, indicating a consistent enhancement in performance with increases in model parameters and pre-training data. This study illuminates the strategic advantage of leveraging a vast and varied repository of generated data to advance the frontiers of AI-driven drug discovery

Streaming End-to-end Speech Recognition For Mobile Devices

End-to-end (E2E) models, which directly predict output character sequences given input speech, are good candidates for on-device speech recognition. E2E models, however, present numerous challenges: In order to be truly useful, such models must decode speech utterances in a streaming fashion, in real time; they must be robust to the long tail of use cases; they must be able to leverage user-specific context (e.g., contact lists); and above all, they must be extremely accurate. In this work, we describe our efforts at building an E2E speech recognizer using a recurrent neural network transducer. In experimental evaluations, we find that the proposed approach can outperform a conventional CTC-based model in terms of both latency and accuracy in a number of evaluation categories

BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition

We summarize the results of a host of efforts using giant automatic speech recognition (ASR) models pre-trained using large, diverse unlabeled datasets containing approximately a million hours of audio. We find that the combination of pre-training, self-training and scaling up model size greatly increases data efficiency, even for extremely large tasks with tens of thousands of hours of labeled data. In particular, on an ASR task with 34k hours of labeled data, by fine-tuning an 8 billion parameter pre-trained Conformer model we can match state-of-the-art (SoTA) performance with only 3% of the training data and significantly improve SoTA with the full training set. We also report on the universal benefits gained from using big pre-trained and self-trained models for a large set of downstream tasks that cover a wide range of speech domains and span multiple orders of magnitudes of dataset sizes, including obtaining SoTA performance on many public benchmarks. In addition, we utilize the learned representation of pre-trained networks to achieve SoTA results on non-ASR tasks.Comment: 14 pages, 7 figures, 13 tables; v2: minor corrections, reference baselines and bibliography updated; v3: corrections based on reviewer feedback, bibliography update

Exosome Derived From Human Umbilical Cord Mesenchymal Stem Cell Mediates MiR-181c Attenuating Burn-induced Excessive Inflammation

Mesenchymal stem cell (MSC)-derived exosomes have diverse functions in regulating wound healing and inflammation, however, the molecular mechanism of human umbilical cord MSC (hUCMSC)-derived exosomes in regulating burn-induced inflammation is not well understood. We found that burn injury significantly increased the inflammatory reaction of rats or macrophages exposed to lipopolysaccharide (LPS), increased tumor necrosis factor α (TNF-α) and interleukin-1β (IL-1β) levels and decreased IL-10 levels. hUCMSC-exosome administration successfully reversed this reaction. Further studies showed that miR-181c in the exosomes played a pivotal role in regulating inflammation. Compared to control hUCMSC-exosomes, hUCMSC-exosomes overexpressing miR-181c more effectively suppressed the TLR4 signaling pathway and alleviated inflammation in burned rats. Administration of miR-181c-expressing hUCMSC-exosomes or TLR4 knockdown significantly reduced LPS-induced TLR4 expression by macrophages and the inflammatory reaction. In summary, miR-181c expression in hUCMSC-exosomes reduces burn-induced inflammation by downregulating the TLR4 signaling pathway

Exosome Derived From Human Umbilical Cord Mesenchymal Stem Cell Mediates MiR-181c Attenuating Burn-induced Excessive Inflammation

Mesenchymal stem cell (MSC)-derived exosomes have diverse functions in regulating wound healing and inflammation; however, the molecular mechanism of human umbilical cord MSC (hUCMSC)-derived exosomes in regulating burn-induced inflammation is not well understood. We found that burn injury significantly increased the inflammatory reaction of rats or macrophages exposed to lipopolysaccharide (LPS), increased tumor necrosis factor α (TNF-α) and interleukin-1β (IL-1β) levels and decreased IL-10 levels. hUCMSC-exosome administration successfully reversed this reaction. Further studies showed that miR-181c in the exosomes played a pivotal role in regulating inflammation. Compared to control hUCMSC-exosomes, hUCMSC-exosomes overexpressing miR-181c more effectively suppressed the TLR4 signaling pathway and alleviated inflammation in burned rats. Administration of miR-181c-expressing hUCMSC-exosomes or TLR4 knockdown significantly reduced LPS-induced TLR4 expression by macrophages and the inflammatory reaction. In summary, miR-181c expression in hUCMSC-exosomes reduces burn-induced inflammation by downregulating the TLR4 signaling pathway

Human umbilical cord mesenchymal stem cells transplantation promotes cutaneous wound healing of severe burned rats.

BACKGROUND: Severe burns are a common and highly lethal trauma. The key step for severe burn therapy is to promote the wound healing as early as possible, and reports indicate that mesenchymal stem cell (MSC) therapy contributes to facilitate wound healing. In this study, we investigated effect of human umbilical cord MSCs (hUC-MSCs) could on wound healing in a rat model of severe burn and its potential mechanism. METHODS: Adult male Wistar rats were randomly divided into sham, burn, and burn transplanted hUC-MSCs. GFP labeled hUC-MSCs or PBS was intravenous injected into respective groups. The rate of wound closure was evaluated by Image Pro Plus. GFP-labeled hUC-MSCs were tracked by in vivo bioluminescence imaging (BLI), and human-specific DNA expression in wounds was detected by PCR. Inflammatory cells, neutrophils, macrophages, capillaries and collagen types I/III in wounds were evaluated by histochemical staining. Wound blood flow was evaluated by laser Doppler blood flow meter. The levels of proinflammatory and anti-inflammatory factors, VEGF, collagen types I/III in wounds were analyzed using an ELISA. RESULTS: We found that wound healing was significantly accelerated in the hUC-MSC therapy group. The hUC-MSCs migrated into wound and remarkably decreased the quantity of infiltrated inflammatory cells and levels of IL-1, IL-6, TNF-α and increased levels of IL-10 and TSG-6 in wounds. Additionally, the neovascularization and levels of VEGF in wounds in the hUC-MSC therapy group were markedly higher than those in other control groups. The ratio of collagen types I and III in the hUC-MSC therapy group were markedly higher than that in the burn group at indicated time after transplantation. CONCLUSION: The study suggests that hUC-MSCs transplantation can effectively improve wound healing in severe burned rat model. Moreover, these data might provide the theoretical foundation for the further clinical application of hUC-MSC in burn areas

Exosome Derived From Human Umbilical Cord Mesenchymal Stem Cell Mediates MiR-181c Attenuating Burn-induced Excessive Inflammation

Mesenchymal stem cell (MSC)-derived exosomes have diverse functions in regulating wound healing and inflammation, however, the molecular mechanism of human umbilical cord MSC (hUCMSC)-derived exosomes in regulating burn-induced inflammation is not well understood. We found that burn injury significantly increased the inflammatory reaction of rats or macrophages exposed to lipopolysaccharide (LPS), increased tumor necrosis factor α (TNF-α) and interleukin-1β (IL-1β) levels and decreased IL-10 levels. hUCMSC-exosome administration successfully reversed this reaction. Further studies showed that miR-181c in the exosomes played a pivotal role in regulating inflammation. Compared to control hUCMSC-exosomes, hUCMSC-exosomes overexpressing miR-181c more effectively suppressed the TLR4 signaling pathway and alleviated inflammation in burned rats. Administration of miR-181c-expressing hUCMSC-exosomes or TLR4 knockdown significantly reduced LPS-induced TLR4 expression by macrophages and the inflammatory reaction. In summary, miR-181c expression in hUCMSC-exosomes reduces burn-induced inflammation by downregulating the TLR4 signaling pathway

Inositol trisphosphate 3-kinase B (InsP3KB) as a physiological modulator of myelopoiesis

Inositol trisphosphate 3-kinase B (InsP3KB) belongs to a family of kinases that convert inositol 1,4,5-trisphosphate (Ins(1,4,5)P3 or IP3) to inositol 1,3,4,5-tetrakisphosphate (Ins(1,3,4,5)P4). Previous studies have shown that disruption of InsP3KB leads to impaired T cell and B cell development as well as hyperactivation of neutrophils. Here, we demonstrate that InsP3KB is also a physiological modulator of myelopoiesis. The InsP3KB gene is expressed in all hematopoietic stem/progenitor cell populations. In InsP3KB null mice, the bone marrow granulocyte monocyte progenitor (GMP) population was expanded, and GMP cells proliferated significantly faster. Consequently, neutrophil production in the bone marrow was enhanced, and the peripheral blood neutrophil count was also substantially elevated in these mice. These effects might be due to enhancement of PtdIns(3,4,5)P3/Akt signaling in the InsP3KB null cells. Phosphorylation of cell cycle-inhibitory protein p21cip1, one of the downstream targets of Akt, was augmented, which can lead to the suppression of the cell cycle-inhibitory effect of p21