Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systems

Neural models have become ubiquitous in automatic speech recognition systems. While neural networks are typically used as acoustic models in more complex systems, recent studies have explored end-to-end speech recognition systems based on neural networks, which can be trained to directly predict text from input acoustic features. Although such systems are conceptually elegant and simpler than traditional systems, it is less obvious how to interpret the trained models. In this work, we analyze the speech representations learned by a deep end-to-end model that is based on convolutional and recurrent layers, and trained with a connectionist temporal classification (CTC) loss. We use a pre-trained model to generate frame-level features which are given to a classifier that is trained on frame classification into phones. We evaluate representations from different layers of the deep model and compare their quality for predicting phone labels. Our experiments shed light on important aspects of the end-to-end model such as layer depth, model complexity, and other design choices.Comment: NIPS 201

Immunohistochemical detection of B3GNT3 protein expression in paraffin-embedded tissues.

<p>Positive B3GNT3 staining was observed mainly in the cytoplasm of cervical cancer cells. (A) a and b, B3GNT3 expression was not detected in normal cervical tissues; c and d, representative images of weak B3GNT3 staining in cervical cancer tissues; e and f, representative images of moderate B3GNT3 staining in cervical cancer tissues; g and h, representative images of strong B3GNT3 staining in cervical cancer tissues. (B) The statistical analyses of the average mean optical density (MOD) of B3GNT3 staining in the lymph node metastasis group and the lymph node metastasis-free group. *P < 0.05. (C) Kaplan-Meier curves of univariate analysis data (log-rank test).The overall survival (OS) and disease-free survival (DFS) for the patients with high versus low B3GNT3 expression.</p

Overexpression of B3GNT3 mRNA and protein in cervical cancer cell lines.

<p>(a and b) Expression of B3GNT3 mRNA and protein in cervical cancer cell lines (C33a, Ca Ski, HeLa, HeLa 229, MS751, ME-180, SiHa and HCC 94) and normal cervical cell lines were examined by western blotting (a) and quantitative real-time PCR (qPCR) (b). The expression levels were normalized against α-tubulin and GAPDH, respectively. The error bars represent the standard deviation of the mean (SD) calculated from three parallel experiments. *P < 0.05.</p

Kaplan-Meier curves of univariate analysis data (log-rank test).

<p>(a) The overall survival (OS) of the patients without lymph node metastasis with high versus low B3GNT3 expression. (b) The OS for the patients with squamous cell carcinoma antigen > 1.5ng/ml with high versus low B3GNT3 expression. (c) The OS for the patients with HPV infection with high versus low B3GNT3 expression. (d) The OS for the patients with deep stromal invasion with high versus low B3GNT3 expression. (e) The OS for the patients at stages Ib1-Ib2 with high versus low B3GNT3 expression. (f) The OS for the patients at stages IIa1-IIa2 with high versus low B3GNT3 expression. (g) The OS for the patients at differentiation 1–2 with high versus low B3GNT3 expression. (h) The OS for the patients at differentiation 2–3 with high versus low B3GNT3 expression. (i) The OS for the patients with high versus low B3GNT3 expression who received chemotherapy.</p

Additional file 5: Table S4. of Oncogenic miR-210-3p promotes prostate cancer cell EMT and bone metastasis via NF-ÎşB signaling pathway

The relationship between miR-210-3p and clinicopathological characteristics in 149 patients with prostate cancer. (PDF 58 kb

Additional file 9: Figure S5. of Oncogenic miR-210-3p promotes prostate cancer cell EMT and bone metastasis via NF-κB signaling pathway

miR-210-3p expression levels was markedly elevated in metastatic bone tissues compared with that in primary PCa tissues with bone metastasis (BM, n = 6; Bone, n = 7). *P < 0.05. (PDF 28 kb

Additional file 2: Table S2. of Oncogenic miR-210-3p promotes prostate cancer cell EMT and bone metastasis via NF-ÎşB signaling pathway

A list of primers used in the reactions for real-time RT-PCR. (PDF 9 kb

Additional file 10: Figure S6. of Oncogenic miR-210-3p promotes prostate cancer cell EMT and bone metastasis via NF-ÎşB signaling pathway

Clinical correaltion of miR-210-3p with SOCS1, TNIP1 and nuclear p65 in human PCa and bone tissues. (a-c) Correlation between miR-210-3p levels and SOCS1, TNIP1 and nuclear p65 expression in PCa and bone tissues.The expression levels of SOCS1, TNIP1 and nuclear p65 were quantified by densitometry using Quantity One Software, and normalized to the levels of Îą-tubulin and p84, respectively. The sample 1 was used as a standard. The relative expressions of miR-210-3p and these proteins were used to perform the correlation analysis. (PDF 88 kb

Additional file 6: Figure S2. of Oncogenic miR-210-3p promotes prostate cancer cell EMT and bone metastasis via NF-κB signaling pathway

Silencing miR-210-3p repressed EMT, invasion and migration in PC-3 cells in vitro. Real-time PCR analysis of miR-210-3p expression in PC-3 cells transduced with antagomiR-210-3p compared to controls. Transcript levels were normalized by U6 expression. Error bars represent the mean ± s.d. of three independent experiments. *P < 0.05

FOXO3a phosphorylation, transcriptional activity and Akt phosphorylation are regulated by SPHK1 in glioma cells.

<p>(A) SPHK1 phosphorylates FOXO3a in glioma cells. (B) FOXO3a-dependent transcription activity is regulated by overexpression (left panel) or knockdown (right panel) of SPHK1. Error bars represent mean ± SD from three independent experiments with similar results. *, <i>p</i><0.05. The GFP expression was used to indicate the transfection efficiency. (C) WB analysis of nuclear FOXO3a protein in indicated cells. (D) SPHK1 knockdown-induced upregulation of Bim could be reversed by silencing of FOXO3a in indicated glioma cells. (E) The phosphorylation status of Akt at Thr³⁰⁸ and Ser⁴⁷³ were assessed by WB in SPHK1 overexpressed and knocked-down glioma cell lines.</p