Internal Language Model Estimation Through Explicit Context Vector Learning for Attention-based Encoder-decoder ASR

An end-to-end (E2E) ASR model implicitly learns a prior Internal Language Model (ILM) from the training transcripts. To fuse an external LM using Bayes posterior theory, the log likelihood produced by the ILM has to be accurately estimated and subtracted. In this paper we propose two novel approaches to estimate the ILM based on Listen-Attend-Spell (LAS) framework. The first method is to replace the context vector of the LAS decoder at every time step with a vector that is learned with training transcripts. Furthermore, we propose another method that uses a lightweight feed-forward network to directly map query vector to context vector in a dynamic sense. Since the context vectors are learned by minimizing the perplexities on training transcripts, and their estimation is independent of encoder output, hence the ILMs are accurately learned for both methods. Experiments show that the ILMs achieve the lowest perplexity, indicating the efficacy of the proposed methods. In addition, they also significantly outperform the shallow fusion method, as well as two previously proposed ILM Estimation (ILME) approaches on several datasets.Comment: Proceedings of INTERSPEEC

Phylogeography and Demographic History of Babina pleuraden (Anura, Ranidae) in Southwestern China

Factors that determine genetic structure of species in southwestern China remain largely unknown. In this study, sequences of two mitochondrial genes (COI and cyt b) were determined to investigate the phylogeography and demography of Babina pleuraden, a pond frog endemic to southwestern China. A total of 262 individuals from 22 populations across the entire range of the species were collected. Our results indicate that B. pleuraden comprises five well-supported mitochondrial lineages roughly corresponding to five geographical areas. The phylogeographic structure of B. pleuraden has been shaped primarily by the unique regional responses of the Yunnan Plateau to the rapid uplift of the Qinghai-Tibetan Plateau occurred c. 2.5 Mya (B phrase of Qingzang Movement) and climatic oscillation during middle Pleistocene (c. 0.64–0.36 Mya), rather than by the paleo-drainage systems. The present wide distribution of the species has resulted from recent population expansion (c. 0.053–0.025 Mya) from multiple refugia prior to the Last Glacial Maximum, corresponding to the scenario of “refugia within refugia”

Bayesian inference tree for <i>B. pleuraden</i> based on the 63 haplotypes from cyt <i>b</i> and COI sequences.

<p>The nodal numbers are BPP, ML, and MP bootstrap values, respectively. Estimated dates in Mya with 95% HPD are given in rectangular boxes.</p

Networks of the 63 haplotypes detected from cyt <i>b</i> and COI sequences of <i>B. pleuraden</i>.

<p>The sizes of the circles are proportional to haplotype frequencies and black dots represent missing haplotypes (not sampled or extinct).</p

Mismatch distributions for total samples and some major lineages of <i>B. pleuraden</i>.

<p>Mismatch distributions for total samples and some major lineages of <i>B. pleuraden</i>.</p

Estimation of the empirical theta values and effective population size.

<p>Estimation of the empirical theta values and effective population size.</p

Models used to test Pleistocene refugial hypotheses.

<p>(a) Single refugium hypothesis: populations from eastern, northwestern, and central Yunnan Plateau are derived from a single refugium and began to expand at the end of the last glacial maximum (<i>c</i>. 18 000 years BP). (b) Multiple refugia hypothesis: two lineages split at the Middle Pleistocene (T = 640 000 years BP), with the northwestern plus central branch diverged into two clades at 360 000 years ago (T3+T4). T1 (200 000 years BP), T2 (110 000 years BP), and T3 (100 000 years BP) were derived from the estimates of TMRCA.</p

<i>S</i>-values for 1000 simulated coalescent genealogies.

<p>(a) Results from simulations within the single refugium hypothesis; (b) Results from simulations within the multiple refugia hypothesis. The black line represents the <i>S</i>-value for our ML genealogy and the dashed lines represent the 95% CI for the distribution.</p

Results of mismatch distribution analysis, neutrality test, and estimation of expansion time.

<p>Results of mismatch distribution analysis, neutrality test, and estimation of expansion time.</p