Translating Phrases in Neural Machine Translation

Phrases play an important role in natural language understanding and machine translation (Sag et al., 2002; Villavicencio et al., 2005). However, it is difficult to integrate them into current neural machine translation (NMT) which reads and generates sentences word by word. In this work, we propose a method to translate phrases in NMT by integrating a phrase memory storing target phrases from a phrase-based statistical machine translation (SMT) system into the encoder-decoder architecture of NMT. At each decoding step, the phrase memory is first re-written by the SMT model, which dynamically generates relevant target phrases with contextual information provided by the NMT model. Then the proposed model reads the phrase memory to make probability estimations for all phrases in the phrase memory. If phrase generation is carried on, the NMT decoder selects an appropriate phrase from the memory to perform phrase translation and updates its decoding state by consuming the words in the selected phrase. Otherwise, the NMT decoder generates a word from the vocabulary as the general NMT decoder does. Experiment results on the Chinese to English translation show that the proposed model achieves significant improvements over the baseline on various test sets.Comment: Accepted by EMNLP 201

Combining linguistics and statistics for high-quality limited domain English-Chinese machine translation

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2008.Includes bibliographical references (p. 86-87).Second language learning is a compelling activity in today's global markets. This thesis focuses on critical technology necessary to produce a computer spoken translation game for learning Mandarin Chinese in a relatively broad travel domain. Three main aspects are addressed: efficient Chinese parsing, high-quality English-Chinese machine translation, and how these technologies can be integrated into a translation game system. In the language understanding component, the TINA parser is enhanced with bottom-up and long distance constraint features. The results showed that with these features, the Chinese grammar ran ten times faster and covered 15% more of the test set. In the machine translation component, a combined method of linguistic and statistical system is introduced. The English-Chinese translation is done via an intermediate language "Zhonglish", where the English-Zhonglish translation is accomplished by a parse-and-paraphrase paradigm using hand-coded rules, mainly for structural reconstruction. Zhonglish-Chinese translation is accomplished by a standard phrase based statistical machine translation system, mostly accomplishing word sense disambiguation and lexicon mapping. We evaluated in an independent test set in IWSLT travel domain spoken language corpus. Substantial improvements were achieved for GIZA alignment crossover: we obtained a 45% decrease in crossovers compared to a traditional phrase-based statistical MT system. Furthermore, the BLEU score improved by 2 points. Finally, a framework of the translation game system is described, and the feasibility of integrating the components to produce reference translation and to automatically assess student's translation is verified.by Yushi Xu.S.M

L1 Impacts on L2 Component Reading Skills, Word Skills, and Overall Reading Achievement

Learning to read in a second language as an adult is different in many ways from learning to read in a first language. Unlike children, adult second language (L2) learners have limited knowledge of the target language but may already have fluent reading skills in their first language (L1). These initial reading skills develop to be specifically tuned to the characteristics of the L1 writing system, and may not be optimized for literacy in the L2 (e.g., Frost, 2012; Koda, 2004). This dissertation consists of a program of research designed to examine the impacts that these L1 writing system characteristics have on the development of literacy skills in English as a second language (ESL). Study 1 examined performance on two fundamental literacy skills, phonological awareness and orthographic knowledge, as a function of L1 background and task demands. These data were collected abroad from native French, Hebrew, and Mandarin Chinese speakers, as well as native English speakers, and show clear influences of both L1 orthography and phonology on literacy skill performance. The large differences in performance associated with varying task demands have implications for accurately measuring and understanding students’ underlying abilities. Study 2 examined the contributions of phonological awareness and orthographic knowledge to three measures of word identification: lexical decision, word naming, and pseudoword decoding, as well as global reading comprehension. These data reveal differential performance on the word identification tasks across L1s, as well as differential contributions of phonological awareness and orthographic knowledge to word identification. Study 2 again revealed the effects of task demands on the relationships between sub-lexical literacy skills and word identification. Finally, Study 3 examined the development of language and literacy in adult ESL classroom learners who received either traditional reading instruction or a set of supplemental lessons providing a phonics-based instructional intervention. The results show influences of L1 background as well as different developmental patterns for phonological and orthographic skills based on the type of curriculum students received. The discussion highlights the contributions of this work to understanding cross-linguistic literacy skills and the importance of considering task demands when choosing language assessment measures

Deep Cascade Multi-task Learning for Slot Filling in Online Shopping Assistant

Slot filling is a critical task in natural language understanding (NLU) for dialog systems. State-of-the-art approaches treat it as a sequence labeling problem and adopt such models as BiLSTM-CRF. While these models work relatively well on standard benchmark datasets, they face challenges in the context of E-commerce where the slot labels are more informative and carry richer expressions. In this work, inspired by the unique structure of E-commerce knowledge base, we propose a novel multi-task model with cascade and residual connections, which jointly learns segment tagging, named entity tagging and slot filling. Experiments show the effectiveness of the proposed cascade and residual structures. Our model has a 14.6% advantage in F1 score over the strong baseline methods on a new Chinese E-commerce shopping assistant dataset, while achieving competitive accuracies on a standard dataset. Furthermore, online test deployed on such dominant E-commerce platform shows 130% improvement on accuracy of understanding user utterances. Our model has already gone into production in the E-commerce platform.Comment: AAAI 201

Implications of Autosegmental Analysis in the Exploration of Prosodic Phonology in Mandarin Chinese

Autosegmental Phonology (Goldsmith, 1979) is a theoretical framework for understanding the phonological effects of suprasegmentals such as tone, stress, etc. Using data taken from an experiment in which Mandarin Chinese tone sandhi (the acknowledged rules governing specific tone shifts across segments) is explored, a number of phonologists, specifically Kenstowicz (2003), have shown that the relationship between the segment and the tone is autonomous. In the experiment, non-sense words with a potential tone sandhi rule are presented to the Mandarin speakers. The speakers automatically apply the tone sandhi rule which is then analyzed using an autosegmental framework. The speakers consciously separate the tones from the non-sense words and apply tone sandhi rules; the application of the tone sandhi rule is independent of semantic meaning. This research is expanded to include the exploration of loanword phonology (the phonological changes that occur when a tonal language borrows non-tonal language words) to further understand the autonomous relationship between tones and segments. As can be seen in the following example, the English word Disney: di2-si1-ni2 (numbers account for the differing tones), certain tones are distributed to loanwords