Universal Dependencies Parsing for Colloquial Singaporean English

Singlish can be interesting to the ACL community both linguistically as a major creole based on English, and computationally for information extraction and sentiment analysis of regional social media. We investigate dependency parsing of Singlish by constructing a dependency treebank under the Universal Dependencies scheme, and then training a neural network model by integrating English syntactic knowledge into a state-of-the-art parser trained on the Singlish treebank. Results show that English knowledge can lead to 25% relative error reduction, resulting in a parser of 84.47% accuracies. To the best of our knowledge, we are the first to use neural stacking to improve cross-lingual dependency parsing on low-resource languages. We make both our annotation and parser available for further research.Comment: Accepted by ACL 201

Effective Approaches to Attention-based Neural Machine Translation

An attentional mechanism has lately been used to improve neural machine translation (NMT) by selectively focusing on parts of the source sentence during translation. However, there has been little work exploring useful architectures for attention-based NMT. This paper examines two simple and effective classes of attentional mechanism: a global approach which always attends to all source words and a local one that only looks at a subset of source words at a time. We demonstrate the effectiveness of both approaches over the WMT translation tasks between English and German in both directions. With local attention, we achieve a significant gain of 5.0 BLEU points over non-attentional systems which already incorporate known techniques such as dropout. Our ensemble model using different attention architectures has established a new state-of-the-art result in the WMT'15 English to German translation task with 25.9 BLEU points, an improvement of 1.0 BLEU points over the existing best system backed by NMT and an n-gram reranker.Comment: 11 pages, 7 figures, EMNLP 2015 camera-ready version, more training detail

Handling Homographs in Neural Machine Translation

Homographs, words with different meanings but the same surface form, have long caused difficulty for machine translation systems, as it is difficult to select the correct translation based on the context. However, with the advent of neural machine translation (NMT) systems, which can theoretically take into account global sentential context, one may hypothesize that this problem has been alleviated. In this paper, we first provide empirical evidence that existing NMT systems in fact still have significant problems in properly translating ambiguous words. We then proceed to describe methods, inspired by the word sense disambiguation literature, that model the context of the input word with context-aware word embeddings that help to differentiate the word sense be- fore feeding it into the encoder. Experiments on three language pairs demonstrate that such models improve the performance of NMT systems both in terms of BLEU score and in the accuracy of translating homographs.Comment: NAACL201

Diffraction Patterns of Layered Close-packed Structures from Hidden Markov Models

We recently derived analytical expressions for the pairwise (auto)correlation functions (CFs) between modular layers (MLs) in close-packed structures (CPSs) for the wide class of stacking processes describable as hidden Markov models (HMMs) [Riechers \etal, (2014), Acta Crystallogr.~A, XX 000-000]. We now use these results to calculate diffraction patterns (DPs) directly from HMMs, discovering that the relationship between the HMMs and DPs is both simple and fundamental in nature. We show that in the limit of large crystals, the DP is a function of parameters that specify the HMM. We give three elementary but important examples that demonstrate this result, deriving expressions for the DP of CPSs stacked (i) independently, (ii) as infinite-Markov-order randomly faulted 2H and 3C stacking structures over the entire range of growth and deformation faulting probabilities, and (iii) as a HMM that models Shockley-Frank stacking faults in 6H-SiC. While applied here to planar faulting in CPSs, extending the methods and results to planar disorder in other layered materials is straightforward. In this way, we effectively solve the broad problem of calculating a DP---either analytically or numerically---for any stacking structure---ordered or disordered---where the stacking process can be expressed as a HMM.Comment: 18 pages, 6 figures, 3 tables; http://csc.ucdavis.edu/~cmg/compmech/pubs/dplcps.ht