64 research outputs found
BattRAE: Bidimensional Attention-Based Recursive Autoencoders for Learning Bilingual Phrase Embeddings
In this paper, we propose a bidimensional attention based recursive
autoencoder (BattRAE) to integrate clues and sourcetarget interactions at
multiple levels of granularity into bilingual phrase representations. We employ
recursive autoencoders to generate tree structures of phrases with embeddings
at different levels of granularity (e.g., words, sub-phrases and phrases). Over
these embeddings on the source and target side, we introduce a bidimensional
attention network to learn their interactions encoded in a bidimensional
attention matrix, from which we extract two soft attention weight distributions
simultaneously. These weight distributions enable BattRAE to generate
compositive phrase representations via convolution. Based on the learned phrase
representations, we further use a bilinear neural model, trained via a
max-margin method, to measure bilingual semantic similarity. To evaluate the
effectiveness of BattRAE, we incorporate this semantic similarity as an
additional feature into a state-of-the-art SMT system. Extensive experiments on
NIST Chinese-English test sets show that our model achieves a substantial
improvement of up to 1.63 BLEU points on average over the baseline.Comment: 7 pages, accepted by AAAI 201
Dependency-based Bilingual Word Embeddings and Neural Machine Translation
Bilingual word embeddings, which represent lexicons from various languages in a
common embedding space, are critical for facilitating semantic and knowledge trans-
fers in a wide range of cross-lingual NLP applications. The significance of learning
bilingual word embedding representations in many Natural Language Processing
(NLP) tasks motivates us to investigate the effect of many factors, including syntac-
tical information, on the learning process for different languages with varying levels
of structural complexity. By analysing the components that influence the learning
process of bilingual word embeddings (BWEs), this thesis examines some factors for
learning bilingual word embeddings effectively. Our findings in this thesis demon-
strate that increasing the embedding size for language pairs has a positive impact
on the learning process for BWEs. While sentence length depends on the language.
Short sentences perform better than long ones in the En-ES experiment. However,
by increasing the sentence, En-Ar and En-De experiment achieve improved model
accuracy. Arabic segmentation, according to En-Ar experiments, is essential to the
learning process for BWEs and can boost model accuracy by up to 10%.
Incorporating dependency features into the learning process enhances the trained
models performance and results in more improved BWEs in all language pairs.
Finally, we investigated how the dependancy-based pretrained BWEs affected the
neural machine translation (NMT) model. The findings indicate that in various
MT evaluation matrices, the trained dependancy-based NMT models outperform
the baseline NMT model
- …