1,272 research outputs found
An analysis of question processing of English and Chinese for the NTCIR 5 cross-language question answering task
An important element in question answering systems is the analysis and interpretation of questions. Using the NTCIR 5 Cross-Language Question Answering (CLQA) question test set we demonstrate that the accuracy of deep question analysis is dependent on the quantity and suitability of the available linguistic resources.
We further demonstrate that applying question analysis tools developed on monolingual training materials to questions translated Chinese-English and English-Chinese using machine translation produces much reduced effectiveness in interpretation of the question. This latter result indicates that question analysis for CLQA should primarily be conducted in the question language prior to translation
That was the last straw, we need more: Are Translation Systems Sensitive to Disambiguating Context?
The translation of ambiguous text presents a challenge for translation
systems, as it requires using the surrounding context to disambiguate the
intended meaning as much as possible. While prior work has studied ambiguities
that result from different grammatical features of the source and target
language, we study semantic ambiguities that exist in the source (English in
this work) itself. In particular, we focus on idioms that are open to both
literal and figurative interpretations (e.g., goose egg), and collect TIDE, a
dataset of 512 pairs of English sentences containing idioms with disambiguating
context such that one is literal (it laid a goose egg) and another is
figurative (they scored a goose egg, as in a score of zero). In experiments, we
compare MT-specific models and language models for (i) their preference when
given an ambiguous subsentence, (ii) their sensitivity to disambiguating
context, and (iii) the performance disparity between figurative and literal
source sentences. We find that current MT models consistently translate English
idioms literally, even when the context suggests a figurative interpretation.
On the other hand, LMs are far more context-aware, although there remain
disparities across target languages. Our findings underline the potential of
LMs as a strong backbone for context-aware translation.Comment: EMNLP 2023 Finding
Are ambiguous conjunctions problematic for machine translation?
The translation of ambiguous words still poses challenges for machine translation.
In this work, we carry out a systematic quantitative analysis regarding the ability of different machine translation systems to disambiguate the source language conjunctions âbutâ and âandâ. We evaluate specialised test sets focused on the translation of these two conjunctions. The test sets contain source languages that do not distinguish different variants of the given conjunction, whereas the target languages do. In total, we evaluate the conjunction âbutâ on 20 translation outputs, and the conjunction âandâ on 10. All machine translation systems almost perfectly recognise one variant of the target conjunction, especially for the source conjunction
âbutâ. The other target variant, however, represents a challenge for machine translation systems, with accuracy varying from 50% to 95% for âbutâ and from 20% to 57% for âandâ. The major error for all systems is replacing the correct target variant with the opposite one
The impact of source-side syntactic reordering on hierarchical phrase-based SMT
Syntactic reordering has been demonstrated
to be helpful and effective for handling
different word orders between source
and target languages in SMT. However, in
terms of hierarchial PB-SMT (HPB), does
the syntactic reordering still has a significant
impact on its performance? This
paper introduces a reordering approach
which explores the { (DE) grammatical
structure in Chinese. We employ
the Stanford DE classifier to recognise
the DE structures in both training and
test sentences of Chinese, and then perform
word reordering to make the Chinese
sentences better match the word order
of English. The annotated and reordered
training data and test data are applied
to a re-implemented HPB system and
the impact of the DE construction is examined.
The experiments are conducted
on the NIST 2008 evaluation data and experimental
results show that the BLEU
and METEOR scores are significantly improved
by 1.83/8.91 and 1.17/2.73 absolute/
relative points respectively
A discriminative latent variable-based "DE" classifier for ChineseâEnglish SMT
Syntactic reordering on the source-side
is an effective way of handling word order
differences. The (DE) construction
is a flexible and ubiquitous syntactic
structure in Chinese which is a major
source of error in translation quality.
In this paper, we propose a new classifier
model â discriminative latent variable
model (DPLVM) â to classify the
DE construction to improve the accuracy
of the classification and hence the translation
quality. We also propose a new feature
which can automatically learn the reordering
rules to a certain extent. The experimental
results show that the MT systems
using the data reordered by our proposed
model outperform the baseline systems
by 6.42% and 3.08% relative points
in terms of the BLEU score on PB-SMT
and hierarchical phrase-based MT respectively.
In addition, we analyse the impact
of DE annotation on word alignment and
on the SMT phrase table
Source-side syntactic reordering patterns with functional words for improved phrase-based SMT
Inspired by previous source-side syntactic reordering methods for SMT, this paper focuses on using automatically learned syntactic reordering patterns with functional words which indicate structural reorderings between the source and target language. This approach takes advantage of phrase alignments and source-side parse trees for pattern extraction, and then filters out those patterns without functional words. Word lattices transformed by the generated patterns are fed into PBSMT systems to incorporate potential reorderings from the inputs. Experiments are carried out on a medium-sized corpus for a ChineseâEnglish SMT task. The proposed method outperforms the baseline system by 1.38% relative on a randomly selected testset and 10.45% relative on the NIST 2008 testset in terms of BLEU score. Furthermore, a system with just 61.88% of the patterns filtered by functional words obtains a comparable performance with the unfiltered one on the randomly selected testset, and achieves 1.74% relative improvements on the NIST 2008 testset
Improved phrase-based SMT with syntactic reordering patterns learned from lattice scoring
In this paper, we present a novel approach to incorporate source-side syntactic reordering patterns into phrase-based SMT. The main contribution of this work is to use the lattice scoring approach to exploit and utilize reordering
information that is favoured by the baseline PBSMT system. By referring to the parse trees of the training corpus, we represent the observed reorderings with source-side
syntactic patterns. The extracted patterns are then used to convert the parsed inputs into word lattices, which contain both the original source sentences and their potential reorderings. Weights of the word lattices are estimated from the observations of the syntactic reordering patterns in the training corpus. Finally, the PBSMT system is tuned
and tested on the generated word lattices to show the benefits of adding potential sourceside reorderings in the inputs. We confirmed the effectiveness of our proposed method on a medium-sized corpus for Chinese-English
machine translation task. Our method outperformed the baseline system by 1.67% relative on a randomly selected testset and 8.56% relative on the NIST 2008 testset in terms of BLEU score
Using TERp to augment the system combination for SMT
TER-Plus (TERp) is an extended TER evaluation metric incorporating morphology, synonymy and paraphrases.
There are three new edit operations in TERp: Stem Matches, Synonym Matches and Phrase Substitutions (Para-phrases). In this paper, we propose a TERp-based augmented system combination in terms of the backbone selection and consensus decoding network. Combining the new properties\ud
of the TERp, we also propose a two-pass decoding strategy for the lattice-based phrase-level confusion network(CN) to generate the final result. The experiments conducted on the NIST2008 Chinese-to-English test set show that our TERp-based augmented system combination framework achieves significant improvements in terms of BLEU and TERp scores compared to the state-of-the-art word-level system combination framework and a TER-based combination strategy
- âŚ