1,414 research outputs found
A Challenge Set Approach to Evaluating Machine Translation
Neural machine translation represents an exciting leap forward in translation
quality. But what longstanding weaknesses does it resolve, and which remain? We
address these questions with a challenge set approach to translation evaluation
and error analysis. A challenge set consists of a small set of sentences, each
hand-designed to probe a system's capacity to bridge a particular structural
divergence between languages. To exemplify this approach, we present an
English-French challenge set, and use it to analyze phrase-based and neural
systems. The resulting analysis provides not only a more fine-grained picture
of the strengths of neural systems, but also insight into which linguistic
phenomena remain out of reach.Comment: EMNLP 2017. 28 pages, including appendix. Machine readable data
included in a separate file. This version corrects typos in the challenge se
Transitive probabilistic CLIR models.
Transitive translation could be a useful technique to enlarge the number of supported language pairs for a cross-language information retrieval (CLIR) system in a cost-effective manner. The paper describes several setups for transitive translation based on probabilistic translation models. The transitive CLIR models were evaluated on the CLEF test collection and yielded a retrieval effectiveness\ud
up to 83% of monolingual performance, which is significantly better than a baseline using the synonym operator
Automatic domain ontology extraction for context-sensitive opinion mining
Automated analysis of the sentiments presented in online consumer feedbacks can facilitate both organizations’ business strategy development and individual consumers’ comparison shopping. Nevertheless, existing opinion mining methods either adopt a context-free sentiment classification approach or rely on a large number of manually annotated training examples to perform context sensitive sentiment classification. Guided by the design science research methodology, we illustrate the design, development, and evaluation of a novel fuzzy domain ontology based contextsensitive opinion mining system. Our novel ontology extraction mechanism underpinned by a variant of Kullback-Leibler divergence can automatically acquire contextual sentiment knowledge across various product domains to improve the sentiment analysis processes. Evaluated based on a benchmark dataset and real consumer reviews collected from Amazon.com, our system shows remarkable performance improvement over the context-free baseline
Information Retrieval: Recent Advances and Beyond
In this paper, we provide a detailed overview of the models used for
information retrieval in the first and second stages of the typical processing
chain. We discuss the current state-of-the-art models, including methods based
on terms, semantic retrieval, and neural. Additionally, we delve into the key
topics related to the learning process of these models. This way, this survey
offers a comprehensive understanding of the field and is of interest for for
researchers and practitioners entering/working in the information retrieval
domain
Latent-Variable PCFGs: Background and Applications
Latent-variable probabilistic context-free grammars are
latent-variable models that are based on context-free grammars.
Nonterminals are associated with latent states that provide
contextual information during the top-down rewriting process of
the grammar.
We survey a few of the techniques used to estimate such grammars
and to parse text with them. We also give an overview of what the latent
states represent for English Penn treebank parsing, and provide
an overview of extensions and related models to these grammars
Probabilistic Inference for Phrase-based Machine Translation: A Sampling Approach
Recent advances in statistical machine translation (SMT) have used dynamic programming
(DP) based beam search methods for approximate inference within probabilistic
translation models. Despite their success, these methods compromise the probabilistic
interpretation of the underlying model thus limiting the application of probabilistically
defined decision rules during training and decoding.
As an alternative, in this thesis, we propose a novel Monte Carlo sampling approach
for theoretically sound approximate probabilistic inference within these models. The
distribution we are interested in is the conditional distribution of a log-linear translation
model; however, often, there is no tractable way of computing the normalisation term
of the model. Instead, a Gibbs sampling approach for phrase-based machine translation
models is developed which obviates the need of computing this term yet produces
samples from the required distribution.
We establish that the sampler effectively explores the distribution defined by a
phrase-based models by showing that it converges in a reasonable amount of time to
the desired distribution, irrespective of initialisation. Empirical evidence is provided to
confirm that the sampler can provide accurate estimates of expectations of functions of
interest. The mix of high probability and low probability derivations obtained through
sampling is shown to provide a more accurate estimate of expectations than merely
using the n-most highly probable derivations.
Subsequently, we show that the sampler provides a tractable solution for finding the
maximum probability translation in the model. We also present a unified approach to
approximating two additional intractable problems: minimum risk training and minimum
Bayes risk decoding. Key to our approach is the use of the sampler which
allows us to explore the entire probability distribution and maintain a strict probabilistic
formulation through the translation pipeline. For these tasks, sampling allies
the simplicity of n-best list approaches with the extended view of the distribution that
lattice-based approaches benefit from, while avoiding the biases associated with beam
search. Our approach is theoretically well-motivated and can give better and more
stable results than current state of the art methods
- …