25 research outputs found
Linguistic evaluation of German-English Machine Translation using a Test Suite
We present the results of the application of a grammatical test suite for
GermanEnglish MT on the systems submitted at WMT19, with a
detailed analysis for 107 phenomena organized in 14 categories. The systems
still translate wrong one out of four test items in average. Low performance is
indicated for idioms, modals, pseudo-clefts, multi-word expressions and verb
valency. When compared to last year, there has been a improvement of function
words, non-verbal agreement and punctuation. More detailed conclusions about
particular systems and phenomena are also presented
Machine Translation: Phrase-Based, Rule-Based and Neural Approaches with Linguistic Evaluation
AbstractIn this article we present a novel linguistically driven evaluation method and apply it to the main approaches of Machine Translation (Rule-based, Phrase-based, Neural) to gain insights into their strengths and weaknesses in much more detail than provided by current evaluation schemes. Translating between two languages requires substantial modelling of knowledge about the two languages, about translation, and about the world. Using English-German IT-domain translation as a case-study, we also enhance the Phrase-based system by exploiting parallel treebanks for syntax-aware phrase extraction and by interfacing with Linked Open Data (LOD) for extracting named entity translations in a post decoding framework.</jats:p
CoNLL 2017 Shared Task : Multilingual Parsing from Raw Text to Universal Dependencies
The Conference on Computational Natural Language Learning (CoNLL) features a shared task, in which participants train and test their learning systems on the same data sets. In 2017, one of two tasks was devoted to learning dependency parsers for a large number of languages, in a real world setting without any gold-standard annotation on input. All test sets followed a unified annotation scheme, namely that of Universal Dependencies. In this paper, we define the task and evaluation methodology, describe data preparation, report and analyze the main results, and provide a brief categorization of the different approaches of the participating systems.Peer reviewe
Relatório de estágio em farmácia comunitária
Relatório de estágio realizado no âmbito do Mestrado Integrado em Ciências Farmacêuticas, apresentado à Faculdade de Farmácia da Universidade de Coimbr
Deeper Machine Translation and Evaluation for German
This paper describes a hybrid Machine Translation (MT) system built for translating from English
to German in the domain of technical documentation. The system is based on three different
MT engines (phrase-based SMT, RBMT, neural) that are joined by a selection mechanism
that uses deep linguistic features within a machine learning process. It also presents a detailed
source-driven manual error analysis we have performed using a dedicated “test suite” that contains
selected examples of relevant phenomena. While automatic scores show huge differences
between the engines, the overall average number or errors they (do not) make is very similar for
all systems. However, the detailed error breakdown shows that the systems behave very differently
concerning the various phenomena