Machine translation evaluation resources and methods: a survey

We introduce the Machine Translation (MT) evaluation survey that contains both manual and automatic evaluation methods. The traditional human evaluation criteria mainly include the intelligibility, fidelity, fluency, adequacy, comprehension, and informativeness. The advanced human assessments include task-oriented measures, post-editing, segment ranking, and extended criteriea, etc. We classify the automatic evaluation methods into two categories, including lexical similarity scenario and linguistic features application. The lexical similarity methods contain edit distance, precision, recall, F-measure, and word order. The linguistic features can be divided into syntactic features and semantic features respectively. The syntactic features include part of speech tag, phrase types and sentence structures, and the semantic features include named entity, synonyms, textual entailment, paraphrase, semantic roles, and language models. The deep learning models for evaluation are very newly proposed. Subsequently, we also introduce the evaluation methods for MT evaluation including different correlation scores, and the recent quality estimation (QE) tasks for MT. This paper differs from the existing works\cite {GALEprogram2009, EuroMatrixProject2007} from several aspects, by introducing some recent development of MT evaluation measures, the different classifications from manual to automatic evaluation measures, the introduction of recent QE tasks of MT, and the concise construction of the content

Improving the translation environment for professional translators

When using computer-aided translation systems in a typical, professional translation workflow, there are several stages at which there is room for improvement. The SCATE (Smart Computer-Aided Translation Environment) project investigated several of these aspects, both from a human-computer interaction point of view, as well as from a purely technological side. This paper describes the SCATE research with respect to improved fuzzy matching, parallel treebanks, the integration of translation memories with machine translation, quality estimation, terminology extraction from comparable texts, the use of speech recognition in the translation process, and human computer interaction and interface design for the professional translation environment. For each of these topics, we describe the experiments we performed and the conclusions drawn, providing an overview of the highlights of the entire SCATE project

Geography, culture, and religion: Explaining the bias in Eurovision song contest voting

This paper analyses votes cast in the Eurovision Song Contest in the period 1975 - 2003. We test whether accusations of 'political' voting among participants can be substantiated by looking at geographical influences. Our approach differs in two ways from earlier studies. First, we take into account a variety of variables to distinguish political voting from preferences based on cultural, linguistic, ethnic, and religious differences and similarities between countries. Secondly, we analyse the determinants of the voting behaviour separately per country, instead of looking at average effects over all participating countries. We find that geographical factors substantially affect the votes. Even after correction for cultural, linguistic and other factors many countries prefer or dislike the songs of surrounding countries. This leads to the suspicion that the geograph¬ical preferences reflect political voting. Also, we show that several countries favour songs of participants with the same religious background, while others prefer the contributions of countries with a different religion. Moreover, using data on the amount of Turkish immigrants across European countries, we document that countries with a substantial Turkish population favour the Turkish songs ('patriotic' voting). Furthermore, we study the repercussions of opening up the voting system to the general public by the introduction of televoting. It turns out that religious and patriotic voting have become considerably stronger since the introduction of the new voting system. Finally, we confront our em¬pirical findings to the publicly debated accusations of political voting made against certain blocks of countries. Although our analysis uncovers significant geographical patterns (suggesting political voting), we do hardly establish any empirical evidence for the claims against these particular countries