62 research outputs found
Findings of the 2019 Conference on Machine Translation (WMT19)
This paper presents the results of the premier shared task organized alongside the Conference on Machine Translation (WMT) 2019.
Participants were asked to build machine translation systems for any of 18 language pairs, to be evaluated on a test set of news stories. The main metric for this task is human judgment of translation quality. The task was also opened up to additional test suites to probe specific aspects of translation
What is SemEval evaluating?: A Systematic Analysis of Evaluation Campaigns in NLP
SemEval is the primary venue in the NLP community for the proposal of new
challenges and for the systematic empirical evaluation of NLP systems. This
paper provides a systematic quantitative analysis of SemEval aiming to evidence
the patterns of the contributions behind SemEval. By understanding the
distribution of task types, metrics, architectures, participation and citations
over time we aim to answer the question on what is being evaluated by SemEval.Comment: 12 pages, 6 figure
- …