10 research outputs found
Overview of the IWSLT 2012 Evaluation Campaign
open5siWe report on the ninth evaluation campaign organized by
the IWSLT workshop. This year, the evaluation offered multiple
tracks on lecture translation based on the TED corpus, and
one track on dialog translation from Chinese
to English based on the Olympic trilingual corpus.
In particular, the TED tracks included a speech transcription
track in English, a speech translation track from English to French,
and text translation tracks from English to French and from Arabic
to English. In addition to the official tracks, ten unofficial
MT tracks were offered that required translating TED talks into English
from either Chinese, Dutch, German, Polish, Portuguese (Brazilian), Romanian, Russian, Slovak,
Slovene, or Turkish.
16 teams participated in the evaluation and submitted a total of 48 primary runs.
All runs were evaluated with objective metrics, while runs of the official translation
tracks were also ranked by crowd-sourced judges.
In particular, subjective ranking for the TED task was performed on a progress test which permitted
direct comparison of the results from this year against the best results from the 2011 round of the evaluation campaign.Marcello Federico; Mauro Cettolo; Luisa Bentivogli; Michael Paul; Sebastian StükerFederico, Marcello; Cettolo, Mauro; Bentivogli, Luisa; Michael, Paul; Sebastian, Stüke
Findings of the iWSLT 2023 evaluation campaign
This paper reports on the shared tasks organized by the 20th IWSLT Conference. The shared tasks address 9 scientific challenges in spoken language translation: simultaneous and offline translation, automatic subtitling and dubbing, speech-to-speech translation, multilingual, dialect and low-resource speech translation, and formality control. The shared tasks attracted a total of 38 submissions by 31 teams. The growing interest towards spoken language translation is also witnessed by the constantly increasing number of shared task organizers and contributors to the overview paper, almost evenly distributed across industry and academia.peer-reviewe
The IWSLT 2015 Evaluation Campaign
The IWSLT 2015 Evaluation Campaign featured three tracks: automatic speech recognition (ASR), spoken language translation (SLT), and machine translation (MT). For ASR we offered two tasks, on English and German, while for SLT and MT a number of tasks were proposed, involving English, German, French, Chinese, Czech, Thai, and Vietnamese. All tracks involved the transcription or translation of TED talks, either made available by the official TED website or by other TEDx events.
A notable change with respect to previous evaluations was the use of unsegmented speech in the SLT track in order to better fit a real application scenario. Thus, from one side participants were encouraged to develop advanced methods for sentence segmentation, from the other side organisers had to cope with the automatic evaluation of SLT outputs not matching the sentence-wise arrangement of the human references. A new evaluation server was also developed to allow participants to score their MT and SLT systems on selected dev and test sets. This year 16 teams participated in the evaluation, for a total of 63 primary submissions. All runs were evaluated with objective metrics, and submissions for two of the MT translation tracks were also evaluated with human post-editing
Findings of the IWSLT 2022 Evaluation Campaign.
The evaluation campaign of the 19th International Conference on Spoken Language Translation featured eight shared tasks: (i) Simultaneous speech translation, (ii) Offline speech translation, (iii) Speech to speech translation, (iv) Low-resource speech translation, (v) Multilingual speech translation, (vi) Dialect speech translation, (vii) Formality control for speech translation, (viii) Isometric speech translation. A total of 27 teams participated in at least one of the shared tasks. This paper details, for each shared task, the purpose of the task, the data that were released, the evaluation metrics that were applied, the submissions that were received and the results that were achieved
Overview of the IWSLT 2017 Evaluation Campaign
The IWSLT 2017 evaluation campaign has organised
three tasks. The Multilingual task, which
is about training machine translation systems
handling many-to-many language directions, including
so-called zero-shot directions. The Dialogue
task, which calls for the integration of
context information in machine translation, in order
to resolve anaphoric references that typically
occur in human-human dialogue turns. And, finally,
the Lecture task, which offers the challenge
of automatically transcribing and translating
real-life university lectures. Following the
tradition of these reports, we will described all
tasks in detail and present the results of all runs
submitted by their participants