Towards predicting post-editing productivity

Machine translation (MT) quality is generally measured via automatic metrics, producing scores that have no meaning for translators who are required to post-edit MT output or for project managers who have to plan and budget for transla- tion projects. This paper investigates correlations between two such automatic metrics (general text matcher and translation edit rate) and post-editing productivity. For the purposes of this paper, productivity is measured via processing speed and cognitive measures of effort using eye tracking as a tool. Processing speed, average fixation time and count are found to correlate well with the scores for groups of segments. Segments with high GTM and TER scores require substantially less time and cognitive effort than medium or low-scoring segments. Future research involving score thresholds and confidence estimation is suggested

Quantifying the effect of machine translation in a high-quality human translation production process

This paper studies the impact of machine translation (MT) on the translation workflow at the Directorate-General for Translation (DGT), focusing on two language pairs and two MT paradigms: English-into-French with statistical MT and English-into-Finnish with neural MT. We collected data from 20 professional translators at DGT while they carried out real translation tasks in normal working conditions. The participants enabled/disabled MT for half of the segments in each document. They filled in a survey at the end of the logging period. We measured the productivity gains (or losses) resulting from the use of MT and examined the relationship between technical effort and temporal effort. The results show that while the usage of MT leads to productivity gains on average, this is not the case for all translators. Moreover, the two technical effort indicators used in this study show weak correlations with post-editing time. The translators' perception of their speed gains was more or less in line with the actual results. Reduction of typing effort is the most frequently mentioned reason why participants preferred working with MT, but also the psychological benefits of not having to start from scratch were often mentioned

Eye-tracking as a measure of cognitive effort for post-editing of machine translation

The three measurements for post-editing effort as proposed by Krings (2001) have been adopted by many researchers in subsequent studies and publications. These measurements comprise temporal effort (the speed or productivity rate of post-editing, often measured in words per second or per minute at the segment level), technical effort (the number of actual edits performed by the post-editor, sometimes approximated using the Translation Edit Rate metric (Snover et al. 2006), again usually at the segment level), and cognitive effort. Cognitive effort has been measured using Think-Aloud Protocols, pause measurement, and, increasingly, eye-tracking. This chapter provides a review of studies of post-editing effort using eye-tracking, noting the influence of publications by Danks et al. (1997), and O’Brien (2006, 2008), before describing a single study in detail. The detailed study examines whether predicted effort indicators affect post-editing effort and results were previously published as Moorkens et al. (2015). Most of the eye-tracking data analysed were unused in the previou

Pre-editing and post-editing

This chapter provides an accessible introductory view of pre-editing and post-editing as the starting-point for research or work in the language industry. It describes source text pre-editing and machine translation post-editing from an industrial as well as academic point of view. In the last ten to fifteen years, there has been a considerable growth in the number of studies and publications dealing with pre-editing, and especially post-editing, that have helped researchers and the industry to understand the impact machine translation technology has on translators’ output and their working environment. This interest is likely to continue in view of the recent developments in neural machine translation and artificial intelligence. Although the latest technology has taken a considerable leap forward, the existing body of work should not be disregarded as it has defined clear research lines and methods, as it is more necessary than ever to look at data in their appropriate context and avoid generalizing in the vast and diverse territory of human and machine translation

Pre-editing and post-editing

This chapter provides an accessible introductory view of pre-editing and post-editing as the starting-point for research or work in the language industry. It describes source text pre-editing and machine translation post-editing from an industrial as well as academic point of view. In the last ten to fifteen years, there has been a considerable growth in the number of studies and publications dealing with pre-editing, and especially post-editing, that have helped researchers and the industry to understand the impact machine translation technology has on translators’ output and their working environment. This interest is likely to continue in view of the recent developments in neural machine translation and artificial intelligence. Although the latest technology has taken a considerable leap forward, the existing body of work should not be disregarded as it has defined clear research lines and methods, as it is more necessary than ever to look at data in their appropriate context and avoid generalizing in the vast and diverse territory of human and machine translation

What Do Professional Translators Do when Post-Editing for the First Time? First Insight into the Spanish-Basque Language Pair

Machine translation post-editing is becoming commonplace and professional translators are often faced with this unknown task with little training and support. Given the different translation processes involved during post-editing, research suggests that untrained translators do not necessarily make good post-editors. Besides, the post-editing activity will be largely influenced by numerous aspects related to the technology and texts used. Training material, therefore, will need to be tailored to the particular conditions under which post-editing is bound to happen. In this work, we provide a first attempt to uncover what activity professional translators carry out when working from Spanish into Basque. Our initial analysis reveals that when working with moderate machine translation output post-editing shifts from the task of identifying and fixing errors, to that of “patchwork” where post-editors identify the machine translated elements to reuse and connect them using their own contributions. Data also reveal that they primarily focus on correcting machine translation errors but often fail to restrain themselves from editing correct structures. Both findings have clear implications for training and are a step forward in tailoring sessions specifically for language combinations of moderate quality

Machine translation and Welsh: analysing free statistical machine translation for the professional translation of an under-researched language pair

This article reports on a key-logging study carried out to test the benefits of post-editing Machine Translation (MT) for the professional translator within a hypothetico-deductive framework, contrasting the outcomes of a number of variables which are inextricably linked to the professional translation process. Given the current trend of allowing the professional translator to connect to Google Translate services within the main Translation Memory (TM) systems via an API, a between-groups design is utilized in which cognitive, technical and temporal effort are gauged between translation and post-editing the statistical MT engine Google Translate. The language pair investigated is English and Welsh. Results show no statistical difference between post-editing and translation in terms of processing time. Using a novel measure of cognitive effort focused on pauses, the cognitive effort exerted by post-editors and translators was also found to be statistically different. Results also show however that a complex relationship exists between post-editing, translation and technical effort, in that aspects of text production processes were seen to be eased by post-editing. Finally, a bilingual review by two different translators found little difference in quality between the translated and post-edited texts, and that both sets of texts were acceptable according to accuracy and fidelity

The role of professional experience in post-editing from a quality and productivity perspective

In this chapter, we present results on the impact of professional experience on the task of post-editing. These results are part of a larger research project where 24 translators and three reviewers were tested to obtain productivity, words per minute, and quality data, errors in final target texts, in the post-editing of machine translation (MT) and fuzzy match segments (in the 85 to 94 range). We will discuss here the results on the participants’ experience according to their responses in a post-assignment questionnaire and explain how they were grouped into different clusters in order to correlate firstly the experience with speed according to the words per minute in the different match categories: Fuzzy matches, MT matches (MT output) and No match and secondly, to correlate them with the quality provided by measuring the errors marked by the three reviewers in each match category. Finally, conclusions will be drawn in relation to the experience and the resulting speed and number of errors

Post-editing for Professional Translators : Cheer or Fear?

Currently, post-editing of machine translation (MT) has been introduced as a regular practice in the translation workflow, especially since the good results in quality obtained by neural MT (NMT). This fact is linked to the efforts LSPs and customers have done to reduce costs due to the recent global crisis and the increasing globalization, which has had a negative impact on translators' revenues and on their working practices. In this context, post-editing is often perceived with a negative bias by translators. We study attitudes of translators post-editing for the first time and relate them to their productivity rates. We also compare the results with a survey answered by professional post-editors assessing their perception of the task in the current marketplace.Actualmente, la posedición de traducción automática (TA) se considera una práctica habitual en el flujo de trabajo de traducción, sobre todo por la buena calidad que se obtiene con la traducción automática neuronal (TAN). Este hecho está asociado a los esfuerzos que han hecho los proveedores de servicios lingüísticos y los clientes para reducir los costos debido a la reciente crisis mundial y a la creciente globalización, que ha tenido un impacto negativo en los ingresos de los traductores y en sus prácticas profesionales. En este contexto, los traductores suelen percibir la posedición con un sesgo negativo. En este artículo se presenta uno de los primeros estudios estudio sobre las actitudes de los traductores ante la posedición y se relacionan con sus tasas de productividad. También cotejamos los resultados con una encuesta contestada por poseditores profesionales que evalúan su percepción de la tarea en el mercado actual.Actualment, la postedició de traducció automàtica (TA) és considerada una pràctica habitual en el flux de treball de la traducció, sobretot per la bona qualitat que s'obté amb la traducció automàtica neuronal (TAN). Aquest fet està assocat als esforços que han fet els proveïdors de serveis lingüístics i els clients per reduir els costos a causa de la crisi mundial dels darrers temps i la creixent globalització, que ha tingut un impacte negatiu sobre els ingressos dels traductors i sobre les seves pràctiques professionals. En aquest cotext, els traductors acostumen a percebre la postedició amb un biaix negatiu. En aquest article es presenta un dels primers estudis sobre les actituds dels traductors envers la postedició i es relacionen amb les seves taxes de productivitat. També acarem els resultats amb una enquesta contestada per posteditors professionals que avaluen la seva percepció de la tasca en el mercat actual

The impact of machine translation error types on post-editing effort indicators

In this paper, we report on a post-editing study for general text types from English into Dutch conducted with master's students of translation. We used a fine-grained machine translation (MT) quality assessment method with error weights that correspond to severity levels and are related to cognitive load. Linear mixed effects models are applied to analyze the impact of MT quality on potential post-editing effort indicators. The impact of MT quality is evaluated on three different levels, each with an increasing granularity. We find that MT quality is a significant predictor of all different types of post-editing effort indicators and that different types of MT errors predict different post-editing effort indicators