Neural Models for Measuring Confidence on Interactive Machine Translation Systems

[EN] Reducing the human effort performed with the use of interactive-predictive neural machine translation (IPNMT) systems is one of the main goals in this sub-field of machine translation (MT). Prior works have focused on changing the human¿machine interaction method and simplifying the feedback performed. Applying confidence measures (CM) to an IPNMT system helps decrease the number of words that the user has to check through the translation session, reducing the human effort needed, although this supposes losing a few points in the quality of the translations. The effort reduction comes from decreasing the number of words that the translator has to review¿it only has to check the ones with a score lower than the threshold set. In this paper, we studied the performance of four confidence measures based on the most used metrics on MT. We trained four recurrent neural network (RNN) models to approximate the scores from the metrics: Bleu, Meteor, Chr-f, and TER. In the experiments, we simulated the user interaction with the system to obtain and compare the quality of the translations generated with the effort reduction. We also compare the performance of the four models between them to see which of them obtains the best results. The results achieved showed a reduction of 48% with a Bleu score of 70 points¿a significant effort reduction to translations almost perfect.This work received funds from the Comunitat Valenciana under project EU-FEDER (ID-IFEDER/2018/025), Generalitat Valenciana under project ALMAMATER (PrometeoII/2014/030), and Ministerio de Ciencia e Investigacion/Agencia Estatal de Investigacion/10.13039/501100011033/and "FEDER Una manera de hacer Europa" under project MIRANDA-DocTIUM (RTI2018-095645-B-C22).Navarro-Martínez, Á.; Casacuberta Nolla, F. (2022). Neural Models for Measuring Confidence on Interactive Machine Translation Systems. Applied Sciences. 12(3):1-16. https://doi.org/10.3390/app1203110011612

MOTYWACJA I MEDIA ELEKTRONICZNE W NAUCZANIU TŁUMACZEŃ SPECJALISTYCZNYCH

Expansion of IT-media in every field of human activity is one of the essential characteristics of modern time. This paper aims at presenting the role of electronic media in teaching translation in the field of law at the University of Osijek, Croatia, and analysing their impact on the motivation of the target group of students in the teaching process. The paper endeavours to provide some insight into the modern teaching practice and to analyse the interconnectedness of the use of electronic media and student motivation rather than to present some empirical research in the field. In the first part of the paper, a theoretical approach to teaching legal translation today is offered. In the main part, teaching legal translation by using modern media is presented on the examples of the Lifelong Learning Programme for Lawyer-Linguists at the Faculty of Law Osijek, and the course on legal translation within the German Language and Literature Studies at the Faculty of Humanities and Social Sciences of Osijek. The usage of electronic media in translation teaching is discussed with reference to the courses Introduction to the Theory of Legal Translation and Online Translation Tools and EU Vocabulary. Specific types of online materials, translation tools and sources are discussed from the point of view of student motivation. New media are also discussed from the perspective of their efficiency at different stages of translation teaching. In the concluding part, application of modern technologies in teaching legal translation is compared with other teaching methods, approaches and techniques. Finally, the author questions using IT as motivation tools in the higher education teaching discourse and argues for application of “moderate approach” in the teaching of legal translation.Ekspansja mediów informatycznych w każdej dziedzinie życia jest jedną z podstawowych cech współczesnego życia. Niniejszy artykuł ma na celu przedstawienie roli mediów elektronicznych w nauczaniu przekładu prawniczego na Uniwersytecie w Osijek w Chorwacji oraz przeanalizowanie ich wpływu na motywację grupy docelowej studentów w procesie nauczania. Autorka stara się przedstawić nowoczesną praktykę dydaktyczną i przeanalizować wzajemne powiązania korzystania z mediów elektronicznych i motywację studentów. W pierwszej części artykułu zaproponowano teoretyczne podejście do nauczania tłumaczenia prawniczego. Na przykładach programu „Lifelong Learning Programme for Lawyer-Linguists” na Wydziale Prawa Osijek oraz kursu tłumaczenia prawniczego w ramach „German Language and Literature Studies” na Wydziale Nauk Humanistycznych i Społecznych w Osijek autorka prezentuje nauczanie tłumaczenia prawniczego przy użyciu nowoczesnych mediów. Wykorzystanie mediów elektronicznych w nauczaniu tłumaczeń omawia się w odniesieniu do kursów „Wprowadzenie do teorii tłumaczenia prawniczego i narzędzi tłumaczenia online oraz słownictwa UE”. Konkretne rodzaje materiałów online, narzędzi tłumaczeniowych i źródeł omawia się z punktu widzenia motywacji studentów. Nowe media są również analizowane pod kątem ich skuteczności na różnych etapach nauczania przekładu. Podsumowując, zastosowanie nowoczesnych technologii w nauczaniu tłumaczenia prawniczego porównuje się z innymi metodami, podejściami i technikami nauczania. Na koniec autorka kwestionuje zasadność wykorzystania narzędzi IT jako motywatorów w dyskursie dydaktycznym szkolnictwa wyższego i opowiada się za zastosowaniem „umiarkowanego podejścia” w nauczaniu tłumaczenia prawniczego

A Neural, Interactive-predictive System for Multimodal Sequence to Sequence Tasks

We present a demonstration of a neural interactive-predictive system for tackling multimodal sequence to sequence tasks. The system generates text predictions to different sequence to sequence tasks: machine translation, image and video captioning. These predictions are revised by a human agent, who introduces corrections in the form of characters. The system reacts to each correction, providing alternative hypotheses, compelling with the feedback provided by the user. The final objective is to reduce the human effort required during this correction process. This system is implemented following a client-server architecture. For accessing the system, we developed a website, which communicates with the neural model, hosted in a local server. From this website, the different tasks can be tackled following the interactive-predictive framework. We open-source all the code developed for building this system. The demonstration in hosted in http://casmacat.prhlt.upv.es/interactive-seq2seq.Comment: ACL 2019 - System demonstration

The impact of machine translation error types on post-editing effort indicators

In this paper, we report on a post-editing study for general text types from English into Dutch conducted with master's students of translation. We used a fine-grained machine translation (MT) quality assessment method with error weights that correspond to severity levels and are related to cognitive load. Linear mixed effects models are applied to analyze the impact of MT quality on potential post-editing effort indicators. The impact of MT quality is evaluated on three different levels, each with an increasing granularity. We find that MT quality is a significant predictor of all different types of post-editing effort indicators and that different types of MT errors predict different post-editing effort indicators

Identifying the machine translation error types with the greatest impact on post-editing effort

Translation Environment Tools make translators' work easier by providing them with term lists, translation memories and machine translation output. Ideally, such tools automatically predict whether it is more effortful to post-edit than to translate from scratch, and determine whether or not to provide translators with machine translation output. Current machine translation quality estimation systems heavily rely on automatic metrics, even though they do not accurately capture actual post-editing effort. In addition, these systems do not take translator experience into account, even though novices' translation processes are different from those of professional translators. In this paper, we report on the impact of machine translation errors on various types of post-editing effort indicators, for professional translators as well as student translators. We compare the impact of MT quality on a product effort indicator (HTER) with that on various process effort indicators. The translation and post-editing process of student translators and professional translators was logged with a combination of keystroke logging and eye-tracking, and the MT output was analyzed with a fine-grained translation quality assessment approach. We find that most post-editing effort indicators (product as well as process) are influenced by machine translation quality, but that different error types affect different post-editing effort indicators, confirming that a more fine-grained MT quality analysis is needed to correctly estimate actual post-editing effort. Coherence, meaning shifts, and structural issues are shown to be good indicators of post-editing effort. The additional impact of experience on these interactions between MT quality and post-editing effort is smaller than expected

Learning to Attend, Copy, and Generate for Session-Based Query Suggestion

Users try to articulate their complex information needs during search sessions by reformulating their queries. To make this process more effective, search engines provide related queries to help users in specifying the information need in their search process. In this paper, we propose a customized sequence-to-sequence model for session-based query suggestion. In our model, we employ a query-aware attention mechanism to capture the structure of the session context. is enables us to control the scope of the session from which we infer the suggested next query, which helps not only handle the noisy data but also automatically detect session boundaries. Furthermore, we observe that, based on the user query reformulation behavior, within a single session a large portion of query terms is retained from the previously submitted queries and consists of mostly infrequent or unseen terms that are usually not included in the vocabulary. We therefore empower the decoder of our model to access the source words from the session context during decoding by incorporating a copy mechanism. Moreover, we propose evaluation metrics to assess the quality of the generative models for query suggestion. We conduct an extensive set of experiments and analysis. e results suggest that our model outperforms the baselines both in terms of the generating queries and scoring candidate queries for the task of query suggestion.Comment: Accepted to be published at The 26th ACM International Conference on Information and Knowledge Management (CIKM2017