Neural Models for Measuring Confidence on Interactive Machine Translation Systems

[EN] Reducing the human effort performed with the use of interactive-predictive neural machine translation (IPNMT) systems is one of the main goals in this sub-field of machine translation (MT). Prior works have focused on changing the human¿machine interaction method and simplifying the feedback performed. Applying confidence measures (CM) to an IPNMT system helps decrease the number of words that the user has to check through the translation session, reducing the human effort needed, although this supposes losing a few points in the quality of the translations. The effort reduction comes from decreasing the number of words that the translator has to review¿it only has to check the ones with a score lower than the threshold set. In this paper, we studied the performance of four confidence measures based on the most used metrics on MT. We trained four recurrent neural network (RNN) models to approximate the scores from the metrics: Bleu, Meteor, Chr-f, and TER. In the experiments, we simulated the user interaction with the system to obtain and compare the quality of the translations generated with the effort reduction. We also compare the performance of the four models between them to see which of them obtains the best results. The results achieved showed a reduction of 48% with a Bleu score of 70 points¿a significant effort reduction to translations almost perfect.This work received funds from the Comunitat Valenciana under project EU-FEDER (ID-IFEDER/2018/025), Generalitat Valenciana under project ALMAMATER (PrometeoII/2014/030), and Ministerio de Ciencia e Investigacion/Agencia Estatal de Investigacion/10.13039/501100011033/and "FEDER Una manera de hacer Europa" under project MIRANDA-DocTIUM (RTI2018-095645-B-C22).Navarro-Martínez, Á.; Casacuberta Nolla, F. (2022). Neural Models for Measuring Confidence on Interactive Machine Translation Systems. Applied Sciences. 12(3):1-16. https://doi.org/10.3390/app1203110011612

Interactive and Adaptive Neural Machine Translation

In this dissertation, we examine applications of neural machine translation to computer aided translation, with the goal of building tools for human translators. We present a neural approach to interactive translation prediction (a form of "auto-complete" for human translators) and demonstrate its effectiveness through both simulation studies, where it outperforms a phrase-based statistical machine translation approach, and a user study. We find that about half of the translators in the study are faster using neural interactive translation prediction than they are when post-editing output of the same underlying machine translation system, and most translators express positive reactions to the tool. We perform an analysis of some challenges that neural machine translation systems face, particularly with respect to novel words and consistency. We experiment with methods of improving translation quality at a fine-grained level to address those challenges. Finally, we bring these two areas -- interactive and adaptive neural machine translation -- together in a simulation that shows that their combination has a positive impact on novel word translation and other metrics