13,283 research outputs found
Interactive-predictive neural multimodal systems
[EN] Despite the advances achieved by neural models in sequence
to sequence learning, exploited in a variety of tasks, they still make errors.
In many use cases, these are corrected by a human expert in a posterior
revision process. The interactive-predictive framework aims to minimize
the human effort spent on this process by considering partial corrections
for iteratively refining the hypothesis. In this work, we generalize the
interactive-predictive approach, typically applied in to machine translation field, to tackle other multimodal problems namely, image and video
captioning. We study the application of this framework to multimodal
neural sequence to sequence models. We show that, following this framework, we approximately halve the effort spent for correcting the outputs
generated by the automatic systems. Moreover, we deploy our systems
in a publicly accessible demonstration, that allows to better understand
the behavior of the interactive-predictive framework.The research leading to these results has received funding from MINECO under grant
IDIFEDER/2018/025 Sistemas de fabricacion inteligentes para la industria 4.0,
action co-funded by the European Regional Development Fund 2014-2020 (FEDER),
and from the European Commission under grant H2020, reference 825111 (DeepHealth). We also acknowledge NVIDIA Corporation for the donation of GPUs used
in this work.Peris, Á.; Casacuberta Nolla, F. (2019). Interactive-predictive neural multimodal systems. Springer. 16-28. https://doi.org/978-3-030-31332-6_2S162
A Neural, Interactive-predictive System for Multimodal Sequence to Sequence Tasks
We present a demonstration of a neural interactive-predictive system for
tackling multimodal sequence to sequence tasks. The system generates text
predictions to different sequence to sequence tasks: machine translation, image
and video captioning. These predictions are revised by a human agent, who
introduces corrections in the form of characters. The system reacts to each
correction, providing alternative hypotheses, compelling with the feedback
provided by the user. The final objective is to reduce the human effort
required during this correction process.
This system is implemented following a client-server architecture. For
accessing the system, we developed a website, which communicates with the
neural model, hosted in a local server. From this website, the different tasks
can be tackled following the interactive-predictive framework. We open-source
all the code developed for building this system. The demonstration in hosted in
http://casmacat.prhlt.upv.es/interactive-seq2seq.Comment: ACL 2019 - System demonstration
Modelling source- and target-language syntactic Information as conditional context in interactive neural machine translation
In interactive machine translation (MT),
human translators correct errors in auto-
matic translations in collaboration with the
MT systems, which is seen as an effective
way to improve the productivity gain in
translation. In this study, we model source-
language syntactic constituency parse and
target-language syntactic descriptions in
the form of supertags as conditional con-
text for interactive prediction in neural
MT (NMT). We found that the supertags
significantly improve productivity gain in
translation in interactive-predictive NMT
(INMT), while syntactic parsing somewhat
found to be effective in reducing human
efforts in translation. Furthermore, when
we model this source- and target-language
syntactic information together as the con-
ditional context, both types complement
each other and our fully syntax-informed
INMT model shows statistically significant
reduction in human efforts for a French–
to–English translation task in a reference-
simulated setting, achieving 4.30 points
absolute (corresponding to 9.18% relative)
improvement in terms of word prediction
accuracy (WPA) and 4.84 points absolute
(corresponding to 9.01% relative) reduc-
tion in terms of word stroke ratio (WSR)
over the baseline
- …