WMT 2016 Multimodal translation system description based on bidirectional recurrent neural networks with double-embeddings

Bidirectional Recurrent Neural Networks (BiRNNs) have shown outstanding results on sequence-to-sequence learning tasks. This architecture becomes specially interesting for multimodal machine translation task, since BiRNNs can deal with images and text. On most translation systems the same word embedding is fed to both BiRNN units. In this paper, we present several experiments to enhance a baseline sequence-to-sequence system (Elliott et al., 2015), for example, by using double embeddings. These embeddings are trained on the forward and backward direction of the input sequence. Our system is trained, validated and tested on the Multi30K dataset (Elliott et al., 2016) in the context of theWMT 2016Multimodal Translation Task. The obtained results show that thedouble-embedding approach performs significantly better than the traditional single-embedding one.Postprint (published version

A differentiable BLEU loss. Analysis and first results

In natural language generation tasks, like neural machine translation and image captioning, there is usually a mismatch between the optimized loss and the de facto evaluation criterion, namely token-level maximum likelihood and corpus-level BLEU score. This article tries to reduce this gap by defining differentiable computations of the BLEU and GLEU scores. We test this approach on simple tasks, obtaining valuable lessons on its potential applications but also its pitfalls, mainly that these loss functions push each token in the hypothesis sequence toward the average of the tokens in the reference, resulting in a poor training signal.Peer ReviewedPostprint (published version

Neural machine translation using bitmap fonts

Recently, translation systems based on neural networks are starting to compete with systems based on phrases. The systems which are based on neural networks use vectorial repre- sentations of words. However, one of the biggest challenges that machine translation still faces, is dealing with large vocabularies and morphologically rich languages. This work aims to adapt a neural machine translation system to translate from Chinese to Spanish, using as input different types of granularity: words, characters, bitmap fonts of Chinese characters or words. The fact of performing the interpretation of every character or word as a bitmap font allows for obtaining more informed vectorial representations. Best results are obtained when using the information of the word bitmap font.Postprint (published version

Chinese-Catalan: A neural machine translation approach based on pivoting and attention mechanisms

This article innovatively addresses machine translation from Chinese to Catalan using neural pivot strategies trained without any direct parallel data. The Catalan language is very similar to Spanish from a linguistic point of view, which motivates the use of Spanish as pivot language. Regarding neural architecture, we are using the latest state-of-the-art, which is the Transformer model, only based on attention mechanisms. Additionally, this work provides new resources to the community, which consists of a human-developed gold standard of 4,000 sentences between Catalan and Chinese and all the others United Nations official languages (Arabic, English, French, Russian, and Spanish). Results show that the standard pseudo-corpus or synthetic pivot approach performs better than cascade.Peer ReviewedPostprint (author's final draft

Latest trends in hybrid machine translation and its applications

This survey on hybrid machine translation (MT) is motivated by the fact that hybridization techniques have become popular as they attempt to combine the best characteristics of highly advanced pure rule or corpus-based MT approaches. Existing research typically covers either simple or more complex architectures guided by either rule or corpus-based approaches. The goal is to combine the best properties of each type. This survey provides a detailed overview of the modification of the standard rule-based architecture to include statistical knowl- edge, the introduction of rules in corpus-based approaches, and the hybridization of approaches within this last single category. The principal aim here is to cover the leading research and progress in this field of MT and in several related applications.Peer ReviewedPostprint (published version

Character-based neural machine translation

Neural Machine Translation (MT) has reached state-of-the-art results. However, one of the main challenges that neural MT still faces is dealing with very large vocabularies and morphologically rich languages. In this paper, we propose a neural MT system using character-based embeddings in combination with convolutional and highway layers to replace the standard lookup-based word representations. The resulting unlimited-vocabulary and affixaware source word embeddings are tested in a state-of-the-art neural MT based on an attention-based bidirectional recurrent neural network. The proposed MT scheme provides improved results even when the source language is not morphologically rich. Improvements up to 3 BLEU points are obtained in the German-English WMT task.Peer ReviewedPostprint (author's final draft

DeepVoice: tecnologías de aprendizaje profundo aplicadas al procesado de voz y audio

Este proyecto propone el desarrollo de nuevas arquitecturas para el procesado de la voz y el audio mediante métodos de aprendizaje profundo, explorando también nuevas aplicaciones y dando continuidad al trabajo inicial del equipo de investigadores solicitante y de toda la comunidad internacional. Las lineas de investigación incluyen: reconocimiento de voz, reconocimiento de eventos acústicos, síntesis de voz y traducción automáticaPeer ReviewedPostprint (author's final draft

DeepVoice: tecnologías de aprendizaje profundo aplicadas al procesado de voz y audio

This project proposes the development of new deep learning methods for speech and audio processing, exploring new applications and continuing the initial work of the research team and the international community. Research lines include: automatic speech recognition, acoustic event detection, speech synthesis and machine translation.Peer ReviewedPostprint (published version

Using linear interpolation and weighted reordering hypotheses in the moses system

This paper proposes to introduce a novel reordering model in the open-source Moses toolkit. The main idea is to provide weighted reordering hypotheses to the SMT decoder. These hypotheses are built using a first-step Ngram-based SMT translation from a source language into a third representation that is called reordered source language. Each hypothesis has its own weight provided by the Ngram-based decoder. This proposed reordering technique offers a better and more efficient translation when compared to both the distance-based and the lexicalized reordering. In addition to this reordering approach, this paper describes a domain adaptation technique which is based on a linear combination of an specific indomain and an extra out-domain translation models. Results for both approaches are reported in the Arabic-to-English 2008 IWSLT task. When implementing the weighted reordering hypotheses and the domain adaptation technique in the final translation system, translation results reach improvements up to 2.5 BLEU compared to a standard state-of-the-art Moses baseline system.Postprint (published version

Functional characterization of two enhancers located downstream FOXP2

Background: Mutations in the coding region of FOXP2 are known to cause speech and language impairment. However, it is not clear how dysregulation of the gene contributes to language deficit. Interestingly, microdeletions of the region downstream the gene have been associated with cognitive deficits. Methods: Here, we investigate changes in FOXP2 expression in the SK-N-MC neuroblastoma human cell line after deletion by CRISPR-Cas9 of two enhancers located downstream of the gene. Results: Deletion of any of these two functional enhancers downregulates FOXP2, but also upregulates the closest 3′ gene MDFIC. Because this effect is not statistically significant in a HEK 293 cell line, derived from the human kidney, both enhancers might confer a tissue specific regulation to both genes. We have also found that the deletion of any of these enhancers downregulates six well-known FOXP2 target genes in the SK-N-MC cell line. Conclusions: We expect these findings contribute to a deeper understanding of how FOXP2 and MDFIC are regulated to pace neuronal development supporting cognition, speech and language.Spanish National Research and Development Plan PI14/01884Instituto de Salud Carlos III PI14/01884FEDER PI14/0188