Regressing Word and Sentence Embeddings for Regularization of Neural Machine Translation

In recent years, neural machine translation (NMT) has become the dominant approach in automated translation. However, like many other deep learning approaches, NMT suffers from overfitting when the amount of training data is limited. This is a serious issue for low-resource language pairs and many specialized translation domains that are inherently limited in the amount of available supervised data. For this reason, in this paper we propose regressing word (ReWE) and sentence (ReSE) embeddings at training time as a way to regularize NMT models and improve their generalization. During training, our models are trained to jointly predict categorical (words in the vocabulary) and continuous (word and sentence embeddings) outputs. An extensive set of experiments over four language pairs of variable training set size has showed that ReWE and ReSE can outperform strong state-of-the-art baseline models, with an improvement that is larger for smaller training sets (e.g., up to +5:15 BLEU points in Basque-English translation). Visualizations of the decoder's output space show that the proposed regularizers improve the clustering of unique words, facilitating correct predictions. In a final experiment on unsupervised NMT, we show that ReWE and ReSE are also able to improve the quality of machine translation when no parallel data are available

ReWE: Regressing word embeddings for regularization of neural machine translation systems

Regularization of neural machine translation is still a significant problem, especially in low-resource settings. To mollify this problem, we propose regressing word embeddings (ReWE) as a new regularization technique in a system that is jointly trained to predict the next word in the translation (categorical value) and its word embedding (continuous value). Such a joint training allows the proposed system to learn the distributional properties represented by the word embeddings, empirically improving the generalization to unseen sentences. Experiments over three translation datasets have showed a consistent improvement over a strong baseline, ranging between 0.91 and 2.54 BLEU points, and also a marked improvement over a state-of-the-art system

NeuSub: A Neural Submodular Approach for Citation Recommendation

Citation recommendation is a task that aims to automatically select suitable references for a working manuscript. This task has become increasingly urgent as the typical pools of candidates continue to grow, in the order of tens or hundreds of thousands or more. While several approaches to citation recommendation have been proposed in the literature, they generally seem to lack principled mechanisms to ensure diversity and other global properties among the recommended citations. For this reason, in this paper we propose a novel citation recommendation approach that leverages a submodular scoring function and a deep document representation to achieve an effective trade-off between relevance to the query and diversity of the references. To optimally train the scoring function and the deep representation, we propose a novel training objective based on a structural/multiclass hinge loss and incremental recommendations. The experimental results over three popular citation datasets have showed that the proposed approach has led to remarkable accuracy improvements, with an increase of up to 1.91 pp of MRR and 3.29 pp of F1@100 score with respect to a state-of-the-art citation recommendation system

Learning Neural Textual Representations for Citation Recommendation

With the rapid growth of the scientific literature, manually selecting appropriate citations for a paper is becoming increasingly challenging and time-consuming. While several approaches for automated citation recommendation have been proposed in the recent years, effective document representations for citation recommendation are still elusive to a large extent. For this reason, in this paper we propose a novel approach to citation recommendation which leverages a deep sequential representation of the documents (Sentence-BERT) cascaded with Siamese and triplet networks in a submodular scoring function. To the best of our knowledge, this is the first approach to combine deep representations and submodular selection for a task of citation recommendation. Experiments have been carried out using a popular benchmark dataset - the ACL Anthology Network corpus - and evaluated against baselines and a state-of-the-art approach using metrics such as the MRR and F1-at-k score. The results show that the proposed approach has been able to outperform all the compared approaches in every measured metric

Findings of the WMT 2021 Biomedical Translation Shared Task: Summaries of Animal Experiments as New Test Set

In the sixth edition of the WMT Biomedical Task, we addressed a total of eight language pairs, namely English/German, English/French, English/Spanish, English/Portuguese, English/Chinese, English/Russian, English/Italian, and English/Basque. Further, our tests were composed of three types of textual test sets. New to this year, we released a test set of summaries of animal experiments, in addition to the test sets of scientific abstracts and terminologies. We received a total of 107 submissions from 15 teams from 6 countries

Listeriolysin O Is Strongly Immunogenic Independently of Its Cytotoxic Activity

The presentation of microbial protein antigens by Major Histocompatibility Complex (MHC) molecules is essential for the development of acquired immunity to infections. However, most biochemical studies of antigen processing and presentation deal with a few relatively inert non-microbial model antigens. The bacterial pore-forming toxin listeriolysin O (LLO) is paradoxical in that it is cytotoxic at nanomolar concentrations as well as being the source of dominant CD4 and CD8 T cell epitopes following infection with Listeria monocytogenes. Here, we examined the relationship of LLO toxicity to its antigenicity and immunogenicity. LLO offered to antigen presenting cells (APC) as a soluble protein, was presented to CD4 T cells at picomolar to femtomolar concentrations- doses 3000–7000-fold lower than free peptide. This presentation required a dose of LLO below the cytotoxic level. Mutations of two key tryptophan residues reduced LLO toxicity by 10–100-fold but had no effect on its presentation to CD4 T cells. Thus there was a clear dissociation between the cytotoxic properties of LLO and its very high antigenicity. Presentation of LLO to CD8 T cells was not as robust as that seen in CD4 T cells, but still occurred in the nanomolar range. APC rapidly bound and internalized LLO, then disrupted endosomal compartments within 4 hours of treatment, allowing endosomal contents to access the cytosol. LLO was also immunogenic after in vivo administration into mice. Our results demonstrate the strength of LLO as an immunogen to both CD4 and CD8 T cells

English-Basque statistical and neural machine translation

© LREC 2018 - 11th International Conference on Language Resources and Evaluation. All rights reserved. Neural Machine Translation (NMT) has attracted increasing attention in the recent years. However, it tends to require very large training corpora which could prove problematic for languages with low resources. For this reason, Statistical Machine Translation (SMT) continues to be a popular approach for low-resource language pairs. In this work, we address English-Basque translation and compare the performance of three contemporary statistical and neural machine translation systems: OpenNMT, Moses SMT and Google Translate. For evaluation, we employ an open-domain and an IT-domain corpora from the WMT16 resources for machine translation. In addition, we release a small dataset (Berriak) of 500 highly-accurate English-Basque translations of complex sentences useful for a thorough testing of the translation systems