The AMU-UEDIN Submission to the WMT16 News Translation Task: Attention-based NMT Models as Feature Functions in Phrase-based SMT

This paper describes the AMU-UEDIN submissions to the WMT 2016 shared task on news translation. We explore methods of decode-time integration of attention-based neural translation models with phrase-based statistical machine translation. Efficient batch-algorithms for GPU-querying are proposed and implemented. For English-Russian, our system stays behind the state-of-the-art pure neural models in terms of BLEU. Among restricted systems, manual evaluation places it in the first cluster tied with the pure neural model. For the Russian-English task, our submission achieves the top BLEU result, outperforming the best pure neural system by 1.1 BLEU points and our own phrase-based baseline by 1.6 BLEU. After manual evaluation, this system is the best restricted system in its own cluster. In follow-up experiments we improve results by additional 0.8 BLEU

Is Neural Machine Translation Ready for Deployment? A Case Study on 30 Translation Directions

In this paper we provide the largest published comparison of translation quality for phrase-based SMT and neural machine translation across 30 translation directions. For ten directions we also include hierarchical phrase-based MT. Experiments are performed for the recently published United Nations Parallel Corpus v1.0 and its large six-way sentence-aligned subcorpus. In the second part of the paper we investigate aspects of translation speed, introducing AmuNMT, our efficient neural machine translation decoder. We demonstrate that current neural machine translation could already be used for in-production systems when comparing words-per-second ratios.Comment: Accepted for presentation at IWSLT 2016, Seattl

Fast Neural Machine Translation Implementation

This paper describes the submissions to the efficiency track for GPUs at the Workshop for Neural Machine Translation and Generation by members of the University of Edinburgh, Adam Mickiewicz University, Tilde and University of Alicante. We focus on efficient implementation of the recurrent deep-learning model as implemented in Amun, the fast inference engine for neural machine translation. We improve the performance with an efficient mini-batching algorithm, and by fusing the softmax operation with the k-best extraction algorithm. Submissions using Amun were first, second and third fastest in the GPU efficiency track

Predicting Target Language CCG Supertags Improves Neural Machine Translation

Neural machine translation (NMT) models are able to partially learn syntactic information from sequential lexical information. Still, some complex syntactic phenomena such as prepositional phrase attachment are poorly modeled. This work aims to answer two questions: 1) Does explicitly modeling target language syntax help NMT? 2) Is tight integration of words and syntax better than multitask training? We introduce syntactic information in the form of CCG supertags in the decoder, by interleaving the target supertags with the word sequence. Our results on WMT data show that explicitly modeling target-syntax improves machine translation quality for German->English, a high-resource pair, and for Romanian->English, a low-resource pair and also several syntactic phenomena including prepositional phrase attachment. Furthermore, a tight coupling of words and syntax improves translation quality more than multitask training. By combining target-syntax with adding source-side dependency labels in the embedding layer, we obtain a total improvement of 0.9 BLEU for German->English and 1.2 BLEU for Romanian->English.Comment: Accepted at the Second Conference on Machine Translation (WMT17). This version includes more results regarding target syntax for Romanian->English and reports fewer results regarding source synta

Marian: Fast Neural Machine Translation in C++

We present Marian, an efficient and self-contained Neural Machine Translation framework with an integrated automatic differentiation engine based on dynamic computation graphs. Marian is written entirely in C++. We describe the design of the encoder-decoder framework and demonstrate that a research-friendly toolkit can achieve high training and translation speed.Comment: Demonstration pape

Neurological symptoms in hospitalised patients with COVID-19 and their association with in-hospital mortality

Objectives. To evaluate the spectrum of neurological symptoms in patients with COVID-19 during the first 14 days of hospitalisation and its association with in-hospital mortality. Material and methods. We included 200 patients with RT-PCR-confirmed COVID-19 admitted to University Hospital in Krakow, Poland. In 164 patients, a detailed questionnaire concerning neurological symptoms and signs was performed prospectively within 14 days of hospitalisation. In the remaining 36 patients, such questionnaires were completed retrospectively based on daily observations in the Department of Neurology. Results. During hospitalisation, 169 patients (84.5%) experienced neurological symptoms; the most common were: fatigue (62.5%), decreased mood (45.5%), myalgia (43.5%), and muscle weakness (42.5%). Patients who died during hospitalisation compared to the remainder were older (79 [70.5–88.5] vs. 63.5 [51–77] years, p = 0.001), and more often had decreased level of consciousness (50.0% vs. 9.3%, p < 0.001), delirium (33.3% vs. 4.4%, p < 0.001), arterial hypotension (50.0% vs. 19.6%, p = 0.005) or stroke during (18.8% vs. 3.3%, p = 0.026) or before hospitalisation (50.0% vs. 7.1, p < 0.001), whereas those who survived more often suffered from headache (42.1% vs. 0%, p = 0.012) or decreased mood (51.7% vs. 0%, p = 0.003). Conclusions. Most hospitalised patients with COVID-19 experience neurological symptoms. Decreased level of consciousness, delirium, arterial hypotension, and stroke during or before hospitalisation increase the risk of in-hospital mortality

The SUMMA Platform Prototype

We present the first prototype of the SUMMA Platform: an integrated platform for multilingual media monitoring. The platform contains a rich suite of low-level and high-level natural language processing technologies: automatic speech recognition of broadcast media, machine translation, automated tagging and classification of named entities, semantic parsing to detect relationships between entities, and automatic construction / augmentation of factual knowledge bases. Implemented on the Docker platform, it can easily be deployed, customised, and scaled to large volumes of incoming media streams

From dataset recycling to multi-property extraction and beyond

This paper investigates various Transformerarchitectures on the WikiReading Informa-tion Extraction and Machine Reading Com-prehension dataset. The proposed dual-sourcemodel outperforms the current state-of-the-art by a large margin.Next, we intro-duce WikiReading Recycled—a newly devel-oped public dataset, and the task of multiple-property extraction. It uses the same data asWikiReading but does not inherit its predeces-sor’s identified disadvantages. In addition, weprovide a human-annotated test set with diag-nostic subsets for a detailed analysis of modelperformance

Going full-TILT boogie on document understanding with text-image-layout transformer

We address the challenging problem of Natural Language Comprehension beyond plain-text documents by introducing the TILT neural network architecture which simultaneously learns layout information, visual features, and textual semantics. Contrary to previous approaches, we rely on a decoder capable of unifying a variety of problems involving natural language. The layout is represented as an attention bias and complemented with contextualized visual information, while the core of our model is a pretrained encoder-decoder Transformer. Our novel approach achieves state-of-the-art results in extracting information from documents and answering questions which demand layout understanding (DocVQA, CORD, SROIE). At the same time, we simplify the process by employing an end-to-end model.Comment: Accepted at ICDAR 202