Improving Lexical Choice in Neural Machine Translation

We explore two solutions to the problem of mistranslating rare words in neural machine translation. First, we argue that the standard output layer, which computes the inner product of a vector representing the context with all possible output word embeddings, rewards frequent words disproportionately, and we propose to fix the norms of both vectors to a constant value. Second, we integrate a simple lexical module which is jointly trained with the rest of the model. We evaluate our approaches on eight language pairs with data sizes ranging from 100k to 8M words, and achieve improvements of up to +4.3 BLEU, surpassing phrase-based translation in nearly all settings.Comment: Accepted at NAACL HLT 201

Masked Language Model Scoring

Pretrained masked language models (MLMs) require finetuning for most NLP tasks. Instead, we evaluate MLMs out of the box via their pseudo-log-likelihood scores (PLLs), which are computed by masking tokens one by one. We show that PLLs outperform scores from autoregressive language models like GPT-2 in a variety of tasks. By rescoring ASR and NMT hypotheses, RoBERTa reduces an end-to-end LibriSpeech model's WER by 30% relative and adds up to +1.7 BLEU on state-of-the-art baselines for low-resource translation pairs, with further gains from domain adaptation. We attribute this success to PLL's unsupervised expression of linguistic acceptability without a left-to-right bias, greatly improving on scores from GPT-2 (+10 points on island effects, NPI licensing in BLiMP). One can finetune MLMs to give scores without masking, enabling computation in a single inference pass. In all, PLLs and their associated pseudo-perplexities (PPPLs) enable plug-and-play use of the growing number of pretrained MLMs; e.g., we use a single cross-lingual model to rescore translations in multiple languages. We release our library for language model scoring at https://github.com/awslabs/mlm-scoring.Comment: ACL 2020 camera-ready (presented July 2020

Cognitive full-duplex relay networks under the peak interference power constraint of multiple primary users

Abstract This paper investigates the outage performance of cognitive spectrum-sharing multi-relay networks in which the relays operate in a full-duplex (FD) mode and employ the decode-and-forward (DF) protocol. Two relay selection schemes, i.e., partial relay selection (PRS) and optimal relay selection (ORS), are considered to enhance the system performance. New exact expressions for the outage probability (OP) in both schemes are derived based on which an asymptotic analysis is carried out. The results show that the ORS strategy outperforms PRS in terms of OP, and increasing the number of FD relays can significantly improve the system performance. Moreover, novel analytical results provide additional insights for system design. In particular, from the viewpoint of FD concept, the primary network parameters (i.e., peak interference at the primary receivers, number of primary receivers, and their locations) should be carefully considered since they significantly affect the secondary network performance

Monitoring for Plasmodium falciparum drug resistance to artemisinin and artesunate in Binh Phuoc Province, Vietnam: 1998-2009

<p>Abstract</p> <p>Background</p> <p>Artemisinin derivatives have been used for malaria treatment in Vietnam since 1989. Reported malaria cases have decreased from 1,672,000 with 4,650 deaths in 1991, to 91,635 with 43 deaths in 2006. Current national guidelines recommend artemisinin-based combination therapy (ACT), although artesunate is still available as monotherapy through the private sector. Recent reports suggest that effectiveness of ACT and artesunate monotherapy has declined in western Cambodia. This study examined <it>Plasmodium falciparum </it>resistance patterns over 10 years in southwest Vietnam in infected patients treated with artemisinin compounds.</p> <p>Methods</p> <p>The study was conducted in two communes in Phuoc Long district, Binh Phuoc province, 100 km west of the Cambodian border. This was chosen as a likely site for emerging artemisinin resistance because of the high prevalence of <it>P. falciparum </it>malaria, and the length of time that artemisinin had been in use. In <it>vivo </it>and <it>in vitro </it>monitoring of <it>P. falciparum </it>susceptibility to anti-malarial drugs was conducted in 1998, 2001, 2004/5, and 2008/9. Patients with confirmed <it>P. falciparum </it>malaria received therapy with 5 or 7 days of artemisinin (1998 and 2001 respectively) or 7 days of artesunate</p> <p>Results</p> <p>In the four surveys, 270 patients were recruited and treated. The mean parasite clearance times differed between 1998, 2001 and 2004/5 (1.8, 2.3 and 2.1 days, P < 0.01) but not between 1998 and 2008/2009. The mean parasite clearance times were correlated with parasite density at day 0 (r = 0.4; P < 0.001). Treatment failure rates after PCR adjustment were 13.8%, 2.9%, 1.2%, and 0% respectively. Susceptibility of <it>P. falciparum </it>to artemisinin in <it>in vitro </it>tests was stable during the period, except for a rise in EC90 and EC99 in 2001.</p> <p>Conclusions</p> <p>This study showed stable levels of <it>P. falciparum </it>sensitivity to artemisinin compounds in the two sites over a ten-year period. The introduction of ACT in this area in 2003 may have protected against the development of artemisinin resistance. Adherence to the latest WHO and Vietnamese guidelines, which recommend ACT as first-line therapy in all malarious areas, and continued monitoring along the Vietnam-Cambodia border will be essential to prevent the spread of artemisinin resistance in Vietnam.</p

Co-infection of human parvovirus B19 with Plasmodium falciparum contributes to malaria disease severity in Gabonese patients

Background: High seroprevalence of parvovirus B19 (B19V) coinfection with Plasmodium falciparum has been previously reported. However, the impact of B19V-infection on the clinical course of malaria is still elusive. In this study, we investigated the prevalence and clinical significance of B19V co-infection in Gabonese children with malaria. Methods: B19V prevalence was analyzed in serum samples of 197 Gabonese children with P. falciparum malaria and 85 healthy controls using polymerase chain reaction (PCR), enzyme-linked immunosorbent assay (ELISA), and direct DNA-sequencing. Results: B19V was detected in 29/282 (10.28%) of Gabonese children. B19V was observed more frequently in P. falciparum malaria patients (14.21%) in comparison to healthy individuals (1.17%) (

Usability and challenges of offshore wind energy in Vietnam revealed by the regional climate model simulation

This study revealed great potential and shortcoming of offshore wind energy in Vietnam by numerical simulations with Weather Research and Forecasting (WRF) model at 10-km resolution for 10 years (2006-2015). The greatest energy potential was found in the offshore area of Phu Quy island (Binh Thuan province). The area, alone, can provide the 38.2 GW power generation capacity corresponding to the increasing renewable-energy demand by 2030 planned by the country. There is also a drawback of the wind resource, which is associated with strong multiple-scale temporal variabilities. The seasonal variability associated with monsoon onsets and daily variability associated with the wind diurnal cycles were found ranging 30-50%. Meanwhile, the inter-annual variability could reach up to 10%. These variabilities must be considered when designing wind farms and grids over the region. Additionally, due to the fact that the WRF model performed climatological features of the winds well against the observations, this results indicate that it can be useful tools for wind-power assessment as compared to other reanalysis or QuikSCAT data with coarser spatio-temporal resolutions