Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translation

Recent studies have revealed a number of pathologies of neural machine translation (NMT) systems. Hypotheses explaining these mostly suggest that there is something fundamentally wrong with NMT as a model or its training algorithm, maximum likelihood estimation (MLE). Most of this evidence was gathered using maximum a posteriori (MAP) decoding, a decision rule aimed at identifying the highest-scoring translation, i.e. the mode, under the model distribution. We argue that the evidence corroborates the inadequacy of MAP decoding more than casts doubt on the model and its training algorithm. In this work, we criticise NMT models probabilistically showing that stochastic samples following the model's own generative story do reproduce various statistics of the training data well, but that it is beam search that strays from such statistics. We show that some of the known pathologies of NMT are due to MAP decoding and not to NMT's statistical assumptions nor MLE. In particular, we show that the most likely translations under the model accumulate so little probability mass that the mode can be considered essentially arbitrary. We therefore advocate for the use of decision rules that take into account statistics gathered from the model distribution holistically. As a proof of concept we show that a straightforward implementation of minimum Bayes risk decoding gives good results outperforming beam search using as little as 30 samples, confirming that MLE-trained NMT models do capture important aspects of translation well in expectation

Auto-Encoding Variational Neural Machine Translation

We present a deep generative model of bilingual sentence pairs for machine translation. The model generates source and target sentences jointly from a shared latent representation and is parameterised by neural networks. We perform efficient training using amortised variational inference and reparameterised gradients. Additionally, we discuss the statistical implications of joint modelling and propose an efficient approximation to maximum a posteriori decoding for fast test-time predictions. We demonstrate the effectiveness of our model in three machine translation scenarios: in-domain training, mixed-domain training, and learning from a mix of gold-standard and synthetic data. Our experiments show consistently that our joint formulation outperforms conditional modelling (i.e. standard neural machine translation) in all such scenarios

Direct frequency comb spectroscopy of trapped ions

Direct frequency comb spectroscopy of trapped ions is demonstated for the first time. It is shown that the 4s^2S_(1/2)-4p^2P_(3/2) transition in calcium ions can be excited directly with a frequency comb laser that is upconverted to 393 nm. Detection of the transition is performed using a shelving scheme to suppress background signal from non-resonant comb modes. The measured transition frequency of f=761 905 012.7(0.5) MHz presents an improvement in accuracy of more than two orders of magnitude.Comment: 4 pages, 5 figur

Frequency metrology on the 4s 2S1/2 - 4p 2P1/2 transition in the calcium ion for a comparison with quasar data

High accuracy frequency metrology on the 4s 2S1/2 - 4p 2P1/2 transition in calcium ions is performed using laser cooled and crystallized ions in a linear Paul trap. Calibration is performed with a frequency comb laser, resulting in a transition frequency of f=755222766.2(1.7) MHz. The accuracy presents an improvement of more than one order of magnitude, and will facilitate a comparison with quasar data in a search for a possible change of the fine structure constant on a cosmological time scale.Comment: Corrected typos (including one on the axis of figure 6

Ion distribution and ablation depth measurements of a fs-ps laser-irradiated solid tin target

The ablation of solid tin surfaces by an 800-nanometer-wavelength laser is studied for a pulse length range from 500 fs to 4.5 ps and a fluence range spanning 0.9 to 22 J/cm^2. The ablation depth and volume are obtained employing a high-numerical-aperture optical microscope, while the ion yield and energy distributions are obtained from a set of Faraday cups set up under various angles. We found a slight increase of the ion yield for an increasing pulse length, while the ablation depth is slightly decreasing. The ablation volume remained constant as a function of pulse length. The ablation depth follows a two-region logarithmic dependence on the fluence, in agreement with the available literature and theory. In the examined fluence range, the ion yield angular distribution is sharply peaked along the target normal at low fluences but rapidly broadens with increasing fluence. The total ionization fraction increases monotonically with fluence to a 5-6% maximum, which is substantially lower than the typical ionization fractions obtained with nanosecond-pulse ablation. The angular distribution of the ions does not depend on the laser pulse length within the measurement uncertainty. These results are of particular interest for the possible utilization of fs-ps laser systems in plasma sources of extreme ultraviolet light for nanolithography.Comment: 8 pages, 7 figure