79 research outputs found
Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input
Non-autoregressive translation (NAT) models, which remove the dependence on
previous target tokens from the inputs of the decoder, achieve significantly
inference speedup but at the cost of inferior accuracy compared to
autoregressive translation (AT) models. Previous work shows that the quality of
the inputs of the decoder is important and largely impacts the model accuracy.
In this paper, we propose two methods to enhance the decoder inputs so as to
improve NAT models. The first one directly leverages a phrase table generated
by conventional SMT approaches to translate source tokens to target tokens,
which are then fed into the decoder as inputs. The second one transforms
source-side word embeddings to target-side word embeddings through
sentence-level alignment and word-level adversary learning, and then feeds the
transformed word embeddings into the decoder as inputs. Experimental results
show our method largely outperforms the NAT baseline~\citep{gu2017non} by
BLEU scores on WMT14 English-German task and BLEU scores on WMT16
English-Romanian task.Comment: AAAI 201
Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translation
Non-autoregressive translation (NAT) models remove the dependence on previous
target tokens and generate all target tokens in parallel, resulting in
significant inference speedup but at the cost of inferior translation accuracy
compared to autoregressive translation (AT) models. Considering that AT models
have higher accuracy and are easier to train than NAT models, and both of them
share the same model configurations, a natural idea to improve the accuracy of
NAT models is to transfer a well-trained AT model to an NAT model through
fine-tuning. However, since AT and NAT models differ greatly in training
strategy, straightforward fine-tuning does not work well. In this work, we
introduce curriculum learning into fine-tuning for NAT. Specifically, we design
a curriculum in the fine-tuning process to progressively switch the training
from autoregressive generation to non-autoregressive generation. Experiments on
four benchmark translation datasets show that the proposed method achieves good
improvement (more than BLEU score) over previous NAT baselines in terms of
translation accuracy, and greatly speed up (more than times) the inference
process over AT baselines.Comment: AAAI 202
Recommended from our members
A new model to downscale urban and rural surface and air temperatures evaluated in Shanghai, China
A simple model, TsT2m (Surface Temperature and near surface air Temperature (at 2 m) model), is developed to downscale numerical model output (such as from ECMWF) to obtain higher temporal and spatial resolution surface and near surface air temperature. It is evaluated in Shanghai, China. Surface temperature (TS) and near surface air temperature (Ta) sub-models account for variations in land covers and their different thermal properties, resulting in spatial variations of surface and air temperature. The Net All Wave Radiation Parameterization (NARP) scheme is used to compute net wave radiation for the surface temperature sub-model, the Objective Hysteresis Model (OHM) is used to calculate the net storage heat fluxes, and the surface temperature is obtained by the force-restore method. The near surface air temperature sub-model considers the horizontal and vertical energy changes for a column of well mixed air above the surface. Modeled surface temperatures reproduce the general pattern of MODIS images well, while providing more detailed patterns of the surface urban heat island. However, the simulated surface temperatures capture the warmer urban land cover and are 10.3°C warmer on average than those derived from the coarser MODIS data. For other land cover types values are more similar. Downscaled, higher temporal and spatial resolution air temperatures are compared to observations at 110 Automatic Weather Stations across Shanghai. After downscaling with the TsT2m model, the average forecast accuracy of near surface air temperature is improved by about 20%. The scheme developed has considerable potential for prediction and mitigation of urban climate conditions, particularly for weather and climate services related to heat stres
Overview to the Hard X-ray Modulation Telescope (Insight-HXMT) Satellite
As China's first X-ray astronomical satellite, the Hard X-ray Modulation
Telescope (HXMT), which was dubbed as Insight-HXMT after the launch on June 15,
2017, is a wide-band (1-250 keV) slat-collimator-based X-ray astronomy
satellite with the capability of all-sky monitoring in 0.2-3 MeV. It was
designed to perform pointing, scanning and gamma-ray burst (GRB) observations
and, based on the Direct Demodulation Method (DDM), the image of the scanned
sky region can be reconstructed. Here we give an overview of the mission and
its progresses, including payload, core sciences, ground calibration/facility,
ground segment, data archive, software, in-orbit performance, calibration,
background model, observations and some preliminary results.Comment: 29 pages, 40 figures, 6 tables, to appear in Sci. China-Phys. Mech.
Astron. arXiv admin note: text overlap with arXiv:1910.0443
Insight-HXMT observations of Swift J0243.6+6124 during its 2017-2018 outburst
The recently discovered neutron star transient Swift J0243.6+6124 has been
monitored by {\it the Hard X-ray Modulation Telescope} ({\it Insight-\rm HXMT).
Based on the obtained data, we investigate the broadband spectrum of the source
throughout the outburst. We estimate the broadband flux of the source and
search for possible cyclotron line in the broadband spectrum. No evidence of
line-like features is, however, found up to . In the absence of
any cyclotron line in its energy spectrum, we estimate the magnetic field of
the source based on the observed spin evolution of the neutron star by applying
two accretion torque models. In both cases, we get consistent results with
, and peak luminosity of which makes the source the first Galactic ultraluminous
X-ray source hosting a neutron star.Comment: publishe
Research status of monitoring and control technology of rapid coal loading system
Based on a simple introduction of rapid coal loading system, the paper emphatically discussed research status of monitoring technologies of its subsystems including coal storage silo, coal conveying system, weighing system, hydraulic system, vehicle location and identification system. It summarized application status of intelligent control technologies including fuzzy control, neural network control and expert control technology in the rapid coal loading system. Finally, it pointed out some important development tendencies of the monitoring and control technologies for the rapid coal loading system
- …