Large Margin Neural Language Model

We propose a large margin criterion for training neural language models. Conventionally, neural language models are trained by minimizing perplexity (PPL) on grammatical sentences. However, we demonstrate that PPL may not be the best metric to optimize in some tasks, and further propose a large margin formulation. The proposed method aims to enlarge the margin between the "good" and "bad" sentences in a task-specific sense. It is trained end-to-end and can be widely applied to tasks that involve re-scoring of generated text. Compared with minimum-PPL training, our method gains up to 1.1 WER reduction for speech recognition and 1.0 BLEU increase for machine translation.Comment: 9 pages. Accepted as a long paper in EMNLP201

Three Dimensional Imaging of the Nucleon and Semi-Inclusive High Energy Reactions

We present a short overview on the studies of transverse momentum dependent parton distribution functions of the nucleon. The aim of such studies is to provide a three dimensional imagining of the nucleon and a comprehensive description of semi-inclusive high energy reactions. By comparing with the theoretical framework that we have for the inclusive deep inelastic lepton-nucleon scattering and the one-dimensional imaging of the nucleon, we summarize what we need to do in order to construct such a comprehensive theoretical framework for semi-inclusive processes in terms of three dimensional gauge invariant parton distributions. After that, we present an overview of what we have already achieved with emphasize on the theoretical framework for semi-inclusive reactions in leading order perturbative QCD but with leading and higher twist contributions. We summarize in particular the results for the differential cross section and the azimuthal spin asymmetries in terms of the gauge invariant transverse momentum dependent parton distribution functions. We also briefly summarize the available experimental results on semi-inclusive reactions and parameterizations of transverse momentum dependent parton distributions extracted from them and make an outlook for the future studies.Comment: 20 pages, 7 figure