Ordering-sensitive and Semantic-aware Topic Modeling

Topic modeling of textual corpora is an important and challenging problem. In most previous work, the "bag-of-words" assumption is usually made which ignores the ordering of words. This assumption simplifies the computation, but it unrealistically loses the ordering information and the semantic of words in the context. In this paper, we present a Gaussian Mixture Neural Topic Model (GMNTM) which incorporates both the ordering of words and the semantic meaning of sentences into topic modeling. Specifically, we represent each topic as a cluster of multi-dimensional vectors and embed the corpus into a collection of vectors generated by the Gaussian mixture model. Each word is affected not only by its topic, but also by the embedding vector of its surrounding words and the context. The Gaussian mixture components and the topic of documents, sentences and words can be learnt jointly. Extensive experiments show that our model can learn better topics and more accurate word distributions for each topic. Quantitatively, comparing to state-of-the-art topic modeling approaches, GMNTM obtains significantly better performance in terms of perplexity, retrieval accuracy and classification accuracy.Comment: To appear in proceedings of AAAI 201

Translating Phrases in Neural Machine Translation

Phrases play an important role in natural language understanding and machine translation (Sag et al., 2002; Villavicencio et al., 2005). However, it is difficult to integrate them into current neural machine translation (NMT) which reads and generates sentences word by word. In this work, we propose a method to translate phrases in NMT by integrating a phrase memory storing target phrases from a phrase-based statistical machine translation (SMT) system into the encoder-decoder architecture of NMT. At each decoding step, the phrase memory is first re-written by the SMT model, which dynamically generates relevant target phrases with contextual information provided by the NMT model. Then the proposed model reads the phrase memory to make probability estimations for all phrases in the phrase memory. If phrase generation is carried on, the NMT decoder selects an appropriate phrase from the memory to perform phrase translation and updates its decoding state by consuming the words in the selected phrase. Otherwise, the NMT decoder generates a word from the vocabulary as the general NMT decoder does. Experiment results on the Chinese to English translation show that the proposed model achieves significant improvements over the baseline on various test sets.Comment: Accepted by EMNLP 201

Quantum coherence of the molecular states and their corresponding currents in nanoscale Aharonov-Bohm interferometers

By considering a nanoscale Aharonov-Bohm (AB) interferometer containing a parrallel-coupled double dot coupled to the source and drain electrodes, we investigate the AB phase oscillations of transport current via the bonding and antibonding state channels. The results we obtained justify the experimental analysis given in [Phys. Rev. Lett. \textbf{106}, 076801 (2011)] that bonding state currents in different energy configurations are almost the same. On the other hand, we extend the analysis to the transient transport current components flowing through different channels, to explore the effect of the parity of bonding and antibonding states on the AB phase dependence of the corresponding current components in the transient regime. The relations of the AB phase dependence between the quantum states and the associated current components are analyzed in details, which provides useful information for the reconstruction of quantum states through the measurement of the transport current in such systems. With the coherent properties in the quantum dot states as well as in the transport currents, we also provide a way to manipulate the bonding and antibonding states by the AB magnetic flux.Comment: 10 pages, 7 figure

Transient probing of the symmetry and the asymmetry of electron interference

The transient processes of electron transport in nano-scale devices exhibit special phenomena that exist only in the transient regime. Besides how fast the steady states are approached, one interesting aspect of transient transport arises from its strong dependence on the initial state of the system. Here we address the issue of how the symmetries embedded in the initial state interplay with those of the system structure in the course of transient transports. We explicitly explore the transient currents arising from various initial occupations in a double-quantum-dot Aharonov-Bohm interferometer. We find symmetry relations between the transient in-tunneling and out-tunneling dynamics for initially empty or full quantum dots when the energy levels in the electrodes are symmetrically distributed with respect to the energy levels in the QDs. This is true for whatever applied fluxes. We also find the flux-even components of the currents and the flux-odd components of the currents exhibit distinct cross-lead symmetric relations.Comment: 10 figure