Distilling Word Embeddings: An Encoding Approach

Distilling knowledge from a well-trained cumbersome network to a small one has recently become a new research topic, as lightweight neural networks with high performance are particularly in need in various resource-restricted systems. This paper addresses the problem of distilling word embeddings for NLP tasks. We propose an encoding approach to distill task-specific knowledge from a set of high-dimensional embeddings, which can reduce model complexity by a large margin as well as retain high accuracy, showing a good compromise between efficiency and performance. Experiments in two tasks reveal the phenomenon that distilling knowledge from cumbersome embeddings is better than directly training neural networks with small embeddings.Comment: Accepted by CIKM-16 as a short paper, and by the Representation Learning for Natural Language Processing (RL4NLP) Workshop @ACL-16 for presentatio

Meson Instability of Quantum Many-body Scars in a 1D Lattice Gauge Theory

We investigate the stability of meson excitations (particle-antiparticle bound states) in quantum many-body scars of a 1D $\mathbb{Z}_2$ lattice gauge theory coupled to a dynamical spin-$\frac{1}{2}$ chain as a matter field. By introducing a string representation of the physical Hilbert space, we express a scar state $\ket {\Psi_{n,l}}$ as a superposition of all string bases with an identical string number $n$ and a total length $l$. For the small-$l$ scar state $\ket {\Psi_{n,l}}$, the gauge-invariant spin exchange correlation function of the matter field hosts an exponential decay as the distance increases, indicating the existence of stable mesons. However, for large $l$, the correlation function exhibits a power-law decay, signaling the emergence of a meson instability. Furthermore, we show that this mesonic-nonmesonic crossover can be detected by the quench dynamics, starting from two low-entangled initial states, respectively, which are experimentally feasible in quantum simulators. Our results expand the physics of quantum many-body scars in lattice gauge theories and reveal that the nonmesonic state can also manifest ergodicity breaking.Comment: 7+2 pages, 4+1 figure