Using Multi-Sense Vector Embeddings for Reverse Dictionaries

Popular word embedding methods such as word2vec and GloVe assign a single vector representation to each word, even if a word has multiple distinct meanings. Multi-sense embeddings instead provide different vectors for each sense of a word. However, they typically cannot serve as a drop-in replacement for conventional single-sense embeddings, because the correct sense vector needs to be selected for each word. In this work, we study the effect of multi-sense embeddings on the task of reverse dictionaries. We propose a technique to easily integrate them into an existing neural network architecture using an attention mechanism. Our experiments demonstrate that large improvements can be obtained when employing multi-sense embeddings both in the input sequence as well as for the target representation. An analysis of the sense distributions and of the learned attention is provided as well

A Formal Proof of the Expressiveness of Deep Learning

International audienceDeep learning has had a profound impact on computer science in recent years, with applications to image recognition, language processing, bioinformatics, and more. Recently , Cohen et al. provided theoretical evidence for the superiority of deep learning over shallow learning. We formalized their mathematical proof using Isabelle/HOL. The Isabelle development simplifies and generalizes the original proof, while working around the limitations of the HOL type system. To support the formalization, we developed reusable libraries of formalized mathematics, including results about the matrix rank, the Borel measure, and multivariate polynomials as well as a library for tensor analysis

Label-Descriptive Patterns and their Application to Characterizing Classification Errors

State-of-the-art deep learning methods achieve human-like performance on many tasks, but make errors nevertheless. Characterizing these errors in easily interpretable terms gives insight into whether a model is prone to making systematic errors, but also gives a way to act and improve the model. In this paper we propose a method that allows us to do so for arbitrary classifiers by mining a small set of patterns that together succinctly describe the input data that is partitioned according to correctness of prediction. We show this is an instance of the more general label description problem, which we formulate in terms of the Minimum Description Length principle. To discover good pattern sets we propose the efficient and hyperparameter-free Premise algorithm, which through an extensive set of experiments we show on both synthetic and real-world data performs very well in practice; unlike existing solutions it ably recovers ground truth patterns, even on highly imbalanced data over many unique items, or where patterns are only weakly associated to labels. Through two real-world case studies we confirm that Premise gives clear and actionable insight into the systematic errors made by modern NLP classifiers

Wigner function quantum molecular dynamics

Classical molecular dynamics (MD) is a well established and powerful tool in various fields of science, e.g. chemistry, plasma physics, cluster physics and condensed matter physics. Objects of investigation are few-body systems and many-body systems as well. The broadness and level of sophistication of this technique is documented in many monographs and reviews, see for example \cite{Allan,Frenkel,mdhere}. Here we discuss the extension of MD to quantum systems (QMD). There have been many attempts in this direction which differ from one another, depending on the type of system under consideration. One direction of QMD has been developed for condensed matter systems and will not discussed here, e.g. \cite{fermid}. In this chapter we are dealing with unbound electrons as they occur in gases, fluids or plasmas. Here, one strategy is to replace classical point particles by wave packets, e.g. \cite{fermid,KTR94,zwicknagel06} which is quite successful. At the same time, this method struggles with problems related to the dispersion of such a packet and difficulties to properly describe strong electron-ion interaction and bound state formation. We, therefore, avoid such restrictions and consider a completely general alternative approach. We start discussion of quantum dynamics from a general consideration of quantum distribution functions.Comment: 18 pages, based on lecture at Hareaus school on computational phyics, Greifswald, September 200

Quasi-classical Molecular Dynamics Simulations of the Electron Gas: Dynamic properties

Results of quasi-classical molecular dynamics simulations of the quantum electron gas are reported. Quantum effects corresponding to the Pauli and the Heisenberg principle are modeled by an effective momentum-dependent Hamiltonian. The velocity autocorrelation functions and the dynamic structure factors have been computed. A comparison with theoretical predictions was performed.Comment: 8 figure

Numerical study of scars in a chaotic billiard

We study numerically the scaling properties of scars in stadium billiard. Using the semiclassical criterion, we have searched systematically the scars of the same type through a very wide range, from ground state to as high as the 1 millionth state. We have analyzed the integrated probability density along the periodic orbit. The numerical results confirm that the average intensity of certain types of scars is independent of $\hbar$ rather than scales with $\sqrt{\hbar}$. Our findings confirm the theoretical predictions of Robnik (1989).Comment: 7 pages in Revtex 3.1, 5 PS figures available upon request. To appear in Phys. Rev. E, Vol. 55, No. 5, 199

Temporal Feedback for Tweet Search with Non-Parametric Density Estimation

This paper investigates the temporal cluster hypothesis: in search tasks where time plays an important role, do relevant documents tend to cluster together in time? We explore this question in the context of tweet search and temporal feedback: starting with an initial set of results from a baseline retrieval model, we estimate the temporal density of relevant documents, which is then used for result reranking. Our contributions lie in a method to characterize this temporal density function using kernel density estimation, with and without human relevance judgments, and an approach to integrating this information into a standard retrieval model. Experiments on TREC datasets confirm that our temporal feedback formulation improves search effectiveness, thus providing support for our hypothesis. Our approach outperforms both a standard baseline and previous temporal retrieval models. Temporal feedback improves over standard lexical feedback (with and without human judgments), illustrating that temporal relevance signals exist independently of document content

Nodal domains statistics - a criterion for quantum chaos

We consider the distribution of the (properly normalized) numbers of nodal domains of wave functions in 2-$d$ quantum billiards. We show that these distributions distinguish clearly between systems with integrable (separable) or chaotic underlying classical dynamics, and for each case the limiting distribution is universal (system independent). Thus, a new criterion for quantum chaos is provided by the statistics of the wave functions, which complements the well established criterion based on spectral statistics.Comment: 4 pages, 5 figures, revte

Temperature-dependent quantum pair potentials and their application to dense partially ionized hydrogen plasmas

Extending our previous work \cite{filinov-etal.jpa03ik} we present a detailed discussion of accuracy and practical applications of finite-temperature pseudopotentials for two-component Coulomb systems. Different pseudopotentials are discussed: i) the diagonal Kelbg potential, ii) the off-diagonal Kelbg potential iii) the {\em improved} diagonal Kelbg potential, iv) an effective potential obtained with the Feynman-Kleinert variational principle v) the ``exact'' quantum pair potential derived from the two-particle density matrix. For the {\em improved} diagonal Kelbg potential a simple temperature dependent fit is derived which accurately reproduces the ``exact'' pair potential in the whole temperature range. The derived pseudopotentials are then used in path integral Monte Carlo (PIMC) and molecular dynamics (MD) simulations to obtain thermodynamical properties of strongly coupled hydrogen. It is demonstrated that classical MD simulations with spin-dependent interaction potentials for the electrons allow for an accurate description of the internal energy of hydrogen in the difficult regime of partial ionization down to the temperatures of about $60 000$ K. Finally, we point out an interesting relation between the quantum potentials and effective potentials used in density functional theory.Comment: 18 pages, 11 figure