3,808 research outputs found
Consistency and Variation in Kernel Neural Ranking Model
This paper studies the consistency of the kernel-based neural ranking model
K-NRM, a recent state-of-the-art neural IR model, which is important for
reproducible research and deployment in the industry. We find that K-NRM has
low variance on relevance-based metrics across experimental trials. In spite of
this low variance in overall performance, different trials produce different
document rankings for individual queries. The main source of variance in our
experiments was found to be different latent matching patterns captured by
K-NRM. In the IR-customized word embeddings learned by K-NRM, the
query-document word pairs follow two different matching patterns that are
equally effective, but align word pairs differently in the embedding space. The
different latent matching patterns enable a simple yet effective approach to
construct ensemble rankers, which improve K-NRM's effectiveness and
generalization abilities.Comment: 4 pages, 4 figures, 2 table
Text Understanding with the Attention Sum Reader Network
Several large cloze-style context-question-answer datasets have been
introduced recently: the CNN and Daily Mail news data and the Children's Book
Test. Thanks to the size of these datasets, the associated text comprehension
task is well suited for deep-learning techniques that currently seem to
outperform all alternative approaches. We present a new, simple model that uses
attention to directly pick the answer from the context as opposed to computing
the answer using a blended representation of words in the document as is usual
in similar models. This makes the model particularly suitable for
question-answering problems where the answer is a single word from the
document. Ensemble of our models sets new state of the art on all evaluated
datasets.Comment: Presented at ACL 201
On Multi-Relational Link Prediction with Bilinear Models
We study bilinear embedding models for the task of multi-relational link
prediction and knowledge graph completion. Bilinear models belong to the most
basic models for this task, they are comparably efficient to train and use, and
they can provide good prediction performance. The main goal of this paper is to
explore the expressiveness of and the connections between various bilinear
models proposed in the literature. In particular, a substantial number of
models can be represented as bilinear models with certain additional
constraints enforced on the embeddings. We explore whether or not these
constraints lead to universal models, which can in principle represent every
set of relations, and whether or not there are subsumption relationships
between various models. We report results of an independent experimental study
that evaluates recent bilinear models in a common experimental setup. Finally,
we provide evidence that relation-level ensembles of multiple bilinear models
can achieve state-of-the art prediction performance
- …