14 research outputs found
Deeper Text Understanding for IR with Contextual Neural Language Modeling
Neural networks provide new possibilities to automatically learn complex
language patterns and query-document relations. Neural IR models have achieved
promising results in learning query-document relevance patterns, but few
explorations have been done on understanding the text content of a query or a
document. This paper studies leveraging a recently-proposed contextual neural
language model, BERT, to provide deeper text understanding for IR. Experimental
results demonstrate that the contextual text representations from BERT are more
effective than traditional word embeddings. Compared to bag-of-words retrieval
models, the contextual language model can better leverage language structures,
bringing large improvements on queries written in natural languages. Combining
the text understanding ability with search knowledge leads to an enhanced
pre-trained BERT model that can benefit related search tasks where training
data are limited.Comment: In proceedings of SIGIR 201
End-to-End Neural Ad-hoc Ranking with Kernel Pooling
This paper proposes K-NRM, a kernel based neural model for document ranking.
Given a query and a set of documents, K-NRM uses a translation matrix that
models word-level similarities via word embeddings, a new kernel-pooling
technique that uses kernels to extract multi-level soft match features, and a
learning-to-rank layer that combines those features into the final ranking
score. The whole model is trained end-to-end. The ranking layer learns desired
feature patterns from the pairwise ranking loss. The kernels transfer the
feature patterns into soft-match targets at each similarity level and enforce
them on the translation matrix. The word embeddings are tuned accordingly so
that they can produce the desired soft matches. Experiments on a commercial
search engine's query log demonstrate the improvements of K-NRM over prior
feature-based and neural-based states-of-the-art, and explain the source of
K-NRM's advantage: Its kernel-guided embedding encodes a similarity metric
tailored for matching query words to document words, and provides effective
multi-level soft matches
Consistency and Variation in Kernel Neural Ranking Model
This paper studies the consistency of the kernel-based neural ranking model
K-NRM, a recent state-of-the-art neural IR model, which is important for
reproducible research and deployment in the industry. We find that K-NRM has
low variance on relevance-based metrics across experimental trials. In spite of
this low variance in overall performance, different trials produce different
document rankings for individual queries. The main source of variance in our
experiments was found to be different latent matching patterns captured by
K-NRM. In the IR-customized word embeddings learned by K-NRM, the
query-document word pairs follow two different matching patterns that are
equally effective, but align word pairs differently in the embedding space. The
different latent matching patterns enable a simple yet effective approach to
construct ensemble rankers, which improve K-NRM's effectiveness and
generalization abilities.Comment: 4 pages, 4 figures, 2 table
Multi-Vector Retrieval as Sparse Alignment
Multi-vector retrieval models improve over single-vector dual encoders on
many information retrieval tasks. In this paper, we cast the multi-vector
retrieval problem as sparse alignment between query and document tokens. We
propose AligneR, a novel multi-vector retrieval model that learns sparsified
pairwise alignments between query and document tokens (e.g. `dog' vs. `puppy')
and per-token unary saliences reflecting their relative importance for
retrieval. We show that controlling the sparsity of pairwise token alignments
often brings significant performance gains. While most factoid questions
focusing on a specific part of a document require a smaller number of
alignments, others requiring a broader understanding of a document favor a
larger number of alignments. Unary saliences, on the other hand, decide whether
a token ever needs to be aligned with others for retrieval (e.g. `kind' from
`kind of currency is used in new zealand}'). With sparsified unary saliences,
we are able to prune a large number of query and document token vectors and
improve the efficiency of multi-vector retrieval. We learn the sparse unary
saliences with entropy-regularized linear programming, which outperforms other
methods to achieve sparsity. In a zero-shot setting, AligneR scores 51.1 points
nDCG@10, achieving a new retriever-only state-of-the-art on 13 tasks in the
BEIR benchmark. In addition, adapting pairwise alignments with a few examples
(<= 8) further improves the performance up to 15.7 points nDCG@10 for argument
retrieval tasks. The unary saliences of AligneR helps us to keep only 20% of
the document token representations with minimal performance loss. We further
show that our model often produces interpretable alignments and significantly
improves its performance when initialized from larger language models