182 research outputs found
A Deep Relevance Matching Model for Ad-hoc Retrieval
In recent years, deep neural networks have led to exciting breakthroughs in
speech recognition, computer vision, and natural language processing (NLP)
tasks. However, there have been few positive results of deep models on ad-hoc
retrieval tasks. This is partially due to the fact that many important
characteristics of the ad-hoc retrieval task have not been well addressed in
deep models yet. Typically, the ad-hoc retrieval task is formalized as a
matching problem between two pieces of text in existing work using deep models,
and treated equivalent to many NLP tasks such as paraphrase identification,
question answering and automatic conversation. However, we argue that the
ad-hoc retrieval task is mainly about relevance matching while most NLP
matching tasks concern semantic matching, and there are some fundamental
differences between these two matching tasks. Successful relevance matching
requires proper handling of the exact matching signals, query term importance,
and diverse matching requirements. In this paper, we propose a novel deep
relevance matching model (DRMM) for ad-hoc retrieval. Specifically, our model
employs a joint deep architecture at the query term level for relevance
matching. By using matching histogram mapping, a feed forward matching network,
and a term gating network, we can effectively deal with the three relevance
matching factors mentioned above. Experimental results on two representative
benchmark collections show that our model can significantly outperform some
well-known retrieval models as well as state-of-the-art deep matching models.Comment: CIKM 2016, long pape
MatchZoo: A Learning, Practicing, and Developing System for Neural Text Matching
Text matching is the core problem in many natural language processing (NLP)
tasks, such as information retrieval, question answering, and conversation.
Recently, deep leaning technology has been widely adopted for text matching,
making neural text matching a new and active research domain. With a large
number of neural matching models emerging rapidly, it becomes more and more
difficult for researchers, especially those newcomers, to learn and understand
these new models. Moreover, it is usually difficult to try these models due to
the tedious data pre-processing, complicated parameter configuration, and
massive optimization tricks, not to mention the unavailability of public codes
sometimes. Finally, for researchers who want to develop new models, it is also
not an easy task to implement a neural text matching model from scratch, and to
compare with a bunch of existing models. In this paper, therefore, we present a
novel system, namely MatchZoo, to facilitate the learning, practicing and
designing of neural text matching models. The system consists of a powerful
matching library and a user-friendly and interactive studio, which can help
researchers: 1) to learn state-of-the-art neural text matching models
systematically, 2) to train, test and apply these models with simple
configurable steps; and 3) to develop their own models with rich APIs and
assistance
Consistency and Variation in Kernel Neural Ranking Model
This paper studies the consistency of the kernel-based neural ranking model
K-NRM, a recent state-of-the-art neural IR model, which is important for
reproducible research and deployment in the industry. We find that K-NRM has
low variance on relevance-based metrics across experimental trials. In spite of
this low variance in overall performance, different trials produce different
document rankings for individual queries. The main source of variance in our
experiments was found to be different latent matching patterns captured by
K-NRM. In the IR-customized word embeddings learned by K-NRM, the
query-document word pairs follow two different matching patterns that are
equally effective, but align word pairs differently in the embedding space. The
different latent matching patterns enable a simple yet effective approach to
construct ensemble rankers, which improve K-NRM's effectiveness and
generalization abilities.Comment: 4 pages, 4 figures, 2 table
DE-PACRR: Exploring Layers Inside the PACRR Model
Recent neural IR models have demonstrated deep learning's utility in ad-hoc
information retrieval. However, deep models have a reputation for being black
boxes, and the roles of a neural IR model's components may not be obvious at
first glance. In this work, we attempt to shed light on the inner workings of a
recently proposed neural IR model, namely the PACRR model, by visualizing the
output of intermediate layers and by investigating the relationship between
intermediate weights and the ultimate relevance score produced. We highlight
several insights, hoping that such insights will be generally applicable.Comment: Neu-IR 2017 SIGIR Workshop on Neural Information Retrieva
- …