20 research outputs found
Deeper Text Understanding for IR with Contextual Neural Language Modeling
Neural networks provide new possibilities to automatically learn complex
language patterns and query-document relations. Neural IR models have achieved
promising results in learning query-document relevance patterns, but few
explorations have been done on understanding the text content of a query or a
document. This paper studies leveraging a recently-proposed contextual neural
language model, BERT, to provide deeper text understanding for IR. Experimental
results demonstrate that the contextual text representations from BERT are more
effective than traditional word embeddings. Compared to bag-of-words retrieval
models, the contextual language model can better leverage language structures,
bringing large improvements on queries written in natural languages. Combining
the text understanding ability with search knowledge leads to an enhanced
pre-trained BERT model that can benefit related search tasks where training
data are limited.Comment: In proceedings of SIGIR 201
A Transformer-based Embedding Model for Personalized Product Search
Product search is an important way for people to browse and purchase items on
E-commerce platforms. While customers tend to make choices based on their
personal tastes and preferences, analysis of commercial product search logs has
shown that personalization does not always improve product search quality. Most
existing product search techniques, however, conduct undifferentiated
personalization across search sessions. They either use a fixed coefficient to
control the influence of personalization or let personalization take effect all
the time with an attention mechanism. The only notable exception is the
recently proposed zero-attention model (ZAM) that can adaptively adjust the
effect of personalization by allowing the query to attend to a zero vector.
Nonetheless, in ZAM, personalization can act at most as equally important as
the query and the representations of items are static across the collection
regardless of the items co-occurring in the user's historical purchases. Aware
of these limitations, we propose a transformer-based embedding model (TEM) for
personalized product search, which could dynamically control the influence of
personalization by encoding the sequence of query and user's purchase history
with a transformer architecture. Personalization could have a dominant impact
when necessary and interactions between items can be taken into consideration
when computing attention weights. Experimental results show that TEM
outperforms state-of-the-art personalization product retrieval models
significantly.Comment: In the proceedings of SIGIR 202
Table Search Using a Deep Contextualized Language Model
Pretrained contextualized language models such as BERT have achieved
impressive results on various natural language processing benchmarks.
Benefiting from multiple pretraining tasks and large scale training corpora,
pretrained models can capture complex syntactic word relations. In this paper,
we use the deep contextualized language model BERT for the task of ad hoc table
retrieval. We investigate how to encode table content considering the table
structure and input length limit of BERT. We also propose an approach that
incorporates features from prior literature on table retrieval and jointly
trains them with BERT. In experiments on public datasets, we show that our best
approach can outperform the previous state-of-the-art method and BERT baselines
with a large margin under different evaluation metrics.Comment: Accepted at SIGIR 2020 (Long
Efficient Document Re-Ranking for Transformers by Precomputing Term Representations
Deep pretrained transformer networks are effective at various ranking tasks,
such as question answering and ad-hoc document ranking. However, their
computational expenses deem them cost-prohibitive in practice. Our proposed
approach, called PreTTR (Precomputing Transformer Term Representations),
considerably reduces the query-time latency of deep transformer networks (up to
a 42x speedup on web document ranking) making these networks more practical to
use in a real-time ranking scenario. Specifically, we precompute part of the
document term representations at indexing time (without a query), and merge
them with the query representation at query time to compute the final ranking
score. Due to the large size of the token representations, we also propose an
effective approach to reduce the storage requirement by training a compression
layer to match attention scores. Our compression technique reduces the storage
required up to 95% and it can be applied without a substantial degradation in
ranking performance.Comment: Accepted at SIGIR 2020 (long
A Feature Analysis for Multimodal News Retrieval
Content-based information retrieval is based on the information contained in
documents rather than using metadata such as keywords. Most information
retrieval methods are either based on text or image. In this paper, we
investigate the usefulness of multimodal features for cross-lingual news search
in various domains: politics, health, environment, sport, and finance. To this
end, we consider five feature types for image and text and compare the
performance of the retrieval system using different combinations. Experimental
results show that retrieval results can be improved when considering both
visual and textual information. In addition, it is observed that among textual
features entity overlap outperforms word embeddings, while geolocation
embeddings achieve better performance among visual features in the retrieval
task.Comment: CLEOPATRA Workshop co-located with ESWC 202
A Graph-based Relevance Matching Model for Ad-hoc Retrieval
To retrieve more relevant, appropriate and useful documents given a query,
finding clues about that query through the text is crucial. Recent deep
learning models regard the task as a term-level matching problem, which seeks
exact or similar query patterns in the document. However, we argue that they
are inherently based on local interactions and do not generalise to ubiquitous,
non-consecutive contextual relationships. In this work, we propose a novel
relevance matching model based on graph neural networks to leverage the
document-level word relationships for ad-hoc retrieval. In addition to the
local interactions, we explicitly incorporate all contexts of a term through
the graph-of-word text format. Matching patterns can be revealed accordingly to
provide a more accurate relevance score. Our approach significantly outperforms
strong baselines on two ad-hoc benchmarks. We also experimentally compare our
model with BERT and show our advantages on long documents.Comment: To appear at AAAI 202