110 research outputs found
Axiomatic thinking for information retrieval: introduction to special issue
4siopenopenAmigo E.; Fang H.; Mizzaro S.; Zhai C.Amigo, E.; Fang, H.; Mizzaro, S.; Zhai, C
Explainable Information Retrieval: A Survey
Explainable information retrieval is an emerging research area aiming to make
transparent and trustworthy information retrieval systems. Given the increasing
use of complex machine learning models in search systems, explainability is
essential in building and auditing responsible information retrieval models.
This survey fills a vital gap in the otherwise topically diverse literature of
explainable information retrieval. It categorizes and discusses recent
explainability methods developed for different application domains in
information retrieval, providing a common framework and unifying perspectives.
In addition, it reflects on the common concern of evaluating explanations and
highlights open challenges and opportunities.Comment: 35 pages, 10 figures. Under revie
How Different are Pre-trained Transformers for Text Ranking?
In recent years, large pre-trained transformers have led to substantial gains
in performance over traditional retrieval models and feedback approaches.
However, these results are primarily based on the MS Marco/TREC Deep Learning
Track setup, with its very particular setup, and our understanding of why and
how these models work better is fragmented at best. We analyze effective
BERT-based cross-encoders versus traditional BM25 ranking for the passage
retrieval task where the largest gains have been observed, and investigate two
main questions. On the one hand, what is similar? To what extent does the
neural ranker already encompass the capacity of traditional rankers? Is the
gain in performance due to a better ranking of the same documents (prioritizing
precision)? On the other hand, what is different? Can it retrieve effectively
documents missed by traditional systems (prioritizing recall)? We discover
substantial differences in the notion of relevance identifying strengths and
weaknesses of BERT that may inspire research for future improvement. Our
results contribute to our understanding of (black-box) neural rankers relative
to (well-understood) traditional rankers, help understand the particular
experimental setting of MS-Marco-based test collections.Comment: ECIR 202
An Intrinsic Framework of Information Retrieval Evaluation Measures
Information retrieval (IR) evaluation measures are cornerstones for
determining the suitability and task performance efficiency of retrieval
systems. Their metric and scale properties enable to compare one system against
another to establish differences or similarities. Based on the representational
theory of measurement, this paper determines these properties by exploiting the
information contained in a retrieval measure itself. It establishes the
intrinsic framework of a retrieval measure, which is the common scenario when
the domain set is not explicitly specified. A method to determine the metric
and scale properties of any retrieval measure is provided, requiring knowledge
of only some of its attained values. The method establishes three main
categories of retrieval measures according to their intrinsic properties. Some
common user-oriented and system-oriented evaluation measures are classified
according to the presented taxonomy.Comment: 23 page
Information Retrieval: Recent Advances and Beyond
In this paper, we provide a detailed overview of the models used for
information retrieval in the first and second stages of the typical processing
chain. We discuss the current state-of-the-art models, including methods based
on terms, semantic retrieval, and neural. Additionally, we delve into the key
topics related to the learning process of these models. This way, this survey
offers a comprehensive understanding of the field and is of interest for for
researchers and practitioners entering/working in the information retrieval
domain
- …