85 research outputs found
Gen-IR @ SIGIR 2023: The First Workshop on Generative Information Retrieval
Generative information retrieval (IR) has experienced substantial growth
across multiple research communities (e.g., information retrieval, computer
vision, natural language processing, and machine learning), and has been highly
visible in the popular press. Theoretical, empirical, and actual user-facing
products have been released that retrieve documents (via generation) or
directly generate answers given an input request. We would like to investigate
whether end-to-end generative models are just another trend or, as some claim,
a paradigm change for IR. This necessitates new metrics, theoretical grounding,
evaluation methods, task definitions, models, user interfaces, etc. The goal of
this workshop (https://coda.io/@sigir/gen-ir) is to focus on previously
explored Generative IR techniques like document retrieval and direct Grounded
Answer Generation, while also offering a venue for the discussion and
exploration of how Generative IR can be applied to new domains like
recommendation systems, summarization, etc. The format of the workshop is
interactive, including roundtable and keynote sessions and tends to avoid the
one-sided dialogue of a mini-conference.Comment: Accepted SIGIR 23 worksho
Visual Named Entity Linking: A New Dataset and A Baseline
Visual Entity Linking (VEL) is a task to link regions of images with their
corresponding entities in Knowledge Bases (KBs), which is beneficial for many
computer vision tasks such as image retrieval, image caption, and visual
question answering. While existing tasks in VEL either rely on textual data to
complement a multi-modal linking or only link objects with general entities,
which fails to perform named entity linking on large amounts of image data. In
this paper, we consider a purely Visual-based Named Entity Linking (VNEL) task,
where the input only consists of an image. The task is to identify objects of
interest (i.e., visual entity mentions) in images and link them to
corresponding named entities in KBs. Since each entity often contains rich
visual and textual information in KBs, we thus propose three different
sub-tasks, i.e., visual to visual entity linking (V2VEL), visual to textual
entity linking (V2TEL), and visual to visual-textual entity linking (V2VTEL).
In addition, we present a high-quality human-annotated visual person linking
dataset, named WIKIPerson. Based on WIKIPerson, we establish a series of
baseline algorithms for the solution of each sub-task, and conduct experiments
to verify the quality of proposed datasets and the effectiveness of baseline
methods. We envision this work to be helpful for soliciting more works
regarding VNEL in the future. The codes and datasets are publicly available at
https://github.com/ict-bigdatalab/VNEL.Comment: 13 pages, 11 figures, published to EMNLP 2022(findings
On the Robustness of Generative Retrieval Models: An Out-of-Distribution Perspective
Recently, we have witnessed generative retrieval increasingly gaining
attention in the information retrieval (IR) field, which retrieves documents by
directly generating their identifiers. So far, much effort has been devoted to
developing effective generative retrieval models. There has been less attention
paid to the robustness perspective. When a new retrieval paradigm enters into
the real-world application, it is also critical to measure the
out-of-distribution (OOD) generalization, i.e., how would generative retrieval
models generalize to new distributions. To answer this question, firstly, we
define OOD robustness from three perspectives in retrieval problems: 1) The
query variations; 2) The unforeseen query types; and 3) The unforeseen tasks.
Based on this taxonomy, we conduct empirical studies to analyze the OOD
robustness of several representative generative retrieval models against dense
retrieval models. The empirical results indicate that the OOD robustness of
generative retrieval models requires enhancement. We hope studying the OOD
robustness of generative retrieval models would be advantageous to the IR
community.Comment: 4 pages, submit to GenIR2
Inducing Causal Structure for Abstractive Text Summarization
The mainstream of data-driven abstractive summarization models tends to
explore the correlations rather than the causal relationships. Among such
correlations, there can be spurious ones which suffer from the language prior
learned from the training corpus and therefore undermine the overall
effectiveness of the learned model. To tackle this issue, we introduce a
Structural Causal Model (SCM) to induce the underlying causal structure of the
summarization data. We assume several latent causal factors and non-causal
factors, representing the content and style of the document and summary.
Theoretically, we prove that the latent factors in our SCM can be identified by
fitting the observed training data under certain conditions. On the basis of
this, we propose a Causality Inspired Sequence-to-Sequence model (CI-Seq2Seq)
to learn the causal representations that can mimic the causal factors, guiding
us to pursue causal information for summary generation. The key idea is to
reformulate the Variational Auto-encoder (VAE) to fit the joint distribution of
the document and summary variables from the training corpus. Experimental
results on two widely used text summarization datasets demonstrate the
advantages of our approach
Learning to Truncate Ranked Lists for Information Retrieval
Ranked list truncation is of critical importance in a variety of professional
information retrieval applications such as patent search or legal search. The
goal is to dynamically determine the number of returned documents according to
some user-defined objectives, in order to reach a balance between the overall
utility of the results and user efforts. Existing methods formulate this task
as a sequential decision problem and take some pre-defined loss as a proxy
objective, which suffers from the limitation of local decision and non-direct
optimization. In this work, we propose a global decision based truncation model
named AttnCut, which directly optimizes user-defined objectives for the ranked
list truncation. Specifically, we take the successful transformer architecture
to capture the global dependency within the ranked list for truncation
decision, and employ the reward augmented maximum likelihood (RAML) for direct
optimization. We consider two types of user-defined objectives which are of
practical usage. One is the widely adopted metric such as F1 which acts as a
balanced objective, and the other is the best F1 under some minimal recall
constraint which represents a typical objective in professional search.
Empirical results over the Robust04 and MQ2007 datasets demonstrate the
effectiveness of our approach as compared with the state-of-the-art baselines
PRADA: Practical Black-Box Adversarial Attacks against Neural Ranking Models
Neural ranking models (NRMs) have shown remarkable success in recent years,
especially with pre-trained language models. However, deep neural models are
notorious for their vulnerability to adversarial examples. Adversarial attacks
may become a new type of web spamming technique given our increased reliance on
neural information retrieval models. Therefore, it is important to study
potential adversarial attacks to identify vulnerabilities of NRMs before they
are deployed.
In this paper, we introduce the Adversarial Document Ranking Attack (ADRA)
task against NRMs, which aims to promote a target document in rankings by
adding adversarial perturbations to its text. We focus on the decision-based
black-box attack setting, where the attackers have no access to the model
parameters and gradients, but can only acquire the rank positions of the
partial retrieved list by querying the target model. This attack setting is
realistic in real-world search engines. We propose a novel Pseudo
Relevance-based ADversarial ranking Attack method (PRADA) that learns a
surrogate model based on Pseudo Relevance Feedback (PRF) to generate gradients
for finding the adversarial perturbations.
Experiments on two web search benchmark datasets show that PRADA can
outperform existing attack strategies and successfully fool the NRM with small
indiscernible perturbations of text
- …