4 research outputs found
Ranking-Incentivized Quality Preserving Content Modification
The Web is a canonical example of a competitive retrieval setting where many
documents' authors consistently modify their documents to promote them in
rankings. We present an automatic method for quality-preserving modification of
document content -- i.e., maintaining content quality -- so that the document
is ranked higher for a query by a non-disclosed ranking function whose rankings
can be observed. The method replaces a passage in the document with some other
passage. To select the two passages, we use a learning-to-rank approach with a
bi-objective optimization criterion: rank promotion and content-quality
maintenance. We used the approach as a bot in content-based ranking
competitions. Analysis of the competitions demonstrates the merits of our
approach with respect to human content modifications in terms of rank
promotion, content-quality maintenance and relevance.Comment: 10 pages. 8 figures. 3 table
PRADA: Practical Black-Box Adversarial Attacks against Neural Ranking Models
Neural ranking models (NRMs) have shown remarkable success in recent years,
especially with pre-trained language models. However, deep neural models are
notorious for their vulnerability to adversarial examples. Adversarial attacks
may become a new type of web spamming technique given our increased reliance on
neural information retrieval models. Therefore, it is important to study
potential adversarial attacks to identify vulnerabilities of NRMs before they
are deployed.
In this paper, we introduce the Adversarial Document Ranking Attack (ADRA)
task against NRMs, which aims to promote a target document in rankings by
adding adversarial perturbations to its text. We focus on the decision-based
black-box attack setting, where the attackers have no access to the model
parameters and gradients, but can only acquire the rank positions of the
partial retrieved list by querying the target model. This attack setting is
realistic in real-world search engines. We propose a novel Pseudo
Relevance-based ADversarial ranking Attack method (PRADA) that learns a
surrogate model based on Pseudo Relevance Feedback (PRF) to generate gradients
for finding the adversarial perturbations.
Experiments on two web search benchmark datasets show that PRADA can
outperform existing attack strategies and successfully fool the NRM with small
indiscernible perturbations of text
Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Models
Neural text ranking models have witnessed significant advancement and are
increasingly being deployed in practice. Unfortunately, they also inherit
adversarial vulnerabilities of general neural models, which have been detected
but remain underexplored by prior studies. Moreover, the inherit adversarial
vulnerabilities might be leveraged by blackhat SEO to defeat better-protected
search engines. In this study, we propose an imitation adversarial attack on
black-box neural passage ranking models. We first show that the target passage
ranking model can be transparentized and imitated by enumerating critical
queries/candidates and then train a ranking imitation model. Leveraging the
ranking imitation model, we can elaborately manipulate the ranking results and
transfer the manipulation attack to the target ranking model. For this purpose,
we propose an innovative gradient-based attack method, empowered by the
pairwise objective function, to generate adversarial triggers, which causes
premeditated disorderliness with very few tokens. To equip the trigger
camouflages, we add the next sentence prediction loss and the language model
fluency constraint to the objective function. Experimental results on passage
ranking demonstrate the effectiveness of the ranking imitation attack model and
adversarial triggers against various SOTA neural ranking models. Furthermore,
various mitigation analyses and human evaluation show the effectiveness of
camouflages when facing potential mitigation approaches. To motivate other
scholars to further investigate this novel and important problem, we make the
experiment data and code publicly available.Comment: 15 pages, 4 figures, accepted by ACM CCS 2022, Best Paper Nominatio