Unbiased learning to rank (ULTR) aims to mitigate various biases existing in
user clicks, such as position bias, trust bias, presentation bias, and learn an
effective ranker. In this paper, we introduce our winning approach for the
"Unbiased Learning to Rank" task in WSDM Cup 2023. We find that the provided
data is severely biased so neural models trained directly with the top 10
results with click information are unsatisfactory. So we extract multiple
heuristic-based features for multi-fields of the results, adjust the click
labels, add true negatives, and re-weight the samples during model training.
Since the propensities learned by existing ULTR methods are not decreasing
w.r.t. positions, we also calibrate the propensities according to the click
ratios and ensemble the models trained in two different ways. Our method won
the 3rd prize with a DCG@10 score of 9.80, which is 1.1% worse than the 2nd and
25.3% higher than the 4th.Comment: 5 pages, 1 figure, WSDM Cup 202