Search engine plays a crucial role in satisfying users' diverse information
needs. Recently, Pretrained Language Models (PLMs) based text ranking models
have achieved huge success in web search. However, many state-of-the-art text
ranking approaches only focus on core relevance while ignoring other dimensions
that contribute to user satisfaction, e.g., document quality, recency,
authority, etc. In this work, we focus on ranking user satisfaction rather than
relevance in web search, and propose a PLM-based framework, namely SAT-Ranker,
which comprehensively models different dimensions of user satisfaction in a
unified manner. In particular, we leverage the capacities of PLMs on both
textual and numerical inputs, and apply a multi-field input that modularizes
each dimension of user satisfaction as an input field. Overall, SAT-Ranker is
an effective, extensible, and data-centric framework that has huge potential
for industrial applications. On rigorous offline and online experiments,
SAT-Ranker obtains remarkable gains on various evaluation sets targeting
different dimensions of user satisfaction. It is now fully deployed online to
improve the usability of our search engine