Traditional measures of search success often overlook the varying information
needs of different demographic groups. To address this gap, we introduce a
novel metric, named Group-aware Search Success (GA-SS). GA-SS redefines search
success to ensure that all demographic groups achieve satisfaction from search
outcomes. We introduce a comprehensive mathematical framework to calculate
GA-SS, incorporating both static and stochastic ranking policies and
integrating user browsing models for a more accurate assessment. In addition,
we have proposed Group-aware Most Popular Completion (gMPC) ranking model to
account for demographic variances in user intent, aligning more closely with
the diverse needs of all user groups. We empirically validate our metric and
approach with two real-world datasets: one focusing on query auto-completion
and the other on movie recommendations, where the results highlight the impact
of stochasticity and the complex interplay among various search success
metrics. Our findings advocate for a more inclusive approach in measuring
search success, as well as inspiring future investigations into the quality of
service of search