1 research outputs found
Behavior-based evaluation of session satisfaction
Nowadays, web search becomes more and more popular all over the world. Many
researchers and developers have done lots of studies on behaviors of search
users. In practice, the full understanding of these behaviors can not only help
to evaluate the usefulness of newly-developed ranking algorithms and other
changes of search engine, but also to guide the growth direction of search
engine. As far as we know, most of past work are mainly focused on single
search evaluation, which do promote the rapid development of search engine in
early stage. However,these page-level behaviors are so limited that can no
longer give explicit feedbacks on minor changes of the search engine. We think
that it will be more accurate and sensitive when more information on search
session are provided. In this paper, a session level evaluation method is
proposed. The session-level features are retrieved and carefully analyzed. Some
linear and non-linear features which can reflect the final degree of
satisfaction are chosen and adopted in evaluation models. A two-layer hybrid
evaluation model with different granularity, which can achieve good precision
and recall, is designed and trained. Lots of real experiments are evaluated by
the model, the result shows it achieved a higher accuracy performance than
traditional page-level evaluation metrics. Furthermore, for practical
application, it is important to interpret the reason of each session's
satisfaction judgement. In all, a session-level evaluation model with improved
performance and well capability on interpretation is proposed and applied in
real practice in search engine companies