Bayesian video search reranking

Abstract

Content-based video search reranking can be regarded as a process that uses visual content to recover the “true ” rank-ing list from the noisy one generated based on textual in-formation. This paper explicitly formulates this problem in the Bayesian framework, i.e., maximizing the ranking score consistency among visually similar video shots while mini-mizing the ranking distance, which represents the disagree-ment between the objective ranking list and the initial text-based. Different from existing point-wise ranking distance measures, which compute the distance in terms of the indi-vidual scores, two new methods are proposed in this paper to measure the ranking distance based on the disagreement in terms of pair-wise orders. Specifically, hinge distance pe-nalizes the pairs with reversed order according to the de-gree of the reverse, while preference strength distance further considers the preference degree. By incorporating the pro-posed distances into the optimization objective, two rerank-ing methods are developed which are solved using quadratic programming and matrix computation respectively. Evalu-ation on TRECVID video search benchmark shows that the performance improvement up to 21 % on TRECVID 2006 and 61.11 % on TRECVID 2007 are achieved relative to text search baseline