Differences in Video Search Behavior between Novices and Archivists

Abstract. Improving the user’s interaction with a video retrieval system requires to examine the search behavior of real users. We present in this article a study comparing the video search behavior between professional archivists and novice users. The comparison focuses on the use and effectiveness of different state-of-the-art video search methods offered by our retrieval system, and the result investigation behavior of the two user groups. We conducted our experiments in the context of TRECVID’s 2009 interactive search task, using the provided collection and topics for our evaluation. The findings are based on a qualitative questionnaire analysis and a quantitative examination of the logged user actions on the search interface. The experimental results indicate that today’s visual search techniques have improved in effectiveness, confirming a trend found in previous user studies. To our surprise, professional archivists used visual concept search in many of their searches. Queries containing visual concepts were more effective, resulting in more relevant shots found than the alternative methods. Overall, we conclude that professional archivists are more focused on recall in carrying out their search tasks, and are better at reflecting on their own search performance.