Location of Repository

The relationship of word error rate to document ranking

By X. Mang Shou, M. Sanderson and N. Tuffs

Abstract

This paper describes two experiments that examine the relationship of Word Error Rate (WER) of retrieved\ud spoken documents returned by a spoken document retrieval system. Previous work has demonstrated that\ud recognition errors do not significantly affect retrieval effectiveness but whether they will adversely affect\ud relevance judgement remains unclear. A user-based experiment measuring ability to judge relevance from\ud the recognised text presented in a retrieved result list was conducted. The results indicated that users were\ud capable of judging relevance accurately despite transcription errors. This lead an examination of the\ud relationship of WER in retrieved audio documents to their rank position when retrieved for a particular\ud query. Here it was shown that WER was somewhat lower for top ranked documents than it was for\ud documents retrieved further down the ranking, thereby indicating a possible explanation for the success of\ud the user experiment

Year: 2004
OAI identifier: oai:eprints.whiterose.ac.uk:4710

Suggested articles

Preview


To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.