We describe our experience with WiQA 2006, a pilot task aimed at studying question answering using Wikipedia. Going beyond traditional factoid questions, the task considered at WiQA 2006 was to identify—given an source article from Wikipedia—snippets from other Wikipedia articles, possibly in languages different from the language of the source article, that add new and important information to the source article, and that do so without repetition. A total of 7 teams took part, submitting 20 runs. Our main findings are two-fold: (i) while challenging, the tasks considered at WiQA are do-able as participants achieved precision@10 scores in the.5 range and MRR scores upwards of.5; (ii) on the bilingual task, substantially higher scores were achieved than on the monolingual tasks. is simply a great resource. It is something we as researchers want to work with, and contribute to, both by facilitating access to it, and, as the distinction between readers and authors has become blurred, by creating tools to support the authoring process. One of the aims of the WiQA 2006 pilot was to set up a challenging, but do-able and measurable information access task using Wikipedia. Another was to experiment with different measures for evaluation within this setting. In this overview we first provide a description of the task we selected for the pilot and of the evaluation and assessment procedures (Section 2). After that we describe the runs submitted by the participants (Section 3) and then we detail the results (Section 4). We end with conclusions (Section 5)
To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.