TREC 2007 CiQA Track at RMIT and CSIRO

following three research questions: 1. What contextual words are helpful in improving answer quality? 2. Given two answer lists of different quality, which list would a user prefer? 3. Would a user’s preference choice be correlated with her own relevance judgement of an individual list? To explore these questions, we submitted: • Four system runs with various query formulation strategies; • Two interactive runs, with one interface for the preference choice, and the other one for the relevance judgement of each answer sentence from an answer list. 2. Experiments 2.1. Two initial runs We used the Indri index and search tools from the Lemur toolkit for all our system runs. When the collection was indexed, words were stemmed using the Krovetz stemmer, and words from the stoplist were removed. We chose a language model with Jelinek-Mercer smoothing (σ = 0.5) to weight and rank documents (Zhai and Lafferty, 2001)