4,729 research outputs found
Performance Prediction for Multi-hop Questions
We study the problem of Query Performance Prediction (QPP) for open-domain
multi-hop Question Answering (QA), where the task is to estimate the difficulty
of evaluating a multi-hop question over a corpus. Despite the extensive
research on predicting the performance of ad-hoc and QA retrieval models, there
has been a lack of study on the estimation of the difficulty of multi-hop
questions. The problem is challenging due to the multi-step nature of the
retrieval process, potential dependency of the steps and the reasoning
involved. To tackle this challenge, we propose multHP, a novel pre-retrieval
method for predicting the performance of open-domain multi-hop questions. Our
extensive evaluation on the largest multi-hop QA dataset using several modern
QA systems shows that the proposed model is a strong predictor of the
performance, outperforming traditional single-hop QPP models. Additionally, we
demonstrate that our approach can be effectively used to optimize the
parameters of QA systems, such as the number of documents to be retrieved,
resulting in improved overall retrieval performance.Comment: 10 page
Question Answering over Curated and Open Web Sources
The last few years have seen an explosion of research on the topic of automated question answering (QA), spanning the communities of information retrieval, natural language processing, and artificial intelligence. This tutorial would cover the highlights of this really active period of growth for QA to give the audience a grasp over the families of algorithms that are currently being used. We partition research contributions by the underlying source from where answers are retrieved: curated knowledge graphs, unstructured text, or hybrid corpora. We choose this dimension of partitioning as it is the most discriminative when it comes to algorithm design. Other key dimensions are covered within each sub-topic: like the complexity of questions addressed, and degrees of explainability and interactivity introduced in the systems. We would conclude the tutorial with the most promising emerging trends in the expanse of QA, that would help new entrants into this field make the best decisions to take the community forward. Much has changed in the community since the last tutorial on QA in SIGIR 2016, and we believe that this timely overview will indeed benefit a large number of conference participants
- …