10 research outputs found
Multi-Hop Paragraph Retrieval for Open-Domain Question Answering
This paper is concerned with the task of multi-hop open-domain Question
Answering (QA). This task is particularly challenging since it requires the
simultaneous performance of textual reasoning and efficient searching. We
present a method for retrieving multiple supporting paragraphs, nested amidst a
large knowledge base, which contain the necessary evidence to answer a given
question. Our method iteratively retrieves supporting paragraphs by forming a
joint vector representation of both a question and a paragraph. The retrieval
is performed by considering contextualized sentence-level representations of
the paragraphs in the knowledge source. Our method achieves state-of-the-art
performance over two well-known datasets, SQuAD-Open and HotpotQA, which serve
as our single- and multi-hop open-domain QA benchmarks, respectively.Comment: ACL 201
Fine-tuning Multi-hop Question Answering with Hierarchical Graph Network
In this paper, we present a two stage model for multi-hop question answering.
The first stage is a hierarchical graph network, which is used to reason over
multi-hop question and is capable to capture different levels of granularity
using the nature structure(i.e., paragraphs, questions, sentences and entities)
of documents. The reasoning process is convert to node classify task(i.e.,
paragraph nodes and sentences nodes). The second stage is a language model
fine-tuning task. In a word, stage one use graph neural network to select and
concatenate support sentences as one paragraph, and stage two find the answer
span in language model fine-tuning paradigm.Comment: the experience result is not as good as I excep
Rethinking Label Smoothing on Multi-hop Question Answering
Multi-Hop Question Answering (MHQA) is a significant area in question
answering, requiring multiple reasoning components, including document
retrieval, supporting sentence prediction, and answer span extraction. In this
work, we analyze the primary factors limiting the performance of multi-hop
reasoning and introduce label smoothing into the MHQA task. This is aimed at
enhancing the generalization capabilities of MHQA systems and mitigating
overfitting of answer spans and reasoning paths in training set. We propose a
novel label smoothing technique, F1 Smoothing, which incorporates uncertainty
into the learning process and is specifically tailored for Machine Reading
Comprehension (MRC) tasks. Inspired by the principles of curriculum learning,
we introduce the Linear Decay Label Smoothing Algorithm (LDLA), which
progressively reduces uncertainty throughout the training process. Experiment
on the HotpotQA dataset demonstrates the effectiveness of our methods in
enhancing performance and generalizability in multi-hop reasoning, achieving
new state-of-the-art results on the leaderboard.Comment: 13 pages, 8 figures, accepted by CCL202
Performance Prediction for Multi-hop Questions
We study the problem of Query Performance Prediction (QPP) for open-domain
multi-hop Question Answering (QA), where the task is to estimate the difficulty
of evaluating a multi-hop question over a corpus. Despite the extensive
research on predicting the performance of ad-hoc and QA retrieval models, there
has been a lack of study on the estimation of the difficulty of multi-hop
questions. The problem is challenging due to the multi-step nature of the
retrieval process, potential dependency of the steps and the reasoning
involved. To tackle this challenge, we propose multHP, a novel pre-retrieval
method for predicting the performance of open-domain multi-hop questions. Our
extensive evaluation on the largest multi-hop QA dataset using several modern
QA systems shows that the proposed model is a strong predictor of the
performance, outperforming traditional single-hop QPP models. Additionally, we
demonstrate that our approach can be effectively used to optimize the
parameters of QA systems, such as the number of documents to be retrieved,
resulting in improved overall retrieval performance.Comment: 10 page
Information Retrieval: Recent Advances and Beyond
In this paper, we provide a detailed overview of the models used for
information retrieval in the first and second stages of the typical processing
chain. We discuss the current state-of-the-art models, including methods based
on terms, semantic retrieval, and neural. Additionally, we delve into the key
topics related to the learning process of these models. This way, this survey
offers a comprehensive understanding of the field and is of interest for for
researchers and practitioners entering/working in the information retrieval
domain
Ranking and Retrieval under Semantic Relevance
This thesis presents a series of conceptual and empirical developments on the ranking and retrieval of candidates under semantic relevance. Part I of the thesis introduces the concept of uncertainty in various semantic tasks (such as recognizing textual entailment) in natural language processing, and the machine learning techniques commonly employed to model these semantic phenomena. A unified view of ranking and retrieval will be presented, and the trade-off between model expressiveness, performance, and scalability in model design will be discussed.
Part II of the thesis focuses on applying these ranking and retrieval techniques to text: Chapter 3 examines the feasibility of ranking hypotheses given a premise with respect to a human's subjective probability of the hypothesis happening, effectively extending the traditional categorical task of natural language inference. Chapter 4 focuses on detecting situation frames for documents using ranking methods. Then we extend the ranking notion to retrieval, and develop both sparse (Chapter 5) and dense (Chapter 6) vector-based methods to facilitate scalable retrieval for potential answer paragraphs in question answering.
Part III turns the focus to mentions and entities in text, while continuing the theme on ranking and retrieval: Chapter 7 discusses the ranking of fine-grained types that an entity mention could belong to, leading to state-of-the-art performance on hierarchical multi-label fine-grained entity typing. Chapter 8 extends the semantic relation of coreference to a cross-document setting, enabling models to retrieve from a large corpus, instead of in a single document, when resolving coreferent entity mentions