6,010 research outputs found
Top K Relevant Passage Retrieval for Biomedical Question Answering
Question answering is a task that answers factoid questions using a large
collection of documents. It aims to provide precise answers in response to the
user's questions in natural language. Question answering relies on efficient
passage retrieval to select candidate contexts, where traditional sparse vector
space models, such as TF-IDF or BM25, are the de facto method. On the web,
there is no single article that could provide all the possible answers
available on the internet to the question of the problem asked by the user. The
existing Dense Passage Retrieval model has been trained on Wikipedia dump from
Dec. 20, 2018, as the source documents for answering questions. Question
answering (QA) has made big strides with several open-domain and machine
comprehension systems built using large-scale annotated datasets. However, in
the clinical domain, this problem remains relatively unexplored. According to
multiple surveys, Biomedical Questions cannot be answered correctly from
Wikipedia Articles. In this work, we work on the existing DPR framework for the
biomedical domain and retrieve answers from the Pubmed articles which is a
reliable source to answer medical questions. When evaluated on a BioASQ QA
dataset, our fine-tuned dense retriever results in a 0.81 F1 score.Comment: 6 pages, 5 figures. arXiv admin note: text overlap with
arXiv:2004.04906 by other author
KGI: An Integrated Framework for Knowledge Intensive Language Tasks
In a recent work, we presented a novel state-of-the-art approach to zero-shot
slot filling that extends dense passage retrieval with hard negatives and
robust training procedures for retrieval augmented generation models. In this
paper, we propose a system based on an enhanced version of this approach where
we train task specific models for other knowledge intensive language tasks,
such as open domain question answering (QA), dialogue and fact checking. Our
system achieves results comparable to the best models in the KILT leaderboards.
Moreover, given a user query, we show how the output from these different
models can be combined to cross-examine each other. Particularly, we show how
accuracy in dialogue can be improved using the QA model. A short video
demonstrating the system is available here -
\url{https://ibm.box.com/v/kgi-interactive-demo}
Strong and Efficient Baselines for Open Domain Conversational Question Answering
Unlike the Open Domain Question Answering (ODQA) setting, the conversational
(ODConvQA) domain has received limited attention when it comes to reevaluating
baselines for both efficiency and effectiveness. In this paper, we study the
State-of-the-Art (SotA) Dense Passage Retrieval (DPR) retriever and
Fusion-in-Decoder (FiD) reader pipeline, and show that it significantly
underperforms when applied to ODConvQA tasks due to various limitations. We
then propose and evaluate strong yet simple and efficient baselines, by
introducing a fast reranking component between the retriever and the reader,
and by performing targeted finetuning steps. Experiments on two ODConvQA tasks,
namely TopiOCQA and OR-QuAC, show that our method improves the SotA results,
while reducing reader's latency by 60%. Finally, we provide new and valuable
insights into the development of challenging baselines that serve as a
reference for future, more intricate approaches, including those that leverage
Large Language Models (LLMs).Comment: Accepted to EMNLP 2023 Finding
Open-Retrieval Conversational Question Answering
Conversational search is one of the ultimate goals of information retrieval.
Recent research approaches conversational search by simplified settings of
response ranking and conversational question answering, where an answer is
either selected from a given candidate set or extracted from a given passage.
These simplifications neglect the fundamental role of retrieval in
conversational search. To address this limitation, we introduce an
open-retrieval conversational question answering (ORConvQA) setting, where we
learn to retrieve evidence from a large collection before extracting answers,
as a further step towards building functional conversational search systems. We
create a dataset, OR-QuAC, to facilitate research on ORConvQA. We build an
end-to-end system for ORConvQA, featuring a retriever, a reranker, and a reader
that are all based on Transformers. Our extensive experiments on OR-QuAC
demonstrate that a learnable retriever is crucial for ORConvQA. We further show
that our system can make a substantial improvement when we enable history
modeling in all system components. Moreover, we show that the reranker
component contributes to the model performance by providing a regularization
effect. Finally, further in-depth analyses are performed to provide new
insights into ORConvQA.Comment: Accepted to SIGIR'2
- …