12 research outputs found
Attentive History Selection for Conversational Question Answering
Conversational question answering (ConvQA) is a simplified but concrete
setting of conversational search. One of its major challenges is to leverage
the conversation history to understand and answer the current question. In this
work, we propose a novel solution for ConvQA that involves three aspects.
First, we propose a positional history answer embedding method to encode
conversation history with position information using BERT in a natural way.
BERT is a powerful technique for text representation. Second, we design a
history attention mechanism (HAM) to conduct a "soft selection" for
conversation histories. This method attends to history turns with different
weights based on how helpful they are on answering the current question. Third,
in addition to handling conversation history, we take advantage of multi-task
learning (MTL) to do answer prediction along with another essential
conversation task (dialog act prediction) using a uniform model architecture.
MTL is able to learn more expressive and generic representations to improve the
performance of ConvQA. We demonstrate the effectiveness of our model with
extensive experimental evaluations on QuAC, a large-scale ConvQA dataset. We
show that position information plays an important role in conversation history
modeling. We also visualize the history attention and provide new insights into
conversation history understanding.Comment: Accepted to CIKM 201
Conversational Question Answering over Passages by Leveraging Word Proximity Networks
Question answering (QA) over text passages is a problem of long-standing
interest in information retrieval. Recently, the conversational setting has
attracted attention, where a user asks a sequence of questions to satisfy her
information needs around a topic. While this setup is a natural one and similar
to humans conversing with each other, it introduces two key research
challenges: understanding the context left implicit by the user in follow-up
questions, and dealing with ad hoc question formulations. In this work, we
demonstrate CROWN (Conversational passage ranking by Reasoning Over Word
Networks): an unsupervised yet effective system for conversational QA with
passage responses, that supports several modes of context propagation over
multiple turns. To this end, CROWN first builds a word proximity network (WPN)
from large corpora to store statistically significant term co-occurrences. At
answering time, passages are ranked by a combination of their similarity to the
question, and coherence of query terms within: these factors are measured by
reading off node and edge weights from the WPN. CROWN provides an interface
that is both intuitive for end-users, and insightful for experts for
reconfiguration to individual setups. CROWN was evaluated on TREC CAsT data,
where it achieved above-median performance in a pool of neural methods.Comment: SIGIR 2020 Demonstration
Open-Retrieval Conversational Question Answering
Conversational search is one of the ultimate goals of information retrieval.
Recent research approaches conversational search by simplified settings of
response ranking and conversational question answering, where an answer is
either selected from a given candidate set or extracted from a given passage.
These simplifications neglect the fundamental role of retrieval in
conversational search. To address this limitation, we introduce an
open-retrieval conversational question answering (ORConvQA) setting, where we
learn to retrieve evidence from a large collection before extracting answers,
as a further step towards building functional conversational search systems. We
create a dataset, OR-QuAC, to facilitate research on ORConvQA. We build an
end-to-end system for ORConvQA, featuring a retriever, a reranker, and a reader
that are all based on Transformers. Our extensive experiments on OR-QuAC
demonstrate that a learnable retriever is crucial for ORConvQA. We further show
that our system can make a substantial improvement when we enable history
modeling in all system components. Moreover, we show that the reranker
component contributes to the model performance by providing a regularization
effect. Finally, further in-depth analyses are performed to provide new
insights into ORConvQA.Comment: Accepted to SIGIR'2
Query Resolution for Conversational Search with Limited Supervision
In this work we focus on multi-turn passage retrieval as a crucial component
of conversational search. One of the key challenges in multi-turn passage
retrieval comes from the fact that the current turn query is often
underspecified due to zero anaphora, topic change, or topic return. Context
from the conversational history can be used to arrive at a better expression of
the current turn query, defined as the task of query resolution. In this paper,
we model the query resolution task as a binary term classification problem: for
each term appearing in the previous turns of the conversation decide whether to
add it to the current turn query or not. We propose QuReTeC (Query Resolution
by Term Classification), a neural query resolution model based on bidirectional
transformers. We propose a distant supervision method to automatically generate
training data by using query-passage relevance labels. Such labels are often
readily available in a collection either as human annotations or inferred from
user interactions. We show that QuReTeC outperforms state-of-the-art models,
and furthermore, that our distant supervision method can be used to
substantially reduce the amount of human-curated data required to train
QuReTeC. We incorporate QuReTeC in a multi-turn, multi-stage passage retrieval
architecture and demonstrate its effectiveness on the TREC CAsT dataset.Comment: SIGIR 2020 full conference pape
NORMY: Non-Uniform History Modeling for Open Retrieval Conversational Question Answering
Open Retrieval Conversational Question Answering (OrConvQA) answers a
question given a conversation as context and a document collection. A typical
OrConvQA pipeline consists of three modules: a Retriever to retrieve relevant
documents from the collection, a Reranker to rerank them given the question and
the context, and a Reader to extract an answer span. The conversational turns
can provide valuable context to answer the final query. State-of-the-art
OrConvQA systems use the same history modeling for all three modules of the
pipeline. We hypothesize this as suboptimal. Specifically, we argue that a
broader context is needed in the first modules of the pipeline to not miss
relevant documents, while a narrower context is needed in the last modules to
identify the exact answer span. We propose NORMY, the first unsupervised
non-uniform history modeling pipeline which generates the best conversational
history for each module. We further propose a novel Retriever for NORMY, which
employs keyphrase extraction on the conversation history, and leverages
passages retrieved in previous turns as additional context. We also created a
new dataset for OrConvQA, by expanding the doc2dial dataset. We implemented
various state-of-the-art history modeling techniques and comprehensively
evaluated them separately for each module of the pipeline on three datasets:
OR-QUAC, our doc2dial extension, and ConvMix. Our extensive experiments show
that NORMY outperforms the state-of-the-art in the individual modules and in
the end-to-end system.Comment: Accepted for publication at IEEE ICSC 202