18 research outputs found
An Analysis of Mixed Initiative and Collaboration in Information-Seeking Dialogues
The ability to engage in mixed-initiative interaction is one of the core
requirements for a conversational search system. How to achieve this is poorly
understood. We propose a set of unsupervised metrics, termed ConversationShape,
that highlights the role each of the conversation participants plays by
comparing the distribution of vocabulary and utterance types. Using
ConversationShape as a lens, we take a closer look at several conversational
search datasets and compare them with other dialogue datasets to better
understand the types of dialogue interaction they represent, either driven by
the information seeker or the assistant. We discover that deviations from the
ConversationShape of a human-human dialogue of the same type is predictive of
the quality of a human-machine dialogue.Comment: SIGIR 2020 short conference pape
BERT with History Answer Embedding for Conversational Question Answering
Conversational search is an emerging topic in the information retrieval
community. One of the major challenges to multi-turn conversational search is
to model the conversation history to answer the current question. Existing
methods either prepend history turns to the current question or use complicated
attention mechanisms to model the history. We propose a conceptually simple yet
highly effective approach referred to as history answer embedding. It enables
seamless integration of conversation history into a conversational question
answering (ConvQA) model built on BERT (Bidirectional Encoder Representations
from Transformers). We first explain our view that ConvQA is a simplified but
concrete setting of conversational search, and then we provide a general
framework to solve ConvQA. We further demonstrate the effectiveness of our
approach under this framework. Finally, we analyze the impact of different
numbers of history turns under different settings to provide new insights into
conversation history modeling in ConvQA.Comment: Accepted to SIGIR 2019 as a short pape
User Intent Prediction in Information-seeking Conversations
Conversational assistants are being progressively adopted by the general
population. However, they are not capable of handling complicated
information-seeking tasks that involve multiple turns of information exchange.
Due to the limited communication bandwidth in conversational search, it is
important for conversational assistants to accurately detect and predict user
intent in information-seeking conversations. In this paper, we investigate two
aspects of user intent prediction in an information-seeking setting. First, we
extract features based on the content, structural, and sentiment
characteristics of a given utterance, and use classic machine learning methods
to perform user intent prediction. We then conduct an in-depth feature
importance analysis to identify key features in this prediction task. We find
that structural features contribute most to the prediction performance. Given
this finding, we construct neural classifiers to incorporate context
information and achieve better performance without feature engineering. Our
findings can provide insights into the important factors and effective methods
of user intent prediction in information-seeking conversations.Comment: Accepted to CHIIR 201
Open-Retrieval Conversational Question Answering
Conversational search is one of the ultimate goals of information retrieval.
Recent research approaches conversational search by simplified settings of
response ranking and conversational question answering, where an answer is
either selected from a given candidate set or extracted from a given passage.
These simplifications neglect the fundamental role of retrieval in
conversational search. To address this limitation, we introduce an
open-retrieval conversational question answering (ORConvQA) setting, where we
learn to retrieve evidence from a large collection before extracting answers,
as a further step towards building functional conversational search systems. We
create a dataset, OR-QuAC, to facilitate research on ORConvQA. We build an
end-to-end system for ORConvQA, featuring a retriever, a reranker, and a reader
that are all based on Transformers. Our extensive experiments on OR-QuAC
demonstrate that a learnable retriever is crucial for ORConvQA. We further show
that our system can make a substantial improvement when we enable history
modeling in all system components. Moreover, we show that the reranker
component contributes to the model performance by providing a regularization
effect. Finally, further in-depth analyses are performed to provide new
insights into ORConvQA.Comment: Accepted to SIGIR'2
Learning to Expand: Reinforced Pseudo-relevance Feedback Selection for Information-seeking Conversations
Intelligent personal assistant systems for information-seeking conversations
are increasingly popular in real-world applications, especially for e-commerce
companies. With the development of research in such conversation systems, the
pseudo-relevance feedback (PRF) has demonstrated its effectiveness in
incorporating relevance signals from external documents. However, the existing
studies are either based on heuristic rules or require heavy manual labeling.
In this work, we treat the PRF selection as a learning task and proposed a
reinforced learning based method that can be trained in an end-to-end manner
without any human annotations. More specifically, we proposed a reinforced
selector to extract useful PRF terms to enhance response candidates and a BERT
based response ranker to rank the PRF-enhanced responses. The performance of
the ranker serves as rewards to guide the selector to extract useful PRF terms,
and thus boost the task performance. Extensive experiments on both standard
benchmark and commercial datasets show the superiority of our reinforced PRF
term selector compared with other potential soft or hard selection methods.
Both qualitative case studies and quantitative analysis show that our model can
not only select meaningful PRF terms to expand response candidates but also
achieve the best results compared with all the baseline methods on a variety of
evaluation metrics. We have also deployed our method on online production in an
e-commerce company, which shows a significant improvement over the existing
online ranking system
Attentive History Selection for Conversational Question Answering
Conversational question answering (ConvQA) is a simplified but concrete
setting of conversational search. One of its major challenges is to leverage
the conversation history to understand and answer the current question. In this
work, we propose a novel solution for ConvQA that involves three aspects.
First, we propose a positional history answer embedding method to encode
conversation history with position information using BERT in a natural way.
BERT is a powerful technique for text representation. Second, we design a
history attention mechanism (HAM) to conduct a "soft selection" for
conversation histories. This method attends to history turns with different
weights based on how helpful they are on answering the current question. Third,
in addition to handling conversation history, we take advantage of multi-task
learning (MTL) to do answer prediction along with another essential
conversation task (dialog act prediction) using a uniform model architecture.
MTL is able to learn more expressive and generic representations to improve the
performance of ConvQA. We demonstrate the effectiveness of our model with
extensive experimental evaluations on QuAC, a large-scale ConvQA dataset. We
show that position information plays an important role in conversation history
modeling. We also visualize the history attention and provide new insights into
conversation history understanding.Comment: Accepted to CIKM 201