44,869 research outputs found
Generate-then-Retrieve: Intent-Aware FAQ Retrieval in Product Search
Customers interacting with product search engines are increasingly
formulating information-seeking queries. Frequently Asked Question (FAQ)
retrieval aims to retrieve common question-answer pairs for a user query with
question intent. Integrating FAQ retrieval in product search can not only
empower users to make more informed purchase decisions, but also enhance user
retention through efficient post-purchase support. Determining when an FAQ
entry can satisfy a user's information need within product search, without
disrupting their shopping experience, represents an important challenge. We
propose an intent-aware FAQ retrieval system consisting of (1) an intent
classifier that predicts when a user's information need can be answered by an
FAQ; (2) a reformulation model that rewrites a query into a natural question.
Offline evaluation demonstrates that our approach improves Hit@1 by 13% on
retrieving ground-truth FAQs, while reducing latency by 95% compared to
baseline systems. These improvements are further validated by real user
feedback, where 71% of displayed FAQs on top of product search results received
explicit positive user feedback. Overall, our findings show promising
directions for integrating FAQ retrieval into product search at scale.Comment: ACL 2023 Industry Trac
Recommended from our members
History Modeling for Conversational Information Retrieval
Conversational search is an embodiment of an iterative and interactive approach to information retrieval (IR) that has been studied for decades. Due to the recent rise of intelligent personal assistants, such as Siri, Alexa, AliMe, Cortana, and Google Assistant, a growing part of the population is moving their information-seeking activities to voice- or text-based conversational interfaces. One of the major challenges of conversational search is to leverage the conversation history to understand and fulfill the users\u27 information needs. In this dissertation work, we investigate history modeling approaches for conversational information retrieval. We start from history modeling for user intent prediction. We analyze information-seeking conversations by user intent distribution, co-occurrence, and flow patterns, followed by a study of user intent prediction in an information-seeking setting with both feature-based methods and deep learning methods. We then move to history modeling for conversational question answering (ConvQA), which can be considered as a simplified setting of conversational search. We first propose a positional history answer embedding (PosHAE) method to seamlessly integrate conversation history into a ConvQA model based on BERT. We then build upon this method and design a history attention mechanism (HAM) to conduct a ``soft selection\u27\u27 for conversation history. After this, we extend the previous ConvQA task to an open-retrieval (ORConvQA) setting to emphasize the fundamental role of retrieval in conversational search. In this setting, we learn to retrieve evidence from a large collection before extracting answers. We build an end-to-end system for ORConvQA, featuring a learnable dense retriever. We conduct experiments with both fully-supervised and weakly-supervised approaches to tackle the training challenges of ORConvQA. Finally, we study history modeling for conversational re-ranking. Given a history of user feedback behaviors, such as issuing a query, clicking a document, and skipping a document, we propose to introduce behavior awareness to a neural ranker. Our experimental results show that the history modeling approaches proposed in this dissertation can effectively improve the performance of different conversation tasks and provide new insights into conversational information retrieval
Ericson: An Interactive Open-Domain Conversational Search Agent
Open-domain conversational search (ODCS) aims to provide valuable, up-to-date
information, while maintaining natural conversations to help users refine and
ultimately answer information needs. However, creating an effective and robust
ODCS agent is challenging. In this paper, we present a fully functional ODCS
system, Ericson, which includes state-of-the-art question answering and
information retrieval components, as well as intent inference and dialogue
management models for proactive question refinement and recommendations. Our
system was stress-tested in the Amazon Alexa Prize, by engaging in live
conversations with thousands of Alexa users, thus providing empirical basis for
the analysis of the ODCS system in real settings. Our interaction data analysis
revealed that accurate intent classification, encouraging user engagement, and
careful proactive recommendations contribute most to the users satisfaction.
Our study further identifies limitations of the existing search techniques, and
can serve as a building block for the next generation of ODCS agents.Comment: pre-prin
User Intent Prediction in Information-seeking Conversations
Conversational assistants are being progressively adopted by the general
population. However, they are not capable of handling complicated
information-seeking tasks that involve multiple turns of information exchange.
Due to the limited communication bandwidth in conversational search, it is
important for conversational assistants to accurately detect and predict user
intent in information-seeking conversations. In this paper, we investigate two
aspects of user intent prediction in an information-seeking setting. First, we
extract features based on the content, structural, and sentiment
characteristics of a given utterance, and use classic machine learning methods
to perform user intent prediction. We then conduct an in-depth feature
importance analysis to identify key features in this prediction task. We find
that structural features contribute most to the prediction performance. Given
this finding, we construct neural classifiers to incorporate context
information and achieve better performance without feature engineering. Our
findings can provide insights into the important factors and effective methods
of user intent prediction in information-seeking conversations.Comment: Accepted to CHIIR 201
Evaluating Variable-Length Multiple-Option Lists in Chatbots and Mobile Search
In recent years, the proliferation of smart mobile devices has lead to the
gradual integration of search functionality within mobile platforms. This has
created an incentive to move away from the "ten blue links'' metaphor, as
mobile users are less likely to click on them, expecting to get the answer
directly from the snippets. In turn, this has revived the interest in Question
Answering. Then, along came chatbots, conversational systems, and messaging
platforms, where the user needs could be better served with the system asking
follow-up questions in order to better understand the user's intent. While
typically a user would expect a single response at any utterance, a system
could also return multiple options for the user to select from, based on
different system understandings of the user's intent. However, this possibility
should not be overused, as this practice could confuse and/or annoy the user.
How to produce good variable-length lists, given the conflicting objectives of
staying short while maximizing the likelihood of having a correct answer
included in the list, is an underexplored problem. It is also unclear how to
evaluate a system that tries to do that. Here we aim to bridge this gap. In
particular, we define some necessary and some optional properties that an
evaluation measure fit for this purpose should have. We further show that
existing evaluation measures from the IR tradition are not entirely suitable
for this setup, and we propose novel evaluation measures that address it
satisfactorily.Comment: 4 pages, in Proceeding of SIGIR 201
Towards evaluation of personalized and collaborative information retrieval
We propose to extend standard information retrieval (IR) ad-hoc test collection design to facilitate research on personalized and collaborative IR by gathering additional meta-information during the topic (query) development process. We propose a controlled query generation process with activity logging for each topic developer. The standard ad-hoc collection will thus be accompanied by a new set of thematically related topics and the associated log information, and has the potential to simulate a real-world search scenario to encourage retrieval systems to mine user information from the logs to improve IR effectiveness. The proposed methodology described in this paper will be applied in a pilot task which is scheduled to run in the FIRE 2011 evaluation campaign. The task aims at investigating the research question of whether personalized and collaborative IR retrieval experiments and evaluation can be pursued by enriching a standard ad-hoc collection with such meta-information
- …