2,697 research outputs found
Offline and Online Satisfaction Prediction in Open-Domain Conversational Systems
Predicting user satisfaction in conversational systems has become critical,
as spoken conversational assistants operate in increasingly complex domains.
Online satisfaction prediction (i.e., predicting satisfaction of the user with
the system after each turn) could be used as a new proxy for implicit user
feedback, and offers promising opportunities to create more responsive and
effective conversational agents, which adapt to the user's engagement with the
agent. To accomplish this goal, we propose a conversational satisfaction
prediction model specifically designed for open-domain spoken conversational
agents, called ConvSAT. To operate robustly across domains, ConvSAT aggregates
multiple representations of the conversation, namely the conversation history,
utterance and response content, and system- and user-oriented behavioral
signals. We first calibrate ConvSAT performance against state of the art
methods on a standard dataset (Dialogue Breakdown Detection Challenge) in an
online regime, and then evaluate ConvSAT on a large dataset of conversations
with real users, collected as part of the Alexa Prize competition. Our
experimental results show that ConvSAT significantly improves satisfaction
prediction for both offline and online setting on both datasets, compared to
the previously reported state-of-the-art approaches. The insights from our
study can enable more intelligent conversational systems, which could adapt in
real-time to the inferred user satisfaction and engagement.Comment: Published in CIKM '19, 10 page
Quantifying the Effects of Prosody Modulation on User Engagement and Satisfaction in Conversational Systems
As voice-based assistants such as Alexa, Siri, and Google Assistant become
ubiquitous, users increasingly expect to maintain natural and informative
conversations with such systems. However, for an open-domain conversational
system to be coherent and engaging, it must be able to maintain the user's
interest for extended periods, without sounding boring or annoying. In this
paper, we investigate one natural approach to this problem, of modulating
response prosody, i.e., changing the pitch and cadence of the response to
indicate delight, sadness or other common emotions, as well as using
pre-recorded interjections. Intuitively, this approach should improve the
naturalness of the conversation, but attempts to quantify the effects of
prosodic modulation on user satisfaction and engagement remain challenging. To
accomplish this, we report results obtained from a large-scale empirical study
that measures the effects of prosodic modulation on user behavior and
engagement across multiple conversation domains, both immediately after each
turn, and at the overall conversation level. Our results indicate that the
prosody modulation significantly increases both immediate and overall user
satisfaction. However, since the effects vary across different domains, we
verify that prosody modulations do not substitute for coherent, informative
content of the responses. Together, our results provide useful tools and
insights for improving the naturalness of responses in conversational systems.Comment: Published in CHIIR 2020, 4 page
A Survey on Asking Clarification Questions Datasets in Conversational Systems
The ability to understand a user's underlying needs is critical for conversational systems, especially with limited input from users in a conversation. Thus, in such a domain, Asking Clarification Questions (ACQs) to reveal users' true intent from their queries or utterances arise as an essential task. However, it is noticeable that a key limitation of the existing ACQs studies is their incomparability, from inconsistent use of data, distinct experimental setups and evaluation strategies. Therefore, in this paper, to assist the development of ACQs techniques, we comprehensively analyse the current ACQs research status, which offers a detailed comparison of publicly available datasets, and discusses the applied evaluation metrics, joined with benchmarks for multiple ACQs-related tasks. In particular, given a thorough analysis of the ACQs task, we discuss a number of corresponding research directions for the investigation of ACQs as well as the development of conversational systems
Multi-Purpose NLP Chatbot : Design, Methodology & Conclusion
With a major focus on its history, difficulties, and promise, this research
paper provides a thorough analysis of the chatbot technology environment as it
exists today. It provides a very flexible chatbot system that makes use of
reinforcement learning strategies to improve user interactions and
conversational experiences. Additionally, this system makes use of sentiment
analysis and natural language processing to determine user moods. The chatbot
is a valuable tool across many fields thanks to its amazing characteristics,
which include voice-to-voice conversation, multilingual support [12], advising
skills, offline functioning, and quick help features. The complexity of chatbot
technology development is also explored in this study, along with the causes
that have propelled these developments and their far-reaching effects on a
range of sectors. According to the study, three crucial elements are crucial:
1) Even without explicit profile information, the chatbot system is built to
adeptly understand unique consumer preferences and fluctuating satisfaction
levels. With the use of this capacity, user interactions are made to meet their
wants and preferences. 2) Using a complex method that interlaces Multiview
voice chat information, the chatbot may precisely simulate users' actual
experiences. This aids in developing more genuine and interesting discussions.
3) The study presents an original method for improving the black-box deep
learning models' capacity for prediction. This improvement is made possible by
introducing dynamic satisfaction measurements that are theory-driven, which
leads to more precise forecasts of consumer reaction.Comment: Multilingual , Voice Conversion , Emotion Recognition , Offline
Service , Financial Advisor , Product Preference , Customer Reaction
Predictio
Rating Prediction in Conversational Task Assistants with Behavioral and Conversational-Flow Features
Predicting the success of Conversational Task Assistants (CTA) can be
critical to understand user behavior and act accordingly. In this paper, we
propose TB-Rater, a Transformer model which combines conversational-flow
features with user behavior features for predicting user ratings in a CTA
scenario. In particular, we use real human-agent conversations and ratings
collected in the Alexa TaskBot challenge, a novel multimodal and multi-turn
conversational context. Our results show the advantages of modeling both the
conversational-flow and behavioral aspects of the conversation in a single
model for offline rating prediction. Additionally, an analysis of the
CTA-specific behavioral features brings insights into this setting and can be
used to bootstrap future systems
- …