4 research outputs found
Retrieval-based Goal-Oriented Dialogue Generation
Most research on dialogue has focused either on dialogue generation for
openended chit chat or on state tracking for goal-directed dialogue. In this
work, we explore a hybrid approach to goal-oriented dialogue generation that
combines retrieval from past history with a hierarchical, neural
encoder-decoder architecture. We evaluate this approach in the customer support
domain using the Multiwoz dataset (Budzianowski et al., 2018). We show that
adding this retrieval step to a hierarchical, neural encoder-decoder
architecture leads to significant improvements, including responses that are
rated more appropriate and fluent by human evaluators. Finally, we compare our
retrieval-based model to various semantically conditioned models explicitly
using past dialog act information, and find that our proposed model is
competitive with the current state of the art (Chen et al., 2019), while not
requiring explicit labels about past machine acts
Entertaining and Opinionated but Too Controlling: A Large-Scale User Study of an Open Domain Alexa Prize System
Conversational systems typically focus on functional tasks such as scheduling
appointments or creating todo lists. Instead we design and evaluate SlugBot
(SB), one of 8 semifinalists in the 2018 AlexaPrize, whose goal is to support
casual open-domain social inter-action. This novel application requires both
broad topic coverage and engaging interactive skills. We developed a new
technical approach to meet this demanding situation by crowd-sourcing novel
content and introducing playful conversational strategies based on storytelling
and games. We collected over 10,000 conversations during August 2018 as part of
the Alexa Prize competition. We also conducted an in-lab follow-up qualitative
evaluation. Over-all users found SB moderately engaging; conversations averaged
3.6 minutes and involved 26 user turns. However, users reacted very differently
to different conversation subtypes. Storytelling and games were evaluated
positively; these were seen as entertaining with predictable interactive
structure. They also led users to impute personality and intelligence to SB. In
contrast, search and general Chit-Chat induced coverage problems; here users
found it hard to infer what topics SB could understand, with these
conversations seen as being too system-driven. Theoretical and design
implications suggest a move away from conversational systems that simply
provide factual information. Future systems should be designed to have their
own opinions with personal stories to share, and SB provides an example of how
we might achieve this.Comment: To appear in 1st International Conference on Conversational User
Interfaces (CUI 2019