59,994 research outputs found
Towards automated generation of scripted dialogue: some time-honoured strategies
The main aim of this paper is to introduce automated generation of scripted dialogue as a worthwhile topic of investigation. In particular the fact that scripted dialogue involves two layers of communication, i.e., uni-directional communication between the author and the audience of a scripted dialogue and bi-directional pretended communication between the characters featuring in the dialogue, is argued to raise some interesting issues. Our hope is that the combined study of the two layers will forge links between research in text generation and dialogue processing. The paper presents a first attempt at creating such links by studying three types of strategies for the automated generation of scripted dialogue. The strategies are derived from examples of human-authored and naturally occurring dialogue
Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generation
Automated metrics such as BLEU are widely used in the machine translation
literature. They have also been used recently in the dialogue community for
evaluating dialogue response generation. However, previous work in dialogue
response generation has shown that these metrics do not correlate strongly with
human judgment in the non task-oriented dialogue setting. Task-oriented
dialogue responses are expressed on narrower domains and exhibit lower
diversity. It is thus reasonable to think that these automated metrics would
correlate well with human judgment in the task-oriented setting where the
generation task consists of translating dialogue acts into a sentence. We
conduct an empirical study to confirm whether this is the case. Our findings
indicate that these automated metrics have stronger correlation with human
judgments in the task-oriented setting compared to what has been observed in
the non task-oriented setting. We also observe that these metrics correlate
even better for datasets which provide multiple ground truth reference
sentences. In addition, we show that some of the currently available corpora
for task-oriented language generation can be solved with simple models and
advocate for more challenging datasets
Constructing the CODA corpus: A parallel corpus ofmonologues and expository dialogues
We describe the construction of the CODA corpus, a parallel corpus of monologues and expository dialogues. The dialogue part of the corpus consists of expository, i.e., information-delivering rather than dramatic, dialogues written by several acclaimed authors. The monologue part of the corpus is a paraphrase in monologue form of these dialogues by a human annotator. The corpus was constructed as a resource for extracting rules for automated generation of dialogue from monologue. Using authored dialogues allows us to analyse the techniques used by accomplished writers for presenting information in the form of dialogue. The dialogues are annotated with dialogue acts and the monologues with rhetorical structure. We developed annotation and translation guidelines together with a custom-developed tool for carrying out translation, alignment and annotation
Attribute Controlled Dialogue Prompting
Prompt-tuning has become an increasingly popular parameter-efficient method
for adapting large pretrained language models to downstream tasks. However,
both discrete prompting and continuous prompting assume fixed prompts for all
data samples within a task, neglecting the fact that inputs vary greatly in
some tasks such as open-domain dialogue generation. In this paper, we present a
novel, instance-specific prompt-tuning algorithm for dialogue generation.
Specifically, we generate prompts based on instance-level control code, rather
than the conversation history, to explore their impact on controlled dialogue
generation. Experiments on popular open-domain dialogue datasets, evaluated on
both automated metrics and human evaluation, demonstrate that our method is
superior to prompting baselines and comparable to fine-tuning with only 5%-6%
of total parameters.Comment: Accepted at ACL 2023 In Finding
Automated Dialogue Generation for Behavior Intervention on Mobile Devices
AbstractCommunication in the form of dialogues between a virtual coach and a human patient (coachee) is one of the pillars in an intervention app for smartphones. The virtual coach is considered as a cooperative partner that supports the individual with various exercises for a behavior intervention therapy. To perform its supportive behavior, the coach follows a certain interaction model and its requirements, such as alignment, mutual commitment and adaptation. In this paper, we propose E-Coach MarkUp Language (ECML), a standard XML specification for scripting discourses that define how the virtual coach maintains a dialogue with a coachee following the interaction model. The format of the language allows messages to be tailored at a fine-grained level. Each sentence is synthesized based on the inferred goals of the coaching process and the current beliefs of the user, incorporating everything that has been said previously in the conversation. The design enables inexpensive implementation on mobile devices for a flexible, seamless coaching dialogue. With expert-based evaluations, we validated the language using scenarios on implemented ECML in the field of insomnia therapy
Correction of Errors in Preference Ratings from Automated Metrics for Text Generation
A major challenge in the field of Text Generation is evaluation: Human
evaluations are cost-intensive, and automated metrics often display
considerable disagreement with human judgments. In this paper, we propose a
statistical model of Text Generation evaluation that accounts for the
error-proneness of automated metrics when used to generate preference rankings
between system outputs. We show that existing automated metrics are generally
over-confident in assigning significant differences between systems in this
setting. However, our model enables an efficient combination of human and
automated ratings to remedy the error-proneness of the automated metrics. We
show that using this combination, we only require about 50% of the human
annotations typically used in evaluations to arrive at robust and statistically
significant results while yielding the same evaluation outcome as the pure
human evaluation in 95% of cases. We showcase the benefits of approach for
three text generation tasks: dialogue systems, machine translation, and text
summarization
A Virtual Conversational Agent for Teens with Autism: Experimental Results and Design Lessons
We present the design of an online social skills development interface for
teenagers with autism spectrum disorder (ASD). The interface is intended to
enable private conversation practice anywhere, anytime using a web-browser.
Users converse informally with a virtual agent, receiving feedback on nonverbal
cues in real-time, and summary feedback. The prototype was developed in
consultation with an expert UX designer, two psychologists, and a pediatrician.
Using the data from 47 individuals, feedback and dialogue generation were
automated using a hidden Markov model and a schema-driven dialogue manager
capable of handling multi-topic conversations. We conducted a study with nine
high-functioning ASD teenagers. Through a thematic analysis of post-experiment
interviews, identified several key design considerations, notably: 1) Users
should be fully briefed at the outset about the purpose and limitations of the
system, to avoid unrealistic expectations. 2) An interface should incorporate
positive acknowledgment of behavior change. 3) Realistic appearance of a
virtual agent and responsiveness are important in engaging users. 4)
Conversation personalization, for instance in prompting laconic users for more
input and reciprocal questions, would help the teenagers engage for longer
terms and increase the system's utility
Fully generated scripted dialogue for embodied agents
This paper presents the NECA approach to the generation of dialogues between Embodied Conversational Agents (ECAs). This approach consist of the automated construction of an abstract script for an entire dialogue (cast in terms of dialogue acts), which is incrementally enhanced by a series of modules and finally ''performed'' by means of text, speech and body language, by a cast of ECAs. The approach makes it possible to automatically produce a large variety of highly expressive dialogues, some of whose essential properties are under the control of a user. The paper discusses the advantages and disadvantages of NECA's approach to Fully Generated Scripted Dialogue (FGSD), and explains the main techniques used in the two demonstrators that were built. The paper can be read as a survey of issues and techniques in the construction of ECAs, focusing on the generation of behaviour (i.e., focusing on information presentation) rather than on interpretation
- …