2,027 research outputs found
Evaluating Competing Agent Strategies for a Voice Email Agent
This paper reports experimental results comparing a mixed-initiative to a
system-initiative dialog strategy in the context of a personal voice email
agent. To independently test the effects of dialog strategy and user expertise,
users interact with either the system-initiative or the mixed-initiative agent
to perform three successive tasks which are identical for both agents. We
report performance comparisons across agent strategies as well as over tasks.
This evaluation utilizes and tests the PARADISE evaluation framework, and
discusses the performance function derivable from the experimental data.Comment: 6 pages latex, uses icassp91.sty, psfi
PARADISE: A Framework for Evaluating Spoken Dialogue Agents
This paper presents PARADISE (PARAdigm for DIalogue System Evaluation), a
general framework for evaluating spoken dialogue agents. The framework
decouples task requirements from an agent's dialogue behaviors, supports
comparisons among dialogue strategies, enables the calculation of performance
over subdialogues and whole dialogues, specifies the relative contribution of
various factors to performance, and makes it possible to compare agents
performing different tasks by normalizing for task complexity.Comment: 10 pages, uses aclap, psfig, lingmacros, time
Agents for educational games and simulations
This book consists mainly of revised papers that were presented at the Agents for Educational Games and Simulation (AEGS) workshop held on May 2, 2011, as part of the Autonomous Agents and MultiAgent Systems (AAMAS) conference in Taipei, Taiwan. The 12 full papers presented were carefully reviewed and selected from various submissions. The papers are organized topical sections on middleware applications, dialogues and learning, adaption and convergence, and agent applications
Ethical Challenges in Data-Driven Dialogue Systems
The use of dialogue systems as a medium for human-machine interaction is an
increasingly prevalent paradigm. A growing number of dialogue systems use
conversation strategies that are learned from large datasets. There are well
documented instances where interactions with these system have resulted in
biased or even offensive conversations due to the data-driven training process.
Here, we highlight potential ethical issues that arise in dialogue systems
research, including: implicit biases in data-driven systems, the rise of
adversarial examples, potential sources of privacy violations, safety concerns,
special considerations for reinforcement learning systems, and reproducibility
concerns. We also suggest areas stemming from these issues that deserve further
investigation. Through this initial survey, we hope to spur research leading to
robust, safe, and ethically sound dialogue systems.Comment: In Submission to the AAAI/ACM conference on Artificial Intelligence,
Ethics, and Societ
The BURCHAK corpus: a Challenge Data Set for Interactive Learning of Visually Grounded Word Meanings
We motivate and describe a new freely available human-human dialogue dataset
for interactive learning of visually grounded word meanings through ostensive
definition by a tutor to a learner. The data has been collected using a novel,
character-by-character variant of the DiET chat tool (Healey et al., 2003;
Mills and Healey, submitted) with a novel task, where a Learner needs to learn
invented visual attribute words (such as " burchak " for square) from a tutor.
As such, the text-based interactions closely resemble face-to-face conversation
and thus contain many of the linguistic phenomena encountered in natural,
spontaneous dialogue. These include self-and other-correction, mid-sentence
continuations, interruptions, overlaps, fillers, and hedges. We also present a
generic n-gram framework for building user (i.e. tutor) simulations from this
type of incremental data, which is freely available to researchers. We show
that the simulations produce outputs that are similar to the original data
(e.g. 78% turn match similarity). Finally, we train and evaluate a
Reinforcement Learning dialogue control agent for learning visually grounded
word meanings, trained from the BURCHAK corpus. The learned policy shows
comparable performance to a rule-based system built previously.Comment: 10 pages, THE 6TH WORKSHOP ON VISION AND LANGUAGE (VL'17
- …