Using Virtual Clinicians to Promote Functional Communication Skills in Aphasia

Persons with aphasia (PWA) re-enter their community after their rehabilitation program is ended. Thus it is incumbent on rehabilitation specialists to incorporate training in using residual language skills for functional communication [1]. Evidence indicates that language abilities improve with continued treatment, even during chronic stages of aphasia (refs) For optimal generalization, PWA need to practice language in everyday living situations. Virtual reality technology is a method of providing home-based therapeutic interventions. A valuable potential of virtual reality technology is that it supports the successful generalization of residual language skills to functional communication situations. Traditionally, role-playing [2] and script training [3] have been used to improve functional communication in PWA. A more recent approach has been the adaptation of scripts through the implementation of virtual technology. [4]. We report progress on a project that aims to develop a virtual clinician that is capable of recognizing a variety of potential responses in the context of functional communication scenarios. Our goal is to develop a virtual clinician-human interaction system that can be used independently by PWA to practice and improve communication skills. This involves development of software that will support a spoken dialog system (SDS) that can interact autonomously with an individual and can be configured to personalize treatment [5]. As use of virtual technology in aphasia rehabilitation increases, questions about the physical and psychosocial factors that influence successful use of residual communication skills need to be resolved. Thus, a second aim of this project, the topic of this paper, is to determine whether interactive dialogues between a client and virtual clinician differ in the quantity and quality of the client’s language output compared to dialogues between client and human clinician. Although the potential of using virtual clinicians is promising, it must be determined if individuals with aphasia (or other language disorder) will be responsive to the virtual clinician and produce as much language in this context as they would during dialogues with human clinicians. We addressed two hypotheses in this study: 1. For PWA, practice with dialogues that focus on everyday activities will improve quality and quantity of verbal output in those dialogues. 2. For PWA, verbal output practiced in dialogues with a virtual clinician and a human clinician will yield similar amounts of verbal output as measured by information units in the dialogues

The Interpretation of Non-Sentential Utterances in Dialogue

Schlangen D, Lascarides A. The Interpretation of Non-Sentential Utterances in Dialogue. In: Rudnicky A, ed. Proceedings of the 4th SIGdial workshop on Discourse and Dialogue. Sapporo, Japan; 2003

Integrating Multiple Knowledge Sources for Utterance-Level Confidence Annotation in the CMU Communicator Spoken Dialog System

In the recent years, automated speech recognition has been the main drive behind the advent of spoken language interfaces, but at the same time a severe limiting factor in the development of these systems. We believe that increased robustness in the face of recognition errors can be achieved by making the systems aware of their own misunderstandings, and employing appropriate recovery techniques when breakdowns in interaction occur. In this paper we address the first problem: the development of an utterance-level confidence annotator for a spoken dialog system. After a brief introduction to the CMU Communicator spoken dialog system (which provided the target platform for the developed annotator), we cast the confidence annotation problem as a machine learning classification task, and focus on selecting relevant features and on empirically identifying the best classification techniques for this task. The results indicate that significant reductions in classification error rate can be obtained using several di#erent classifiers. Furthermore, we propose a data driven approach to assessing the impact of the errors committed by the confidence annotator on dialog performance, with a view to optimally fine-tuning the annotator. Several models were constructed, and the resulting error costs were in accordance with our intuition. We found, surprisingly, that, at least for a mixed-initiative spoken dialog system as the CMU Communicator, these errors trade-o# equally over a wide operating characteristic range

Language Modeling for Dialog System

take two forms. Human input can be constrained through a directed dialog, allowing the decoder to use a state-specific language model to improve recognition accuracy. Mixedinitiative systems allow for human input that while domainspecific might not be state-specific. Nevertheless, for the most part human input to a mixed-initiative system is predictable, particularly when given information about the immediately preceding system prompt. The work reported in this paper addresses the problem of balancing state-specific and general language modeling in a mixed-initiative dialog system. By incorporating dialog state adaptation of the language model, we have reduced the recognition error rate by 11.5%

Can Artificial Neural Networks Learn Language Models?

Currently, N-gram models are the most common and widely used models for statistical language modeling. In this paper, we investigated an alternative way to build language models, i.e., using artificial neural networks to learn the language model. Our experiment result shows that the neural network can learn a language model that has performance even better than standard statistical methods