Natural Language, Mixed-Initiative Personal Assistant Agents

The increasing popularity and use of personal voice assistant technologies, such as Siri and Google Now, is driving and expanding progress toward the long-term and lofty goal of using artificial intelligence to build human-computer dialog systems capable of understanding natural language. While dialog-based systems such as Siri support utterances communicated through natural language, they are limited in the flexibility they afford to the user in interacting with the system and, thus, support primarily action-requesting and information-seeking tasks. Mixed-initiative interaction, on the other hand, is a flexible interaction technique where the user and the system act as equal participants in an activity, and is often exhibited in human-human conversations. In this paper, we study user support for mixed-initiative interaction with dialog-based systems through natural language using a bag-of-words model and k-nearest-neighbor classifier. We study this problem in the context of a toolkit we developed for automated, mixed-initiative dialog system construction, involving a dialog authoring notation and management engine based on lambda calculus, for specifying and implementing task-based, mixed-initiative dialogs. We use ordering at Subway through natural language, human-computer dialogs as a case study. Our results demonstrate that the dialogs authored with our toolkit support the end user\u27s completion of a natural language, human-computer dialog in a mixed-initiative fashion. The use of natural language in the resulting mixed-initiative dialogs afford the user the ability to experience multiple self-directed paths through the dialog and makes the flexibility in communicating user utterances commensurate with that in dialog completion paths---an aspect missing from commercial assistants like Siri

Do humans adapt their emails so that agents can understand?

Proceedings of AAAI Symposium on Mixed Initiative Problem Solving Assistants, November 2005. Retrieved 3/16/2006 from http://www.cs.drexel.edu/~pmodi/papers/modi-aaaiFallSymp-05.pdf.Natural language processing is a difficult problem and is currently a technical barrier in building personal assistant agents that aim to interact with humans in natural language. One possible solution is to rely on humans to restrict or adapt their language into more computer friendly ways. We study the feasibility of this approach in the context of a personal assistant agent that parses emails relevant to meeting scheduling in order to assist the user in calendar management tasks. We design an experiment in which humans are given email writing tasks and then given feedback on how well those emails were understood by an agent. We wish to see if the humans learn to adapt their email writing to what is understandable by their agent

A Personalized System for Conversational Recommendations

Searching for and making decisions about information is becoming increasingly difficult as the amount of information and number of choices increases. Recommendation systems help users find items of interest of a particular type, such as movies or restaurants, but are still somewhat awkward to use. Our solution is to take advantage of the complementary strengths of personalized recommendation systems and dialogue systems, creating personalized aides. We present a system -- the Adaptive Place Advisor -- that treats item selection as an interactive, conversational process, with the program inquiring about item attributes and the user responding. Individual, long-term user preferences are unobtrusively obtained in the course of normal recommendation dialogues and used to direct future conversations with the same user. We present a novel user model that influences both item search and the questions asked during a conversation. We demonstrate the effectiveness of our system in significantly reducing the time and number of interactions required to find a satisfactory item, as compared to a control group of users interacting with a non-adaptive version of the system

Design and evaluation of acceleration strategies for speeding up the development of dialog applications

In this paper, we describe a complete development platform that features different innovative acceleration strategies, not included in any other current platform, that simplify and speed up the definition of the different elements required to design a spoken dialog service. The proposed accelerations are mainly based on using the information from the backend database schema and contents, as well as cumulative information produced throughout the different steps in the design. Thanks to these accelerations, the interaction between the designer and the platform is improved, and in most cases the design is reduced to simple confirmations of the “proposals” that the platform dynamically provides at each step. In addition, the platform provides several other accelerations such as configurable templates that can be used to define the different tasks in the service or the dialogs to obtain or show information to the user, automatic proposals for the best way to request slot contents from the user (i.e. using mixed-initiative forms or directed forms), an assistant that offers the set of more probable actions required to complete the definition of the different tasks in the application, or another assistant for solving specific modality details such as confirmations of user answers or how to present them the lists of retrieved results after querying the backend database. Additionally, the platform also allows the creation of speech grammars and prompts, database access functions, and the possibility of using mixed initiative and over-answering dialogs. In the paper we also describe in detail each assistant in the platform, emphasizing the different kind of methodologies followed to facilitate the design process at each one. Finally, we describe the results obtained in both a subjective and an objective evaluation with different designers that confirm the viability, usefulness, and functionality of the proposed accelerations. Thanks to the accelerations, the design time is reduced in more than 56% and the number of keystrokes by 84%

CSE: U: Mixed-initiative Personal Assistant Agents

Specification and implementation of flexible human-computer dialogs is challenging because of the complexity involved in rendering the dialog responsive to a vast number of varied paths through which users might desire to complete the dialog. To address this problem, we developed a toolkit for modeling and implementing task-based, mixed-initiative dialogs based on metaphors from lambda calculus. Our toolkit can automatically operationalize a dialog that involves multiple prompts and/or sub-dialogs, given a high-level dialog specification of it. The use of natural language with the resulting dialogs makes the flexibility in communicating user utterances commensurate with that in dialog completion paths—an aspect missing from commercial assistants like Siri. Our results demonstrate that the dialogs authored with our toolkit support the end user’s completion of a human-computer dialog in a manner that is most natural to them—in a mixed-initiative fashion—that resembles human-human interaction

A Review of Verbal and Non-Verbal Human-Robot Interactive Communication

In this paper, an overview of human-robot interactive communication is presented, covering verbal as well as non-verbal aspects of human-robot interaction. Following a historical introduction, and motivation towards fluid human-robot communication, ten desiderata are proposed, which provide an organizational axis both of recent as well as of future research on human-robot communication. Then, the ten desiderata are examined in detail, culminating to a unifying discussion, and a forward-looking conclusion

Inter-cultural differences in response to a computer-based anti-bullying intervention

Background and purpose: Many holistic anti-bullying interventions have been attempted, with mixed success, while little work has been done to promote a 'self-help' approach to victimisation. The rise of the ICT curriculum and computer support in schools now allows for approaches that benefit from technology to be implemented. This study evaluates the cross-cultural effects of a computer-based anti-bullying intervention on primary school-aged children's knowledge about bullying and relevant coping strategies. Programme description: FearNot! is an interactive computer-based virtual learning environment designed for use as an anti-bullying intervention. It includes interactive virtual agents who assume the most common participant roles found in episodes of bullying. FearNot! was used by children over three consecutive weeks to allow its effectiveness to be evaluated in a longitudinal in situ programme. Sample: Two comparable samples were drawn from the UK and Germany. In the UK, 651 participants (aged 8-11) were recruited from primary schools in Hertfordshire, Coventry and Warwickshire, whereas the 535 German participants (aged 7-10) were sourced from Grundschulen in the Bayern and Hessen regions. Because of lack of parental consent, late joiners and absences/missing responses, data from 908 participants (UK 493; Germany 415) were analysed. Design and methods: A quasi-experimental, pre/post-tests control group design employed pre-published and bespoke questionnaires to collect data. Descriptive and inferential analyses were conducted. Results: UK students possessed higher coping strategy knowledge scores than German participants, but German children's scores improved over time and as a result of the FearNot! intervention. Conclusions: Overall, while not effective at increasing children's coping strategy knowledge in this study, the FearNot! intervention could prove a useful classroom tool to approach the issue of bullying as part of a wider initiative. Cultural differences at baseline and reactions to the intervention are discussed

Decision-Oriented Dialogue for Human-AI Collaboration

We describe a class of tasks called decision-oriented dialogues, in which AI assistants must collaborate with one or more humans via natural language to help them make complex decisions. We formalize three domains in which users face everyday decisions: (1) choosing an assignment of reviewers to conference papers, (2) planning a multi-step itinerary in a city, and (3) negotiating travel plans for a group of friends. In each of these settings, AI assistants and users have disparate abilities that they must combine to arrive at the best decision: assistants can access and process large amounts of information, while users have preferences and constraints external to the system. For each task, we build a dialogue environment where agents receive a reward based on the quality of the final decision they reach. Using these environments, we collect human-human dialogues with humans playing the role of assistant. To compare how current AI assistants communicate in these settings, we present baselines using large language models in self-play. Finally, we highlight a number of challenges models face in decision-oriented dialogues, ranging from efficient communication to reasoning and optimization, and release our environments as a testbed for future modeling work