A Dialogue-Management Evaluation Study

We present a highly portable and cooperative dialogue-manager component of a developing, Slovenian and Croatian spoken dialogue system for weather-information retrieval. In order to evaluate the performance of this component, two Wizard-of-Oz experiments were performed. The only difference between both experiment settings was in the dialogue-management manner, i.e., while in the first experiment dialogue management was performed by a human, the wizard, in the second experiment it was performed by the newly-implemented dialogue-manager component. The data from both Wizard-of-Oz experiments was evaluated with the PARADISE evaluation framework, a potential general methodology for evaluating and comparing different versions of spoken-language dialogue systems. The study ascertains remarkable differences in the performance functions when taking different satisfaction-measure sums or even individual scores as the target to be predicted, it demonstrates the need for the introduced dialogue costs em database parameters, and it confirms the dialogue manager\u27s cooperativity subject to the incorporated knowledge representation

Capturing Situational Context in an Augmented Memory System

Bookmarking a moment is a new approach being introduced to capture past experience and insert information into an augmented memory system. This idea is inspired from the concept of the bookmark in web browsers. Semi-automatic bookmarking different moments when time is limited and revisiting these moments before inserting them into an augmented memory system will help people to remember their past experience. An exploratory study was conducted to discover and shape the design requirements for a system called CatchIt. It aims to understand end-users’ needs to capture their personal experience, which is an important and complex issue in the case of capture and access of personal experiences. CatchIt is a system to bookmark the significant moments during the day before enriching them, and entering them into the augmented memory system called Digital Parrot. The conceptual design of CatchIt will be the main aim of this study. The primary requirements were derived from the scenarios and analysis of the findings of five different study stages were designed to inspect these: unobserved field visit, shadowing, using indictors, Wizard of Oz and using technology. Thirty participants were involved in field visit, survey and follows up interview. Each stage had different tasks to be performed and the findings of each stage contributed to understanding different parts of user needs and system design requirements. The results of this study indicated the system should automatically record the context information, especially the time and location since they were typically neglected by the participants. Different information such as textual and visual information should be manually recorded based on the users’ setting or situations. A single button is a promising input mechanism to bookmark a moment and it should be fast and effort- less. The result showed no clear correlation between learning style and type of the information that had been captured. Also, we found that there might be a correlation between passive capture and false memories. All these findings were used to provide a foundation for further work to implement the bookmark system and evaluate this approach. Some issues raised in this study need further research. The work will contribute to a greater understanding of human memory and selective capture

Towards human-like conversational search systems

Voice search is currently widely available on the majority of mobile devices via use of Virtual Personal Assistants. However, despite its general availability, the use of voice interaction remains sporadic and is limited to basic search tasks such as checking weather updates and looking up answers to factual queries. Present-day voice search systems struggle to use relevant contextual information to maintain conversational state, and lack conversational initiative needed to clarify user’s intent, which hampers their usability and prevents users from engaging in more complex interaction activities. This research investigates the potential of a hypothesised interactive information retrieval system with human-like conversational abilities. To this end, we propose a series of usability studies that involve a working prototype of a conversational system that uses real time speech synthesis. The proposed experiments seek to provide empirical evidence that enabling a voice search system with human-like conversational abilities can lead to increased likelihood of its adoption

Applying a User-centred Approach to Interactive Visualization Design

Analysing users in their context of work and finding out how and why they use different information resources is essential to provide interactive visualisation systems that match their goals and needs. Designers should actively involve the intended users throughout the whole process. This chapter presents a user-centered approach for the design of interactive visualisation systems. We describe three phases of the iterative visualisation design process: the early envisioning phase, the global specification hase, and the detailed specification phase. The whole design cycle is repeated until some criterion of success is reached. We discuss different techniques for the analysis of users, their tasks and domain. Subsequently, the design of prototypes and evaluation methods in visualisation practice are presented. Finally, we discuss the practical challenges in design and evaluation of collaborative visualisation environments. Our own case studies and those of others are used throughout the whole chapter to illustrate various approaches

Dialogue based interfaces for universal access.

Conversation provides an excellent means of communication for almost all people. Consequently, a conversational interface is an excellent mechanism for allowing people to interact with systems. Conversational systems are an active research area, but a wide range of systems can be developed with current technology. More sophisticated interfaces can take considerable effort, but simple interfaces can be developed quite rapidly. This paper gives an introduction to the current state of the art of conversational systems and interfaces. It describes a methodology for developing conversational interfaces and gives an example of an interface for a state benefits web site. The paper discusses how this interface could improve access for a wide range of people, and how further development of this interface would allow a larger range of people to use the system and give them more functionality

Sources of Noise in Dialogue and How to Deal with Them

Training dialogue systems often entails dealing with noisy training examples and unexpected user inputs. Despite their prevalence, there currently lacks an accurate survey of dialogue noise, nor is there a clear sense of the impact of each noise type on task performance. This paper addresses this gap by first constructing a taxonomy of noise encountered by dialogue systems. In addition, we run a series of experiments to show how different models behave when subjected to varying levels of noise and types of noise. Our results reveal that models are quite robust to label errors commonly tackled by existing denoising algorithms, but that performance suffers from dialogue-specific noise. Driven by these observations, we design a data cleaning algorithm specialized for conversational settings and apply it as a proof-of-concept for targeted dialogue denoising.Comment: 23 pages, 6 Figures, 5 tables. Accepted at SIGDIAL 202