Towards Integration of Cognitive Models in Dialogue Management: Designing the Virtual Negotiation Coach Application

This paper presents an approach to flexible and adaptive dialogue management driven by cognitive modelling of human dialogue behaviour. Artificial intelligent agents, based on the ACT-R cognitive architecture, together with human actors are participating in a (meta)cognitive skills training within a negotiation scenario. The agent  employs instance-based learning to decide about its own actions and to reflect on the behaviour of the opponent. We show that task-related actions can be handled by a cognitive agent who is a plausible dialogue partner.  Separating task-related and dialogue control actions enables the application of sophisticated models along with a flexible architecture  in which  various alternative modelling methods can be combined. We evaluated the proposed approach with users assessing  the relative contribution of various factors to the overall usability of a dialogue system. Subjective perception of effectiveness, efficiency and satisfaction were correlated with various objective performance metrics, e.g. number of (in)appropriate system responses, recovery strategies, and interaction pace. It was observed that the dialogue system usability is determined most by the quality of agreements reached in terms of estimated Pareto optimality, by the user's negotiation strategies selected, and by the quality of system recognition, interpretation and responses. We compared human-human and human-agent performance with respect to the number and quality of agreements reached, estimated cooperativeness level, and frequency of accepted negative outcomes. Evaluation experiments showed promising, consistently positive results throughout the range of the relevant scales

Using Speech to Interrupt Complex Tasks

Interacting with computers using speech promises the benefit of multitasking while one’s hands and eyes are occupied by another task. Users of spoken dialogue sys- tems have not seen this technology as living up to its potential however, in part be- cause speech agents interactions largely behave like traditional interface interactions, initiated by the user. In order to harness the full multitasking benefit of speech as an interaction modality, speech agents must interactive proactively, but doing so means that agents will need to interrupt users engaged in other tasks. While general guide- lines have been proposed for the design of proactive agents in general, the design of proactive speech which interrupts has not been explored in specific. Interrupting speech from proactive agents can take design inspiration from the ways people use speech to interrupt other people, but human speech interruptions are likewise not well understood. The first study of this thesis uses a mixed methods approach to investi- gate the effect of the urgency on people’s timing and strategies for interrupting with speech. The second study complements that data by comparing the effect of urgency on interruption timings and strategies to the effect of the difficulty of the task which is interrupted. The third study uses a data-driven approach to classify the interruptible moments of a complex task in order to analyse the extent to which participants from the prior studies utilised these dynamic characteristics of the ongoing task to shape their interruptions. Finally, the fourth study applies findings from human speech inter- ruption to the design of a proactive agent to investigate the effects of human-inspired adaptivity to context on people’s perceptions of a proactive speech agent. Findings suggest that human speech interruptions are highly diverse and adaptive to context, but such adaptivity may be seen as inappropriate and inconsistent when applied to a speech agent. The implications of this research and its limitations are discussed in the closing chapter.Science Foundation Irelan

Cognitive architecture of multimodal multidimensional dialogue management

Numerous studies show that participants of real-life dialogues happen to get involved in rather dynamic non-sequential interactions. This challenges the dialogue system designs based on a reactive interlocutor paradigm and calls for dialog systems that can be characterised as a proactive learner, accomplished multitasking planner and adaptive decision maker. Addressing this call, the thesis brings innovative integration of cognitive models into the human-computer dialogue systems. This work utilises recent advances in Instance-Based Learning of Theory of Mind skills and the established Cognitive Task Analysis and ACT-R models. Cognitive Task Agents, producing detailed simulation of human learning, prediction, adaption and decision making, are integrated in the multi-agent Dialogue Man-ager. The manager operates on the multidimensional information state enriched with representations based on domain- and modality-specific semantics and performs context-driven dialogue acts interpretation and generation. The flexible technical framework for modular distributed dialogue system integration is designed and tested. The implemented multitasking Interactive Cognitive Tutor is evaluated as showing human-like proactive and adaptive behaviour in setting goals, choosing appropriate strategies and monitoring processes across contexts, and encouraging the user exhibit similar metacognitive competences

Toward an Ontology-Based Chatbot Endowed with Natural Language Processing and Generation

International audienceWith the last evolution of the web, several new means of communication have showed up. In the commercial domain, chatbot tech-nologies are now considered as essential for providing a wide range of ser-vices (e.g. search, FAQ, assistance) to the end-user, and to make a client a faithful customer. In this paper, we propose an on-going work on the denition and implementation of SynchroBot, an ontology-based chat-bot that relies on Semantic Web and NLP models and technologies to support user-machine dialogical interaction in the e-commerce domain

The use of social media, e-mail and instant messaging as the predictors of an employee’s work performance / Ainaa Idayu Iskandar, Ahmad Fadhly Arham and Najiah Shohaime

Computer mediated communication tools such as e-mail, instant messaging and social media has become common tools used to deliver messages to get prompt responses. It is an opportunity for organizations to use and fully utilise the functions of e-mail, instant messaging and social network to improve work performance. This study was conducted at one of the public service departments in Malaysia with 132 number of questionnaires distributed. The data was analysed using multiple regression. Findings showed that there is significant relationship between the use of instant messaging and social network with performance, whereas the results for the use of e-mail showed otherwise. This paper provides a better knowledge and information on how to increase the work performance of employees through the use of instant messaging and social network. The findings may also contribute to the existing literature

Adaptive Cognitive Interaction Systems

Adaptive kognitive Interaktionssysteme beobachten und modellieren den Zustand ihres Benutzers und passen das Systemverhalten entsprechend an. Ein solches System besteht aus drei Komponenten: Dem empirischen kognitiven Modell, dem komputationalen kognitiven Modell und dem adaptiven Interaktionsmanager. Die vorliegende Arbeit enthält zahlreiche Beiträge zur Entwicklung dieser Komponenten sowie zu deren Kombination. Die Ergebnisse werden in zahlreichen Benutzerstudien validiert

CUI @ Auto-UI:Exploring the Fortunate and Unfortunate Futures of Conversational Automotive User Interfaces

This work aims to connect the Automotive User Interfaces (Auto-UI) and Conversational User Interfaces (CUI) communities through discussion of their shared view of the future of automotive conversational user interfaces. The workshop aims to encourage creative consideration of optimistic and pessimistic futures, encouraging attendees to explore the opportunities and barriers that lie ahead through a game. Considerations of the future will be mapped out in greater detail through the drafting of research agendas, by which attendees will get to know each other's expertise and networks of resources. The two day workshop, consisting of two 90-minute sessions, will facilitate greater communication and collaboration between these communities, connecting researchers to work together to influence the futures they imagine in the workshop.Comment: Workshop published and presented at Automotive User Interfaces 2021 (AutoUI 21

Optimizing The Design Of Multimodal User Interfaces

Due to a current lack of principle-driven multimodal user interface design guidelines, designers may encounter difficulties when choosing the most appropriate display modality for given users or specific tasks (e.g., verbal versus spatial tasks). The development of multimodal display guidelines from both a user and task domain perspective is thus critical to the achievement of successful human-system interaction. Specifically, there is a need to determine how to design task information presentation (e.g., via which modalities) to capitalize on an individual operator\u27s information processing capabilities and the inherent efficiencies associated with redundant sensory information, thereby alleviating information overload. The present effort addresses this issue by proposing a theoretical framework (Architecture for Multi-Modal Optimization, AMMO) from which multimodal display design guidelines and adaptive automation strategies may be derived. The foundation of the proposed framework is based on extending, at a functional working memory (WM) level, existing information processing theories and models with the latest findings in cognitive psychology, neuroscience, and other allied sciences. The utility of AMMO lies in its ability to provide designers with strategies for directing system design, as well as dynamic adaptation strategies (i.e., multimodal mitigation strategies) in support of real-time operations. In an effort to validate specific components of AMMO, a subset of AMMO-derived multimodal design guidelines was evaluated with a simulated weapons control system multitasking environment. The results of this study demonstrated significant performance improvements in user response time and accuracy when multimodal display cues were used (i.e., auditory and tactile, individually and in combination) to augment the visual display of information, thereby distributing human information processing resources across multiple sensory and WM resources. These results provide initial empirical support for validation of the overall AMMO model and a sub-set of the principle-driven multimodal design guidelines derived from it. The empirically-validated multimodal design guidelines may be applicable to a wide range of information-intensive computer-based multitasking environments