Towards Integration of Cognitive Models in Dialogue Management: Designing the Virtual Negotiation Coach Application

This paper presents an approach to flexible and adaptive dialogue management driven by cognitive modelling of human dialogue behaviour. Artificial intelligent agents, based on the ACT-R cognitive architecture, together with human actors are participating in a (meta)cognitive skills training within a negotiation scenario. The agent  employs instance-based learning to decide about its own actions and to reflect on the behaviour of the opponent. We show that task-related actions can be handled by a cognitive agent who is a plausible dialogue partner.  Separating task-related and dialogue control actions enables the application of sophisticated models along with a flexible architecture  in which  various alternative modelling methods can be combined. We evaluated the proposed approach with users assessing  the relative contribution of various factors to the overall usability of a dialogue system. Subjective perception of effectiveness, efficiency and satisfaction were correlated with various objective performance metrics, e.g. number of (in)appropriate system responses, recovery strategies, and interaction pace. It was observed that the dialogue system usability is determined most by the quality of agreements reached in terms of estimated Pareto optimality, by the user's negotiation strategies selected, and by the quality of system recognition, interpretation and responses. We compared human-human and human-agent performance with respect to the number and quality of agreements reached, estimated cooperativeness level, and frequency of accepted negative outcomes. Evaluation experiments showed promising, consistently positive results throughout the range of the relevant scales

Semantic and pragmatic precision in conversational AI systems

For a conversational agent, to display intelligent interactive behavior implies the ability to respond to the user's intentions and expectations with correct, consistent and relevant actions with appropriate form and content in a timely fashion. In this paper, we present a data-driven analytical approach to embed intelligence into a conversational AI agent. The method requires a certain amount of (ideally) authentic conversational data, which is transformed in a meaningful way to support intelligent dialog modeling and the design of intelligent conversational agents. These transformations rely on the ISO 24617-2 dialog act annotation standard, and are specified in the Dialogue Act Markup Language (DiAML), extended with plug-ins for articulate representations of domain-specific semantic content and customized communicative functionality. ISO 24617-2 is shown to enable systematic in-depth interaction analysis and to facilitate the collection of conversational data of sufficient quality and quantity of instances of interaction phenomena. The paper provides the theoretical and methodological background of extending the ISO standard and DiAML specifications for use in interaction analysis and conversational AI agent design. The expert-assisted design methodology is introduced, with example applications in the healthcare domain, and is validated in human-agent conversational data collection experiments

D3.2 Instructional Designs for learning analytics and reflection support

In the context of the METALOGUE project this deliverable discusses the role and scope of Learner Analytics in METALOGUE, what type of data is available and should be used, the use of Learning Dashboards and visualisations to enable the learner (and tutor) to access the outcomes of the learner analytics, a set of initial example visualisations to be used in METALOGUE, and, finally, it concludes with an instructional design blueprint giving a global outline of a set of tasks with stepwise increasing complexity and the feedback proposed.METALOGUE10000-01-0

D3.1 Instructional Designs for Real-time Feedback

The main objective of METALOGUE is to produce a multimodal dialogue system that is able to implement an interactive behaviour that seems natural to users and is flexible enough to exploit the full potential of multimodal interaction. The METALOGUE system will be arranged in the context of educational use-case scenarios, i.e. for training active citizens (Youth Parliament) and call centre employees. This deliverable describes the intended real-time feedback and reflection in-action support to support the training. Real-time feedback informs learners how they perform key skills and enables them to monitor their progress and thus reflect in-action. This deliverable examines the theoretical considerations of reflection in-action, what type of data is available and should be used, the timing and type of real-time feedback and, finally, concludes with an instructional design blueprint giving a global outline of a set of tasks with stepwise increasing complexity and the feedback proposed.The underlying research project is partly funded by the METALOGUE project. METALOGUE is a Seventh Framework Programme collaborative project funded by the European Commission, grant agreement number: 611073 (http://www.metalogue.eu)

Downward compatible revision of dialogue annotation

This paper discusses some aspects of revising the ISO standard for dialogue act annotation (ISO 24617-2). The revision is aimed at making annotations using the ISO scheme more accurate and at providing more powerful tools for building natural language based dialogue systems, without invalidating the annotated resources that have been built, with the current version of the standard. In support of the revision of the standard, an analysis is provided of the downward compatibility of a revised annotation scheme with the original scheme at the levels of abstract syntax, concrete syntax, and semantics of annotations