Dialogue Policies for Confusion Mitigation in Situated HRI

Confusion is a mental state triggered by cognitive disequilibrium that can occur in many types of task-oriented interaction, including Human-Robot Interaction (HRI). People may become confused while interacting with robots due to communicative or even task-centred challenges. To build a smooth and engaging HRI, it is insufficient for an agent to simply detect confusion; instead, the system should aim to mitigate the situation. In light of this, in this paper, we present our approach to a linguistic design of dialogue policies to build a dialogue framework to alleviate interlocutor confusion. We also outline our sketch and discuss challenges with respect to its operationalisation

Transferring Studies Across Embodiments: A Case Study in Confusion Detection

Human-robot studies are expensive to conduct and difficult to control, and as such researchers sometimes turn to human-avatar interaction in the hope of faster and cheaper data collection that can be transferred to the robot domain. In terms of our work, we are particularly interested in the challenge of detecting and modelling user confusion in interaction, and as part of this research programme, we conducted situated dialogue studies to investigate users\u27 reactions in confusing scenarios that we give in both physical and virtual environments. In this paper, we present a combined review of these studies and the results that we observed across these two embodiments. For the physical embodiment, we used a Pepper Robot, while for the virtual modality, we used a 3D avatar. Our study shows that despite attitudinal differences and technical control limitations, there were a number of similarities detected in user behaviour and self-reporting results across embodiment options. This work suggests that, while avatar interaction is no true substitute for robot interaction studies, sufficient care in study design may allow well executed human-avatar studies to supplement more challenging human-robot studies

A Framework for Confusion Mitigation in Task-Oriented Interactions

Confusion is a mental state that can be triggered in task-oriented interactions and which can if left unattended lead to boredom, frustration, or disengagement from the task at hand. Previous work has demonstrated that confusion can be detected in situated human-robot interactions from visual and auditory cues. Therefore, in the next step, we propose appropriate interaction structures in this study, which should be used to mitigate confusion. We motivate and describe this dialogue mechanism through an information state-style dialogue framework and policies, and also outline the approach we are taking to integrate such a meta-conversational goal alongside core task-oriented considerations in modern data-driven conversational techniques.https://arrow.tudublin.ie/cddpos/1023/thumbnail.jp

Hmm, You Seem Confused! Tracking Interlocutor Confusion for Situated Task-Oriented HRI

Our research seeks to develop a long-lasting and high-quality en- gagement between the user and the social robot, which in turn requires a more sophisticated alignment of the user and the system than is currently commonly available. Close monitoring of inter- locutors’ states, and we argue their confusion state in particular, and adjusting dialogue policies based on this state of confusion is needed for successful joint activity. In this paper, we present an ini- tial study of a human-robot conversation scenarios using a Pepper robot to investigate the confusion states of users. A Wizard-of-Oz (WoZ) HRI experiment is illustrated in detail with stimuli strategies to trigger confused states from interlocutors. For the collected data, we estimated emotions, head pose, and eye gaze, and these features were analysed against the silence duration time of the speech data and the post-study self-reported confusion states that are reported by participants. Our analysis found a significant relationship be- tween confusion states and most of these features. We see these results as being particularly significant for multimodal situated dialogues for human-robot interaction and beyond

Technical Report: A Framework for Confusion Mitigation in Task-Oriented Interactions

Confusion is a mental state that can be triggered in task-oriented interactions and which can if left unattended lead to boredom, frustration, or disengagement from the task at hand. Since previous work has demonstrated that confusion can be detected in embodied situated interactions from visual and auditory cues, in this technique report, we propose appropriate interaction structures which should be used to mitigate confusion. We motivate and describe this dialogue mechanism through an information state-style policy with examples, and also outline the approach we are taking to integrate such a meta-conversational goal alongside core task-oriented considerations in modern data driven conversational techniques. While the current policy design is a starting point, we believe it raises some interesting challenges for the integration of a reusable meta-conversational policy with highly data-driven approaches which have been enabled by large language models

A Framework for Confusion Mitigation in Task-Oriented Interactions

Confusion is a mental state that can be triggered in task-oriented interactions and which can if left unattended lead to boredom, frustration, or disengagement from the task at hand. Previous work has demonstrated that confusion can be detected in situated human-robot interactions from visual and auditory cues. Therefore, in the next step, we propose appropriate interaction structures in this study, which should be used to mitigate confusion. We motivate and describe this dialogue mechanism through an information state-style dialogue framework and policies, and also outline the approach we are taking to integrate such a meta-conversational goal alongside core task-oriented considerations in modern data-driven conversational techniques

Detecting Interlocutor Confusion in Situated Human-Avatar Dialogue: A Pilot Study

In order to enhance levels of engagement with conversational systems, our long term research goal seeks to monitor the confusion state of a user and adapt dialogue policies in response to such user confusion states. To this end, in this paper, we present our initial research centred on a user-avatar dialogue scenario that we have developed to study the manifestation of confusion and in the long term its mitigation. We present a new definition of confusion that is particularly tailored to the requirements of intelligent conversational system development for task-oriented dialogue. We also present the details of our Wizard-of-Oz based data collection scenario wherein users interacted with a conversational avatar and were presented with stimuli that were in some cases designed to invoke a confused state in the user. Post study analysis of this data is also presented. Here, three pre-trained deep learning models were deployed to estimate base emotion, head pose and eye gaze. Despite a small pilot study group, our analysis demonstrates a significant relationship between these indicators and confusion states. We see this as a useful step forward in the automated analysis of the pragmatics of dialogue

Effects of lowering body temperature via hyperhydration, with and without glycerol ingestion and practical precooling on cycling time trial performance in hot and humid conditions

Background Hypohydration and hyperthermia are factors that may contribute to fatigue and impairment of endurance performance. The purpose of this study was to investigate the effectiveness of combining glycerol hyperhydration and an established precooling technique on cycling time trial performance in hot environmental conditions. Methods Twelve well-trained male cyclists performed three 46.4-km laboratory-based cycling trials that included two climbs, under hot and humid environmental conditions (33.3 ± 1.1°C; 50 ± 6% r.h.). Subjects were required to hyperhydrate with 25 g.kg-1 body mass (BM) of a 4°C beverage containing 6% carbohydrate (CON) 2.5 h prior to the time trial. On two occasions, subjects were also exposed to an established precooling technique (PC) 60 min prior to the time trial, involving 14 g.kg-1 BM ice slurry ingestion and applied iced towels over 30 min. During one PC trial, 1.2 g.kg-1 BM glycerol was added to the hyperhydration beverage in a double-blind fashion (PC+G). Statistics used in this study involve the combination of traditional probability statistics and a magnitude-based inference approach. Results Hyperhydration resulted in large reductions (−0.6 to −0.7°C) in rectal temperature. The addition of glycerol to this solution also lowered urine output (330 ml, 10%). Precooling induced further small (−0.3°C) to moderate (−0.4°C) reductions in rectal temperature with PC and PC+G treatments, respectively, when compared with CON (0.0°C, P<0.05). Overall, PC+G failed to achieve a clear change in cycling performance over CON, but PC showed a possible 2% (30 s, P=0.02) improvement in performance time on climb 2 compared to CON. This improvement was attributed to subjects’ lower perception of effort reported over the first 10 km of the trial, despite no clear performance change during this time. No differences were detected in any other physiological measurements throughout the time trial. Conclusions Despite increasing fluid intake and reducing core temperature, performance and thermoregulatory benefits of a hyperhydration strategy with and without the addition of glycerol, plus practical precooling, were not superior to hyperhydration alone. Further research is warranted to further refine preparation strategies for athletes competing in thermally stressful events to optimize health and maximize performance outcomes