The DBOX Corpus Collection of Spoken Human-Human and Human-Machine Dialogues

This paper describes the data collection and annotation carried out within the DBOX project ( Eureka project, number E! 7152). This project aims to develop interactive games based on spoken natural language human-computer dialogues, in 3 European languages: English, German and French. We collect the DBOX data continuously. We first start with human-human Wizard of Oz experiments to collect human-human data in order to model natural human dialogue behaviour, for better understanding of phenomena of human interactions and predicting interlocutors actions, and then replace the human Wizard by an increasingly advanced dialogue system, using evaluation data for system improvement. The designed dialogue system relies on a Question-Answering (QA) approach, but showing truly interactive gaming behaviour, e.g., by providing feedback, managing turns and contact, producing social signals and acts, e.g., encouraging vs. downplaying, polite vs. rude, positive vs. negative attitude towards players or their actions, etc. The DBOX dialogue corpus has required substantial investment. We expect it to have a great impact on the rest of the project. The DBOX project consortium will continue to maintain the corpus and to take an interest in its growth, e.g., expand to other languages. The resulting corpus will be publicly released

Cognitive architecture of multimodal multidimensional dialogue management

Numerous studies show that participants of real-life dialogues happen to get involved in rather dynamic non-sequential interactions. This challenges the dialogue system designs based on a reactive interlocutor paradigm and calls for dialog systems that can be characterised as a proactive learner, accomplished multitasking planner and adaptive decision maker. Addressing this call, the thesis brings innovative integration of cognitive models into the human-computer dialogue systems. This work utilises recent advances in Instance-Based Learning of Theory of Mind skills and the established Cognitive Task Analysis and ACT-R models. Cognitive Task Agents, producing detailed simulation of human learning, prediction, adaption and decision making, are integrated in the multi-agent Dialogue Man-ager. The manager operates on the multidimensional information state enriched with representations based on domain- and modality-specific semantics and performs context-driven dialogue acts interpretation and generation. The flexible technical framework for modular distributed dialogue system integration is designed and tested. The implemented multitasking Interactive Cognitive Tutor is evaluated as showing human-like proactive and adaptive behaviour in setting goals, choosing appropriate strategies and monitoring processes across contexts, and encouraging the user exhibit similar metacognitive competences

Towards Integration of Cognitive Models in Dialogue Management: Designing the Virtual Negotiation Coach Application

This paper presents an approach to flexible and adaptive dialogue management driven by cognitive modelling of human dialogue behaviour. Artificial intelligent agents, based on the ACT-R cognitive architecture, together with human actors are participating in a (meta)cognitive skills training within a negotiation scenario. The agent  employs instance-based learning to decide about its own actions and to reflect on the behaviour of the opponent. We show that task-related actions can be handled by a cognitive agent who is a plausible dialogue partner.  Separating task-related and dialogue control actions enables the application of sophisticated models along with a flexible architecture  in which  various alternative modelling methods can be combined. We evaluated the proposed approach with users assessing  the relative contribution of various factors to the overall usability of a dialogue system. Subjective perception of effectiveness, efficiency and satisfaction were correlated with various objective performance metrics, e.g. number of (in)appropriate system responses, recovery strategies, and interaction pace. It was observed that the dialogue system usability is determined most by the quality of agreements reached in terms of estimated Pareto optimality, by the user's negotiation strategies selected, and by the quality of system recognition, interpretation and responses. We compared human-human and human-agent performance with respect to the number and quality of agreements reached, estimated cooperativeness level, and frequency of accepted negative outcomes. Evaluation experiments showed promising, consistently positive results throughout the range of the relevant scales

Anatomy of dialogue in out-of-hospital cardiac arrest resuscitation

Research on medical teams constantly recognise the crucial value of communication. Studies on various medical teams, such as surgery and trauma, provide evidence for how communication either affects or is affected by a range of outcomes and variables. Nevertheless, much of this work has focused on in-hospital communication. Less is known about the patterns of communication amongst medical practitioners in high-stakes emergency care outside of the hospital. This thesis presents an investigation of dialogue during pre-hospital resuscitations when paramedics are responding to out-of-hospital cardiac arrest (OHCA). A bespoke dialogue annotation system, called the Dialogue Annotation for Resuscitation coding scheme (DARe), is developed for this purpose. DARe is used to annotate four simulated and 40 real-life OHCA resuscitation attempts by paramedics who are based in Edinburgh, Scotland. We examine (1) the distributions of communicative functions and subject matters (threads); (2) specific statements used by team members to align themselves; (3) the prevalence and forms of mitigated directives; (4) the verbal manners of planning; (5) the occurrence of closed-loop communication and other structures of verbal communication loops; and (6) the prevalence of socioemotionally-related utterances. For the real-life resuscitation dialogues, the study additionally investigates (7) the correlations between the distributions of the dialogue patterns with the assessed performance of resuscitation team leaders and with the time taken to successfully deploy a mechanical chest compression device (AutoPulse). Analysis for the simulation dialogues was performed from the start of simulation until the end or near the end of the procedure, whilst analysis for the real-life dialogues concentrated on the first five minutes. Despite this difference in timing, the results showed that simulated and real-life OHCA dialogues comprised similarly high frequencies of statements, directives, acceptances, and acknowledgments. Both simulated and real-life dialogues also contained sociolinguistic influences from the linguistic context that these were derived from, i.e. Scottish English. In considering the threads across both settings, the largest proportion of threads revolved around planning and execution of tasks, followed by threads on patient history and related instrument/equipment. Dialogues during real-life OHCA resuscitations differed from the simulated resuscitations in the additional presence of two communicative techniques, namely Alerters (used to attract hearer’s attention) and Affective performatives (used to convey affective or socioemotional statements). Additionally, real-life resuscitation dialogues contained a larger proportion of threads pertaining to patient positioning due to the use of the AutoPulse. Resuscitation team members often used a statement structure called State-awareness to align themselves with one another in terms of their current state or task. Directives were frequently mitigated, with strategies ranging from simple use of softeners (e.g. please) to less straightforward directive structures (e.g. suggestion). Plans were verbalised in temporal clusters, i.e. distinguishable in terms of the immediacy of the task to be performed. Few verbal affective behaviours (e.g. humour, gratitude, compliments) were observed. Team members also used very few exchanges that resembled the standard, three-level closed-loop communication structure typically required from professionals in other high-stakes dialogue environments. Correlation analyses revealed that the frequencies of both the communicative functions and threads were associated with the performance scores of resuscitation team leaders. Teams led by higher rated leaders (the ideal score group) showed higher proportions of Alerters, Affective performatives, State-awareness, and Plan of action in their dialogues compared to teams led by lower rated leaders (the low score group). There were also variations in the concentrations of chest compressions, patient history, and rhythm threads in the two groups, indicating that both discussed the same threads but at different junctures of the procedure. Meanwhile, the time taken to deploy the AutoPulse was positively correlated with the communicative function Acknowledge and the threads Patient history and Movement other than patient, and negatively correlated with the communicative function Open-option and the threads Ventilation and Airway access. Based on these results, several potential measures for optimising OHCA resuscitation are proposed: the use of sewn-on name badges for paramedics; shorter time dedicated for the extraction of patient history; verbal reports of vital points throughout the procedure; the use of non or less mitigated directives; and standardisation of resuscitation phrases. Each suggestion is also discussed in terms of anticipated challenges and possible solutions. The results presented in this thesis provide grounds for further research on the features of pre-hospital resuscitation dialogues. DARe has been demonstrated to be useful in discriminating linguistic patterns, suggesting that dialogue annotation analysis can be utilised to further investigate this area and ultimately contribute to resuscitation performance

(Računalna) pragmatika: temeljni pojmovi i korpusnopragmatičke analize

(Računalna) pragmatika: temeljni pojmovi i korpusnopragmatičke analize studija je posvećena prikazu lingvističke pragmatike, s posebnim osvrtom na njezine dvije grane – računalnu i korpusnu pragmatiku, čija je primjena prikazana na primjerima korpusnopragmatičkih analiza gramatičko-pragmatičkih pojavnosti u hrvatskom i srpskom jeziku. Knjiga je organizirana u dva dijela. Prvi, teorijski dio uključuje opći prikaz lingvističke pragmatike, njezinih područja bavljenja te istraživačkih ciljeva (Cjelina 1), pregled povijesnog razvoja pragmatike (Cjelina 2) te pregled metoda prikupljanja i obrade jezične građe koje se primjenjuju u pragmatičkim istraživanjima (Cjelina 3). Posebna pažnja posvećena je prikazu računalne pragmatike i njezina odnosa s područjem obrade prirodnog jezika te predstavljanju korpusnopragmatičkog pristupa jeziku u upotrebi (Cjelina 4). Drugi dio knjige posvećen je odnosu gramatike i (korpusne) pragmatike. U njemu su (p)opisane gramatičke kategorije u hrvatskom i srpskom jeziku koje su izravno povezane s kontekstom jezične upotrebe (gramatičko-pragmatičke kategorije) (Cjelina 5) te su predstavljeni primjeri (kontrastivnih) korpusnopragmatičkih analiza gramatičko-pragmatičkih i drugih pragmatičkih pojavnosti u hrvatskom i srpskom jeziku (Cjeline 6–8). Posljednja cjelina knjige posvećena je prikazu izrade specijaliziranog korpusa govornih činova s pragmatičkom anotacijom (Cjelina 9).(Računalna) pragmatika: temeljni pojmovi i korpusnopragmatičke analize studija je posvećena prikazu lingvističke pragmatike, s posebnim osvrtom na njezine dvije grane – računalnu i korpusnu pragmatiku, čija je primjena prikazana na primjerima korpusnopragmatičkih analiza gramatičko-pragmatičkih pojavnosti u hrvatskom i srpskom jeziku. Knjiga je organizirana u dva dijela. Prvi, teorijski dio uključuje opći prikaz lingvističke pragmatike, njezinih područja bavljenja te istraživačkih ciljeva (Cjelina 1), pregled povijesnog razvoja pragmatike (Cjelina 2) te pregled metoda prikupljanja i obrade jezične građe koje se primjenjuju u pragmatičkim istraživanjima (Cjelina 3). Posebna pažnja posvećena je prikazu računalne pragmatike i njezina odnosa s područjem obrade prirodnog jezika te predstavljanju korpusnopragmatičkog pristupa jeziku u upotrebi (Cjelina 4). Drugi dio knjige posvećen je odnosu gramatike i (korpusne) pragmatike. U njemu su (p)opisane gramatičke kategorije u hrvatskom i srpskom jeziku koje su izravno povezane s kontekstom jezične upotrebe (gramatičko-pragmatičke kategorije) (Cjelina 5) te su predstavljeni primjeri (kontrastivnih) korpusnopragmatičkih analiza gramatičko-pragmatičkih i drugih pragmatičkih pojavnosti u hrvatskom i srpskom jeziku (Cjeline 6–8). Posljednja cjelina knjige posvećena je prikazu izrade specijaliziranog korpusa govornih činova s pragmatičkom anotacijom (Cjelina 9)