An embodied conversational agent for intelligent web interaction on pandemic crisis communication

In times of crisis, an effective communication mechanism is paramount in providing accurate and timely information to the community. In this paper we study the use of an intelligent embodied conversational agent (EGA) as the front end interface with the public for a Crisis Communication Network Portal (CCNet). The proposed system, CCNet, is an integration of the intelligent conversation agent, AINI, and an Automated Knowledge Extraction Agent (AKEA). AKEA retrieves first hand information from relevant sources such as government departments and news channels. In this paper, we compare the interaction of AINI against two popular search engines, two question answering systems and two conversational systems

Towards a more natural and intelligent interface with embodied conversation agent

Conversational agent also known as chatterbots are computer programs which are designed to converse like a human as much as their intelligent allows. In many ways, they are the embodiment of Turing's vision. The ability for computers to converse with human users using natural language would arguably increase their usefulness. Recent advances in Natural Language Processing (NLP) and Artificial Intelligence (AI) in general have advances this field in realizing the vision of a more humanoid interactive system. This paper presents and discusses the use of embodied conversation agent (ECA) for the imitation games. This paper also presents the technical design of our ECA and its performance. In the interactive media industry, it can also been observed that the ECA are getting popular

Simulating emotional reactions in medical dramas

Presenting information on emotionally charged topics is a delicate task: if bare facts alone are conveyed, there is a risk of boring the audience, or coming across as cold and unfeeling; on the other hand, emotional presentation can be appropriate when carefully handled, but when overdone or mishandled risks being perceived as patronising or in poor taste. When Natural Language Generation (NLG) systems present emotionally charged information linguistically, by generating scripts for embodied agents, emotional/affective aspects cannot be ignored. It is important to ensure that viewers consider the presentation appropriate and sympathetic. We are investigating the role of affect in communicating medical information in the context of an NLG system that generates short medical dramas enacted by embodied agents. The dramas have both an informational and an educational purpose in that they help patients review their medical histories whilst receiving explanations of less familiar medical terms and demonstrations of their usage. The dramas are also personalised since they are generated from the patients' own medical records. We view generation of natural/appropriate emotional language as a way to engage and maintain the viewers' attention. For our medical setting, we hypothesize that viewers will consider dialogues more natural when they have an enthusiastic and sympathetic emotional tone. Our second hypothesis proposes that such dialogues are also better for engaging the viewers' attention. As well as describing our NLG system for generating natural emotional language in medical dialogue, we present a pilot study with which we investigate our two hypotheses. Our results were not quite as unequivocal as we had hoped. Firstly, our participants did notice whether a character sympathised with the patient and was enthusiastic. This did not, however, lead them to judge such a character as behaving more naturally or the dialogue as being more engaging. However, when pooling data from our two conditions, dialogues with versus dialogues without emotionally appropriate language use, we discovered, somewhat surprisingly, that participants did consider a dialogue more engaging if they believed that the characters showed sympathy towards the patient, were not cold and unfeeling, and were natural (true for the female agent only)

Robust Dialog Management Through A Context-centric Architecture

This dissertation presents and evaluates a method of managing spoken dialog interactions with a robust attention to fulfilling the human user’s goals in the presence of speech recognition limitations. Assistive speech-based embodied conversation agents are computer-based entities that interact with humans to help accomplish a certain task or communicate information via spoken input and output. A challenging aspect of this task involves open dialog, where the user is free to converse in an unstructured manner. With this style of input, the machine’s ability to communicate may be hindered by poor reception of utterances, caused by a user’s inadequate command of a language and/or faults in the speech recognition facilities. Since a speech-based input is emphasized, this endeavor involves the fundamental issues associated with natural language processing, automatic speech recognition and dialog system design. Driven by ContextBased Reasoning, the presented dialog manager features a discourse model that implements mixed-initiative conversation with a focus on the user’s assistive needs. The discourse behavior must maintain a sense of generality, where the assistive nature of the system remains constant regardless of its knowledge corpus. The dialog manager was encapsulated into a speech-based embodied conversation agent platform for prototyping and testing purposes. A battery of user trials was performed on this agent to evaluate its performance as a robust, domain-independent, speech-based interaction entity capable of satisfying the needs of its users

Eyebrow raising in dialogue: discourse structure, utterance function, and pitch accents

Some studies have suggested a relationship between eyebrow raising and different aspects of the verbal message, but our knowledge about this link is still very limited. If we could establish and characterise a relation between eyebrow raises and the linguistic signal we could better understand human multimodal communication behaviour. We could also improve the credibility and efficiency of computer animated conversational agents in multimodal communication systems.This thesis investigated eyebrow raising in a corpus of task-oriented English dialogues. Applying a standard dialogue coding scheme (Conversational Game Analysis, Carletta et al., 1997), eyebrow raises were studied in connection with discourse structure and utterance function. Supporting the prediction, more frequent and longer eyebrow raising occurred in the initial utterance of highlevel discourse segments than anywhere else in the dialogue (where 'high-level discourse segment' = transaction, and 'utterance' = move, following Carletta et al.). Additionally, eyebrow raises were more frequent in instructions than in requests for or acknowledgements of information. Instructions also had longer eyebrow raising than any other type of utterance. Contrary to the prediction, the start of a lower-level discourse segment (conversational game) did not have more eyebrow raising than any other position in the dialogue, and queries did not have more eyebrow raising than any other type of utterance.Eyebrow raises were also studied in relation to intonational events, namely pitch accents. Results showed evidence of alignment between the brow raise start and the start of a pitch accent. Most pitch accents were not associated with brow raising, but when brow raises occurred they tended to immediately precede a pitch accent on the speech signal. To investigate what could explain the alignment between the two events, pitch accents aligned with eyebrow raises were compared to all other pitch accents in terms of: phonological characteristics (primary vs. secondary pitch accents, and downstep-initial vs. non-initial pitch accents), information structure (given vs. new information in referring expressions, and the last quarter vs. earlier parts of the utterance length) and type of utterance in which they occurred (instruction vs. non-instruction). Those comparisons suggested that brow raises may be aligned more frequently with pitch accents in downstepinitial position and in instructions. No differences were found in terms of information structure or between primary/secondary accents.The results provide evidence of a link between eyebrow raising and spoken language. Eyebrow raises may signal the start of linguistic units such as discourse segments and some prosodic phenomena, they may be related to utterance function, and they are aligned with pitch accents. Possible linguistic functions are proposed, such as structuring and emphasising information in the verbal message

A Conversational Agent in mHealth for Self-Management of Parkinson’s Disease

Nos dias que correm vivemos rodeados de tecnologia, onde os “smartphones” preenchem um espaço muito importante nas nossas vidas. O uso de serviços móveis pelos “smartphones” no âmbito da saúde tem sido cada vez mais próspero, com um uso acessível por parte de todos. Com os avanços ao nível de inteligência artificial, especialmente no que toca à criação de sistemas inteligentes que comuniquem de forma natural com os humanos, torna-se possível criar agentes de conversação adequados para uma interação pessoa-máquina com distintos objetivos. Um dos objetivos que o projeto ONParkinson tem é o de aumentar a adesão terapêutica por parte das pessoas com doença de Parkinson. Sendo que a execução recorrente de exercício físico é essencial na gestão dos sintomas da doença de Parkinson. Por isso, existe a necessidade de interagir, educar e motivar os pacientes com doença de Parkinson para uma maior adesão aos exercícios terapêuticos. Este trabalho propõe uma solução, no âmbito do projeto ONParkinson, que envolve a criaçãao de um agente de conversação com unidades de conhecimento mais focadas nos exercicios terapêuticos e com unidades que visam motivar e manter a pessoa com doença de Parkinson motivada para a realização de exercícios terapêuticos. A avaliação da solução envolve fisioterapeutas e pessoas com doença de Parkinson. O plano de avaliação estabelece o estudo do desempenho técnico, da experiência do utilizador e da investigação na área da Saúde. Grande parte do conjunto dos pacientes com doença de Parkinson tem uma idade avançada, o que poderia levar a uma maior resistência ao uso das novas tecnologias. No entanto, os valores obtidos nos indicadores referentes à perspetiva de utilidade, facilidade de uso e satisfação da utilização demonstram um bom nível de usabilidade da solução proposta. Como a investigação de eficácia clínica ainda não foi conduzida, não é possível concluir a efiácia da solução proposta no aumento da adesão terapêutica por parte dos pacientes com doença de Parkinson.Nowadays, we live surrounded by technology, where smartphones fill a very important space in our lives. The use of mobile services by smartphones in the health sector has been increasingly prosperous, with accessible use by everyone. With advances in artificial intelligence methodologies, regarding the creation of intelligent systems that communicate naturally with humans, it is possible to create conversational agents for person-machine interaction with different objectives. One of the goals of the ONParkinson project is to increase therapeutic adherence by people with Parkinson’s disease. The recurrent execution of physical exercise is essential in the management of the symptoms of Parkinson’s disease. Therefore, there is a need to interact, educate and motivate patients with Parkinson’s disease for greater adherence to therapeutic exercises. This work proposes a solution, within the scope of the ONParkinson project, which involves the creation of a conversation agent with units of knowledge more focused on therapeutic exercises and with units aiming to motivate and keep the person with Parkinson’s disease motivated to perform therapeutic exercises. The evaluation of the solution involves physical therapists and patients with Parkinson’s disease. The evaluation plan establishes the study of technical performance, the study of user experience and Health research study. A large part of the set of patients with Parkinson’s disease is of advanced age, which could lead to greater resistance to the use of new technologies. However, the values obtained in the indicators referring to the perception of usefulness, ease of use and interaction satisfaction demonstrate a good level of usability of the proposed solution. As the investigation of clinical efficacy has not yet been conducted, it is not possible to conclude the effectiveness of the proposed solution in increasing therapeutic adherence by patients with Parkinson’s disease