Bi, Y

McTear, Mike

Varghese Marokkie, Sheen

English

With life expectancy growing rapidly over the past century, there is a need to find new ways to support active and healthy ageing and address those factors that can affect the well-being and emotional state of older adults. Various digital solutions have been proposed, including the use of a virtual coach (or chatbot) that can use technologies from Conversational AI to provide companionship and support where tradi- tional resources have become scare and costly. This paper describes how some use cases that were identified in a large, ongoing joint European and Japanese research and development project can be implemented us- ing the Rasa Conversational AI framework. The focus of the paper is mainly on the development process showing how Rasa can be used ef- fectively to provide conversational support across a number of relevant use cases and how to test and validate the implementations. The paper concludes with a brief usability study and suggestions for future work

Ulster University's Research Portal

                      A virtual companion for older adults using the Rasa Conversational AI frameworkVarghese Marokkie, S., McTear, M., & Bi, Y. (2022). A virtual companion for older adults using the RasaConversational AI framework. In A. Følstad (Ed.), CONVERSATIONS 2022 (pp. 1-7).https://conversations2022.files.wordpress.com/2022/11/conversations_2022_positionpaper_10_mctear.pdfLink to publication record in Ulster University Research PortalPublished in:CONVERSATIONS 2022Publication Status:Published (in print/issue): 22/11/2022Document VersionPeer reviewed versionGeneral rightsCopyright for the publications made accessible via Ulster University's Research Portal is retained by the author(s) and / or other copyrightowners and it is a condition of accessing these publications that users recognise and abide by the legal requirements associated with theserights.Take down policyThe Research Portal is Ulster University's institutional repository that provides access to Ulster's research outputs. Every effort has beenmade to ensure that content in the Research Portal does not infringe any person's rights, or applicable UK laws. If you discover content inthe Research Portal that you believe breaches copyright or violates any law, please contact pure-support@ulster.ac.uk.Download date: 20/01/2023A virtual companion for older adults using theRasa Conversational AI frameworkSheen Varghese Marokkie1, Michael McTear1, and Yaxin Bi1Ulster University {Varghese_Marokkie-S, mf.mctear, y.bi}@ulster.ac.ukhttp://www.ulster.ac.ukAbstract. This position paper describes how the Rasa Conversa-tional AI framework can be used to implement a variety of differentuse cases involving conversational support for older adults.Keywords: Active and healthy ageing · Conversational AI · Rasaframework.1 IntroductionThis position paper describes how a set of use cases were implementedusing the Rasa Conversational AI framework. The use cases were devel-oped in the e-VITA project, which is a 3 year joint European (H2020) andJapanese (MIC) funded research project that addresses a wide range ofdomains within active and healthy ageing [1].12 Related WorkThere has been increasing interest in recent years in exploring hownew developments in conversational AI can be applied to support ac-tive and healthy ageing. [2] presents a comprehensive state-of-the-artreview, while [3] reports a month-long study of interactions between avirtual agent and older adults in their homes. [4] shows how dialoguetechnology can provide natural interaction, while [5] describes how in-formation about the older adult’s emotional status was extracted froman analysis of their verbal and non-verbal communication. [6] is an on-going project in the Netherlands with the aim of improving the lives ofolder adults using voice technology. [7] reviews current work on virtualcoaches, discussing theories and future research directions.1 The e-VITA project has received funding from the European Union H2020 Pro-gramme under grant agreement no. 101016453. The Japanese consortiumreceived funding from the Japanese Ministry of Internal Affairs and Commu-nication (MIC), Grant no. JPJ000595.2 S. Varghese et al.3 The e-VITA virtual coach requirementsRequirements for the e-VITA virtual coach were gathered in a dynamicprocess of participatory design in real life settings involving interviewswith older community-dwelling adults in Germany, Italy, France, andJapan. Based on the interviews a number of use cases were identified.The current work focuses on use cases associated with support of dailyliving, helping to perform tasks, and providing information.4 The Rasa Conversational AI frameworkRasa is an open-source and customizable framework using state-of-the-art machine learning-based technologies that enable developers to cre-ate advanced conversational assistants [8]. The main concepts in Rasaare: intents, entities, responses, forms,slots, actions, and stories.Intents and entities are extracted from the user’s utterances usingRasa’s Natural Language Understanding (NLU) module which uses ma-chine learning-based technologies to classify the intents of the user’sutterances and extract entities from the utterances. The developer pro-vides examples of typical user utterances that are used to train the sys-tem and create an NLU model.Responses are the system outputs and can take the form of text orspeech as well as buttons, attachments, and other media. Forms areused to collect information from the user, for example, for the itemsrequired to make a flight booking. This information is stored in slotsas key-value pairs e.g. city:London, acting as the system’s memory andpotentially influencing the system’s future behaviour.Actions specify what the system should do following a user’s input,including making a response, making an external API call, or queryinga database or other knowledge source. Actions defined by the developerare called custom actions.Stories are declarative representations of the conversation flow andare used along with rules and a Transformer Embedding Dialogue (TED)policy to determine the next action that the system should take in theconversation. Knowledge Graph technology and Knowledge Bases are in-tegrated in Rasa using specific knowledge-based actions that can supplycontent for the system’s responses [9].5 Use cases in the current workThis section describes how the use cases identified for the e-VITA virtualcoach can be implemented using the Rasa framework. The code reposi-tory for the examples is available in [10] and a video recording of someinteractions with the virtual coach that was developed in this work canbe viewed in [11].A virtual companion for older adults using Rasa 3Fig. 1. Storing information about the user.5.1 Storing information about the userIn an initial interaction with a user the virtual coach should ask for theuser’s name and other relevant information. In Rasa this can be doneusing a form in which there are required slots to store the user’s first andlast names. Fig.1 shows a simple example in which the coach requeststhe user’s name at the beginning of an initial interaction and then usesthis information in a subsequent exchange. Forms can also be used formore complex use cases to request and store useful details about theuser, such as next-of-kin, carer, preferences, hobbies, etc. that will beuseful in future encounters involving daily support.5.2 Monitoring the user’s emotional stateA virtual coach that provides daily support and companionship for anolder adult should be able to engage in small talk (also known as chit-chat) in order to provide companionship as well as enabling the coachto obtain potentially important information about the user’s well being.Fig.2 shows a simple example where the user says that they are feelingand on hearing this the chatbot asks for more information and thensuggests playing some music.In the e-VITA project an emotion detection system is being used toassess the user’s emotional state based on analysis of their spoken ut-terances. Since the interactions in the current work were text-based, anapproach using sentiment analysis was developed in which the user’sutterance was submitted to the spaCy NLP sentiment analysis modulespacytextblob which can be used as a pipeline component in the Rasaframework [13]. The sentiment analysis module returns a range of po-larities where numbers above 0 indicate a positive emotional state andnumbers below 0 indicate a negative emotional state. The coach can thenrespond in an appropriate way. For example, the coach’s response to a4 S. Varghese et al.Fig. 2. Sentiment analysis in chat.negative sentiment could be anything like asking to share feelings, mu-sic recommendations, jokes, etc., as shown in Fig. 2. The output couldalso be tailored to some aspect of the the user’s personal profile thatwould help to formulate a response to a negative sentiment.In the current work there were some situations where this approachdid not work well. For example, for statements like I don’t feel so good ornot bad, the sentiment analysis module was unable to classify the cor-rect sentiments and responded with confusing statements. More workis needed using more sophisticated NLU analysis to train the module oninputs such as these.5.3 Helping the user perform a taskThe virtual coach can help the user perform tasks such as sending anemail or text message, making a phone call, or accessing a music APIwithout having to interact with the complex functions of an app on adevice such as a smartphone. In this use case a form can be used toelicit the values of the required slots. In Fig. 3 the system uses a customaction involving an email form with slots to elicit the recipient’s emailaddress, the subject of the email, and the text of the message. Havingfilled these slots a method is called to submit the values in the form toan email API.5.4 Obtaining information from an external APIThe virtual coach can get information such as weather updates, news,or sports results from an external API and output the information to theuser. Fig. 4 shows an example where the user asks what is the weatherlike?. This utterance is classified as a weather intent that triggers aweather story that includes an action to extract the city – in this case,as the user has not specified the city, the default is set to Belfast. Inother cases the coach could ask for the city and other slots such asdate and time. The value for city is then submitted to a weather APIusing a custom action and a response is created showing the maximumand minimum temperature and the overall weather status of the city.A virtual companion for older adults using Rasa 5Fig. 3. Sending a email.Fig. 4. Query for weather information.5.5 Accessing a knowledge sourceIn the e-VITA project a range of knowledge sources are used, includingdatabases, internet resources, sensor information, emotion detection,common knowledge, and knowledge about the user. Rasa can maintaina knowledge base for data access. One way in which this can be done is touse an in-memory knowledge base in the form of a JSON file. The virtualcoach in the current work makes use of an extract from a knowledge basedeveloped in the e-VITA project in which there is an object_type calleddiseases, as shown in Fig. 5.Fig. 6 shows an example of some sample utterances to query theknowledge base using the query_knowledge_base intent.If the user’s utterance is classified as an intent, an action is invokedwithin a story to access the knowledge base and retrieve a response,mapping the attribute name identified in the utterance (e.g. symptoms,cure) on to the attribute in the JSON knowledge base and retrieving itsassociated value as the coach’s response as shown in Fig.7.6 Conclusions and Future WorkA virtual coach can be of great assistance for older adults and the usecases described here are examples of how Conversational AI can be usedto create such a virtual coach. Future technical work will extend the ap-proach to sentiment analysis by integrating with an emotion detection6 S. Varghese et al.Fig. 5. JSON code for diseases.Fig. 6. Utterances to query the knowledge base.Fig. 7. Interactions with the diseases knowledge base.A virtual companion for older adults using Rasa 7system linked to a speech-based interface in order to detect emotionalcues in acoustic parameters in the user’s speech. The current knowledgebase will be extended using knowledge graphs to enable more complexknowledge modelling covering a range of topics and to enable tailoringto individual users through personal profiles. The range of dialogues in-volving information retrieval is being extended using access to APIs suchas Wikidata in different languages, and dialogues are being developedthat will require multi-turn conversations about topics such as fall pre-vention, nutrition, and physical and cognitive exercise. The virtual coachwill be tested in living labs and in the homes of older adults to ensureuser acceptance and general usability.References1. EU-Japan virtual coach for smart ageing, https://www.e-vita.coach/. Lastaccessed 4 Oct 20172. Lima, M.R., Wairagkar, M., Gupta, M., Rodriguez y Baena, F., Barnaghi, P.,Sharp, D.J.,Vaidyanathan, R.: Conversational Affective Social Robots for Age-ing and Dementia Support. IEEE Trans. Cogn. Dev. Syst., (99):1-1 (2021)3. Sidner, C.L., Bickmore, T., Nooraie, B., Rich, C., Ring, L., Shayganfar, M., Var-doulakis, L.: Creating new technologies for companionable agents to supportisolated older adults. ACM Trans. Interact. Intell. Syst. 1, 1, Article 1 (2017)4. Jokinen, K.: Dialogue Models for Socially Intelligent Robots. In: SocialRobotics. ICSR 2018. Lecture Notes in Computer Science, vol. 11357.Springer, Cham., 127-138v (2018)5. Justo, R., Ben Letaifa, L., Palmero, C., Gonzalez-Fraile, E., Torp Johansen,A., Vázquez, A., Cordasco, G., Schlögl, S., Fernandez-Ruanova, B., Silva, M.,Escalera, S., deVelasco, M., Tenorio-Laranga, J., Esposito, A., Korsnes, M.,Torres, M.I.: Analysis of the interaction between elderly people and a simu-lated virtual coach. J Ambient Intell Humaniz Comput, 11:6125–6140 (2020)6. Project Zilver: Research and expertise to develop voice technology for olderadults, https://projectzilver.com/en/Zilver. Last accessed 4 Oct 20177. Weimann, T.G., Schlieter, H., Brendel, A.B..: Virtual coaches: Background,theories, and future research directions. Bus Inf Syst Eng 64(4): 515-528https://doi.org/10.1007/s12599-022-00757-9 (2022)8. Bocklisch, T., Faulkner, J., Pawlowski, N., & Nichol, A.: Rasa: Opensource language understanding and dialogue management. arXiv preprintarXiv:1712.05181 (2017)9. Wilcock, G., Jokinen, K.: Conversational AI and Knowledge Graphs for SocialRobot Interaction. Late-Breaking Reports. The 17th ACM/IEEE InternationalConference on Human-Robot Interaction (HRI-2022) (2022)10. Github source https://github.com/sheenv2004/CompanionBot11. Video recording of some interactions with the chatbot.https://ulster.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=99639857-40ae-4113-984e-af0301885d4712. spaCy models and languages. https://spacy.io/usage/models/

A virtual companion for older adults using the Rasa Conversational AI framework

https://pure.ulster.ac.uk/files/110783206/conversations_2022_positionpaper_10_mctear.pdf

A virtual companion for older adults using the Rasa Conversational AI framework

Abstract

Similar works

Full text

Available Versions

Ulster University's Research Portal