Usability and acceptability assessment of an empathic virtual agent to prevent major depression

In Human-Computer Interaction, the adaptation of the content and the way of how this content is communicated to the users in interactive sessions is a critical issue to promote the acceptability and usability of any computational system. We present a user-adapted interactive platform to identify and provide an early intervention for symptoms of depression and suicide. In particular, we describe the work performed to assess users' system acceptability and usability. An empathic Virtual Agent is the main interface with the user, and it has been designed to generate the appropriate dialogues and emotions during the interactions according to the detected user's specific needs. This personalization is based on a dynamic user model nurtured with clinical, demographical and behavioural information. The evaluation was performed with 60 participants from the university community. The obtained results were promising, allowing the execution of a further clinical trial. The system's usability score was 75.7%, and the score of the user-adapted content and the emotional responses of the Virtual Agent was 70.9%.The work presented in this manuscript has been partially funded by the Conselleria de Sanidad of Generalitat Valenciana, in the research project entitled 'Sistema computacional de ayuda a la prevencion de episodios de depresion y suicidio - PREVENDEP'. We thank the company Faceshift (www.faceshift.com) for providing their software to perform facial motion capture in order to develop the talking head that represent our empathic virtual agent.Bresó Guardado, A.; Martinez-Miranda, J.; Botella Arbona, C.; Baños Rivera, RM.; García Gómez, JM. (2016). Usability and acceptability assessment of an empathic virtual agent to prevent major depression. Expert Systems. 33(4):297-312. doi:10.1111/exsy.12151S29731233

A Computational Framework for Planning Therapeutical Sessions aimed to Support the Prevention and Treatment of Mental Health Disorders using Emotional Virtual Agents

[EN] Interaction is defined as the realization of a reciprocal action between two or more people or things. Particularly in computer science, the term interaction refers to the discipline that studies the exchange of information between people and computers, and is generally known by the term Human-Computer Interaction (HCI). Good design decisions and an adequate development of the software is required for efficient HCI to facilitate the acceptability of computer-based applications by the users. In clinical settings it is essential to eliminate any barrier and facilitate the interaction between patients and the system. A smooth communication between the user and the computer-based application is fundamental to maximise the advantages and functionalities offered by the system. The design of these applications must consider the personal and current needs of the user by applying a User-Centered Design methodology. The main purpose of this research work is to contribute in the improvement of HCI-based applications addressed to the clinical context, particularly to enhance computer-based interactive sessions to support people suffering from a mental disorder such as Major Depression (MD). Thanks to the advances in Artificial Intelligence techniques, it is now possible to partially automate complex tasks such as the continuous provision of Cognitive-Behavioural Therapies (CBTs) to patients. These CBTs require good levels of adaptability and variability during the interaction with the patient that facilitates the acceptability in the user, an optimal usability and good level of engagement for a successful mid/long term use of the application and treatment adherence. The modelling of complex deliberative and affective processes in artificial systems can be applied to support the prevention and treatment of mental health related issues, enhancing the continuous and remote assistance of patients, saving some economical and clinical resources and reducing the waiting lists in the health services. In this regard, the efforts of this Thesis have been concentrated on the research of two main lines: (1) the generation and planning of adequate contents in an interactive system to support the prevention and treatment of MD based on characteristics of the user; and (2) the modelling of relevant affective processes able to communicate the contents in an emotional effective way taking into account the importance of the affective conditions associated with the MD in the users. Rule Based Systems and the appraisal theory of emotions have been the roots used to develop the main two modules of the computational Framework presented: the Contents Management and the Emotional Modules. Finally, the obtained Framework was integrated into two interactive systems to evaluate the achievement of the research objectives. The first system has been developed in the context of the Help4Mood European research project and its main aim was to support the remote treatment of patients with MD. The second scenario was a system developed to prevent MD and suicidal thoughts in the University community, which was developed in the context of the local PrevenDep research project. These evaluations have indicated that the proposed Framework has reached good levels of usability and acceptability in the target users thanks to the personalizations and adaptation capabilities of the contents and in the way how these contents are communicated to the user. The research work and the obtained results in this Thesis has contributed to the state of the art in HCI-based systems used as support in therapeutic interventions for the prevention and treatment of MD. This was obtained by the combination of a personalized content management to the patient, and the management of the affective processes associated to these pathologies. The developed work also identifies some research lines that need to be addressed in future works to get better HCI systems used for therapeutic purposes.[ES] Interactuar se define como la realización de una acción recíproca entre dos o más personas o cosas. Particularmente en informática, el término interacción se refiere a la disciplina que estudia el intercambio de información entre las personas y computadoras, y suele conocerse por el término anglosajón Human-Computer Interaction (HCI). Un buen diseño y un adecuado desarrollo del software es necesario para lograr una HCI eficiente que facilite la aceptabilidad del sistema por el usuario. En entornos clínicos es fundamental eliminar cualquier tipo de barrera y facilitar la interacción entre los pacientes y el computador. Es de vital importancia que haya una buena comunicación entre usuario y computador, por este motivo el sistema debe de estar diseñado pensando en las necesidades actuales, cambiantes y personales del usuario, basándose en la metodología de diseño centrado en el usuario. El propósito principal de esta investigación es la identificación de mejoras en HCI aplicada en entornos clínicos, en concreto para dar soporte a personas con trastornos mentales como la Depresión Mayor (DM) y que precisan de terapias psicológicas adecuadas y continuas. Gracias a técnicas de Inteligencia Artificial, es posible automatizar eficientemente ciertas acciones asociadas a los procesos de las terapias cognitivo-conductuales (CBTs, del inglés Cognitive-Behavioural Therapies). Los sistemas de ayuda a la CBT, requieren de una adaptabilidad y variabilidad en la interacción para favorecer la usabilidad del sistema y asegurar la continuidad de la motivación del paciente. Una buena gestión de esta automatización influiría en la aceptabilidad de los pacientes y podría mejorar su adherencia a los tratamientos y por consiguiente mejorar su estado de salud. Adicionalmente, la unión de procesos deliberativos dinámicos pueden liberar recursos clínicos, mejorando el control de los pacientes, y reduciendo los tiempos de espera y los costes económicos. En este sentido, los esfuerzos de esta Tesis se han centrado en la investigación de dos líneas diferentes: (1) la selección y planificación adecuada de los contenidos presentados durante la interacción a través de una planificación dinámica y personalizada, y (2) la adecuación de la comunicación de los contenidos hacia el paciente tomando en cuenta la importancia de los procesos afectivos asociados a estas patologías. Los Sistemas Basados en Reglas (SBR) han sido la herramienta utilizada para dar soporte a los dos módulos principales que componen el Framework presentado en esta Tesis: el módulo de gestión de los contenidos y el módulo emocional. Concluida la fase de diseño, desarrollo y testeo, el Framework fue adaptado e integrado en sistemas reales, para validar la viabilidad y la adecuación del marco de trabajo de esta Tesis. En primer lugar, el sistema se aplicó durante tres años en el tratamiento de la DM en varios centros clínicos europeos en el contexto del Proyecto Europeo de investigación Help4Mood. Finalmente, el sistema fue evaluado en la tarea de prevención de la DM y del suicidio en el Proyecto Local de investigación PrevenDep, de un año de duración. El feedback de estas evaluaciones demostraron que el HCI del Framework tiene unos niveles altos de usabilidad y aceptación, gracias a la personalización, variabilidad y adaptación de los contenidos y de la comunicación de los mismos. Los experimentos computacionales llevados a cabo en esta Tesis han permitido avanzar el estado del arte de sistemas computacionales emocionales aplicados en entornos terapéuticos para la prevención y tratamiento de la DM. Principalmente, gracias a la combinación de una gestión personalizada de los contenidos hacia el paciente tomando en cuenta la importancia de los procesos afectivos asociados a estas patologías. Este trabajo abre nuevas líneas de investigación, como la aplicación de este sistema en otras patologías de salud mental en las qu[CA] Interactuar es defineix com la realització d'una acció recíproca entre dos o més persones o coses. Particularment en informàtica, el terme interacció es refereix a la disciplina que estudia l'intercanvi d'informació entre les persones i computadores, i es sol conèixer pel terme anglosaxó Human-Computer Interaction (HCI). Un bon disseny i un adequat desenvolupament del software és necessari per aconseguir una HCI eficient que faciliti l'acceptabilitat del sistema per l'usuari. En entorns clínics és fonamental eliminar qualsevol tipus de barrera i facilitar la interacció entre els pacients i el computador. És de vital importància que hi hagi una bona comunicació entre l'usuari (o pacient) i el computador, per aquest motiu el sistema ha d'estar dissenyat pensant en les necessitats actuals, cambiants i personals de l'usuari, basant-se en la metodologia de disseny centrat en l'usuari. El propòsit principal d'aquesta investigació és la identificació de millores en HCI aplicada en entorns clínics, en concret per donar suport a persones amb trastorns mentals com la Depressió Major (DM) i que precisen de teràpies psicològiques adequades i contínues. Gràcies a tècniques d'Intel·ligència Artificial, és possible automatitzar eficientment certes accions asociades al processos de les teràpies cognitiu-conductuals. Els sistemes computacionals de ajuda a la CBT, requereixen d'una adaptabilitat i variabilitat en la interacció per afavorir la usabilitat del sistema i assegurar la continuïtat de la motiviació del pacient. Una bona gestió d'aquesta automatització influiria en l'acceptabilitat dels pacients i podria millorar la seva adherència als tractaments i per tant millorar el seu estat de salut. Addicionalment, la unió de processos deliberatius dinàmics poden alliberar recursos clínics, millorant el control dels pacients, i reduint els temps d'espera i els costos econòmics. En aquest sentit, els esforços d'aquesta Tesi s'han centrat en la investigació de dues línies diferents: (1) la selecció i planificació adequada dels continguts presentats durant la interacció a través d'una planificació dinàmica i personalitzada, i (2) l'adequació de la comunicació dels continguts cap al pacient tenint en compte la importància dels processos afectius associats a aquestes patologies. Els Sistemes Basats en Regles (SBR) han estat la eina utilitzada per donar suport als dos mòduls principals que componen el Framework presentat en aquesta Tesi: el mòdul de gestió dels continguts oferits a l'usuari; i el mòdul emocional. Conclosa la fase de disseny, desenvolupament i testeig, el Framework va ser adaptat als dominis corresponents i integrat en sistemes madurs per ser avaluat en dos escenaris reals, per validar la viabilitat i l'adequació del Framework d'aquesta tesi. Primerament, el sistema es va aplicar durant tres anys en el tractament de la DM major en diversos centres clínics europeus en el context del Projecte Europeu d'investigació Help4Mood. Finalment, el sistema va ser avaluat en la tasca de prevenció de la DM i del suïcidi al Projecte Local d'investigació PrevenDep, d'un any de durada. El feedback de les avaluacions han demostrat que el HCI del Framework obté uns nivells alts d'usabilitat i acceptació, gràcies a la personalització, variabilitat i adaptació dels continguts i de la comunicació. Els experiments computacionals duts a terme en aquesta Tesi han permès avançar l'estat de l'art de sistemes computacionals emocionals aplicats en entorns terapèutics per a la prevenció i tractament de la DM. Principalment, gracies a la combinació d'una gestió personalitzada dels continguts cap al pacient tenint en compte la importància dels processos afectius associats a aquestes patologies. Aquest treball obre noves línies d'investigació, com l'aplicació d'aquest sistema en altres patologies de salut mental en què sigui recomanable l'aplicació de sessions terapèutiques.Bresó Guardado, A. (2016). A Computational Framework for Planning Therapeutical Sessions aimed to Support the Prevention and Treatment of Mental Health Disorders using Emotional Virtual Agents [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/64082TESI

Conversational Agents for depression screening: a systematic review

Objective: This work explores the advances in conversational agents aimed at the detection of mental health disorders, and specifically the screening of depression. The focus is put on those based on voice interaction, but other approaches are also tackled, such as text-based interaction or embodied avatars. Methods: PRISMA was selected as the systematic methodology for the analysis of existing literature, which was retrieved from Scopus, PubMed, IEEE Xplore, APA PsycINFO, Cochrane, and Web of Science. Relevant research addresses the detection of depression using conversational agents, and the selection criteria utilized include their effectiveness, usability, personalization, and psychometric properties. Results: Of the 993 references initially retrieved, 36 were finally included in our work. The analysis of these studies allowed us to identify 30 conversational agents that claim to detect depression, specifically or in combination with other disorders such as anxiety or stress disorders. As a general approach, screening was implemented in the conversational agents taking as a reference standardized or psychometrically validated clinical tests, which were also utilized as a golden standard for their validation. The implementation of questionnaires such as Patient Health Questionnaire or the Beck Depression Inventory, which are used in 65% of the articles analyzed, stand out. Conclusions: The usefulness of intelligent conversational agents allows screening to be administered to different types of profiles, such as patients (33% of relevant proposals) and caregivers (11%), although in many cases a target profile is not clearly of (66% of solutions analyzed). This study found 30 standalone conversational agents, but some proposals were explored that combine several approaches for a more enriching data acquisition. The interaction implemented in most relevant conversational agents is textbased, although the evolution is clearly towards voice integration, which in turns enhances their psychometric characteristics, as voice interaction is perceived as more natural and less invasive.Agencia Estatal de Investigación | Ref. PID2020-115137RB-I0

An Embodied Conversational Agent to Minimize the Effects of Social Isolation During Hospitalization

Social isolation and loneliness contribute to the development of depression and anxiety. Comorbidity of mental health issues in hospitalized patients increases the length of stay in hospital by up to 109% and costs the healthcare sector billions of dollars each year. This study aims to understand the potential suitability of embodied conversational agents (ECAs) to reduce feelings of social isolation and loneliness among hospital patients. To facilitate this, a video prototype of an ECA was developed for use in single-occupant hospital rooms. The ECA was designed to act as an intelligent assistant, a rehabilitation guide, and a conversational partner. A co-design workshop involving five healthcare professionals was conducted. The thematic analysis of the workshop transcripts identified some major themes including improving health literacy, reducing the time burden on healthcare professionals, preventing secondary mental health issues, and supporting higher acceptance of digital technologies by elderly patients

An overview of the features of chatbots in mental health: A scoping review

Background: Chatbots are systems that are able to converse and interact with human users using spoken, written, and visual languages. Chatbots have the potential to be useful tools for individuals with mental disorders, especially those who are reluctant to seek mental health advice due to stigmatization. While numerous studies have been conducted about using chatbots for mental health, there is a need to systematically bring this evidence together in order to inform mental health providers and potential users about the main features of chatbots and their potential uses, and to inform future research about the main gaps of the previous literature. Objective: We aimed to provide an overview of the features of chatbots used by individuals for their mental health as reported in the empirical literature. Methods: Seven bibliographic databases (Medline, Embase, PsycINFO, Cochrane Central Register of Controlled Trials, IEEE Xplore, ACM Digital Library, and Google Scholar) were used in our search. In addition, backward and forward reference list checking of the included studies and relevant reviews was conducted. Study selection and data extraction were carried out by two reviewers independently. Extracted data were synthesised using a narrative approach. Chatbots were classified according to their purposes, platforms, response generation, dialogue initiative, input and output modalities, embodiment, and targeted disorders. Results: Of 1039 citations retrieved, 53 unique studies were included in this review. Those studies assessed 41 different chatbots. Common uses of chatbots were: therapy (n = 17), training (n = 12), and screening (n = 10). Chatbots in most studies were rule-based (n = 49) and implemented in stand-alone software (n = 37). In 46 studies, chatbots controlled and led the conversations. While the most frequently used input modality was writing language only (n = 26), the most frequently used output modality was a combination of written, spoken and visual languages (n = 28). In the majority of studies, chatbots included virtual representations (n = 44). The most common focus of chatbots was depression (n = 16) or autism (n = 10). Conclusion: Research regarding chatbots in mental health is nascent. There are numerous chatbots that are used for various mental disorders and purposes. Healthcare providers should compare chatbots found in this review to help guide potential users to the most appropriate chatbot to support their mental health needs. More reviews are needed to summarise the evidence regarding the effectiveness and acceptability of chatbots in mental health

Technical Metrics Used to Evaluate Health Care Chatbots: Scoping Review

Dialog agents (chatbots) have a long history of application in health care, where they have been used for tasks such as supporting patient self-management and providing counseling. Their use is expected to grow with increasing demands on health systems and improving artificial intelligence (AI) capability. Approaches to the evaluation of health care chatbots, however, appear to be diverse and haphazard, resulting in a potential barrier to the advancement of the field. This study aims to identify the technical (nonclinical) metrics used by previous studies to evaluate health care chatbots. Studies were identified by searching 7 bibliographic databases (eg, MEDLINE and PsycINFO) in addition to conducting backward and forward reference list checking of the included studies and relevant reviews. The studies were independently selected by two reviewers who then extracted data from the included studies. Extracted data were synthesized narratively by grouping the identified metrics into categories based on the aspect of chatbots that the metrics evaluated. Of the 1498 citations retrieved, 65 studies were included in this review. Chatbots were evaluated using 27 technical metrics, which were related to chatbots as a whole (eg, usability, classifier performance, speed), response generation (eg, comprehensibility, realism, repetitiveness), response understanding (eg, chatbot understanding as assessed by users, word error rate, concept error rate), and esthetics (eg, appearance of the virtual agent, background color, and content). The technical metrics of health chatbot studies were diverse, with survey designs and global usability metrics dominating. The lack of standardization and paucity of objective measures make it difficult to compare the performance of health chatbots and could inhibit advancement of the field. We suggest that researchers more frequently include metrics computed from conversation logs. In addition, we recommend the development of a framework of technical metrics with recommendations for specific circumstances for their inclusion in chatbot studies

Insights from mechanistic and digital intervention approaches

Eine effektive Förderung des Sozialverhaltens autistischer Kinder erfordert ein tiefgehendes Verständnis der Ursachen maladaptiver Reaktionen und die Bereitstellung eines leicht verfügbaren Förderangebots. Hier haben digitale Angebote ein großes Potential. Ebenso wie bei der Ursachenforschung mangelt es jedoch an Forschungsansätzen, welche eine Vielzahl an kognitiven und emotionalen Prozessen in die digitale Förderung integrieren. Dementsprechend untersuchte die Dissertation zunächst das Zusammenspiel verschiedener Ursachen aggressives Sozialverhaltens anhand eines etablierten Modells der sozial-kognitiven Informationsverarbeitung. Durch die Integration verschiedener Facetten der Empathie und deren zugrundeliegende Kompetenzen in ein digitales Förderangebot sollte im zweiten Schritt eine Verbesserung des Sozialverhaltens autistischer Grundschüler:innen erreicht werden. Es zeigte sich, dass Emotionsdysregulation verschiedene Formen aggressiven Sozialverhaltens und damit assoziierte feindselige Attributionen verstärkt. Letztere stand vor allem mit verbalen und verdeckten Aggressionsformen sowie mit guten Emotionserkennungsfertigkeiten im Zusammenhang. Eine Verbesserung des Sozialverhaltens und der Emotionsregulation konnte mittelfristig durch das sechswöchige, eltern-begleitete eLearningprogramm „Zirkus Empathico“ erreicht werden. Die multizentrische, randomisiert kontrollierte Studie ergab zudem kurzfristige und moderate Interventionseffekte für Empathie und Emotionserkennung als primäre Endpunkte. Insgesamt unterstreicht die Dissertation die Validität etablierter Modelle der sozialen Informationsverarbeitung sowie die Relevanz, diese zukünftigen Forschungs- und Interventionsansätzen zugrunde zu legen. Durch die Integration verschiedener sozio-emotionaler Kompetenzen scheint die digitale Intervention Zirkus Empathico prosoziales Verhalten autistischer Kinder auf effektive und praktikable Weise zu fördern.Effective training of autistic children`s social behavior requires an in-depth understanding of the causes of maladaptive responses and the provision of easily accessible support services. In this context, digital interventions have great potential. However, there is a lack of research approaches that integrate a variety of cognitive and emotional processes into both, explanation and digital support. The present dissertation first examined the interplay of different causes of aggressive social behavior by applying an established model of social-cognitive information processing to a sample of autistic elementary school students. Second, by integrating different facets of empathy and their underlying competencies into a digital program, the social behavior of autistic elementary school children should be improved. First, it was shown that emotion dysregulation strengthens various forms of aggressive social behavior and associated hostile attribution biases. The latter was mainly related to verbal and covert forms of aggression and good emotion recognition skills. Second, the parent-assisted eLearning program "Zirkus Empathico" led to a medium-term improvement in social behavior and emotion regulation after a six-week training. In addition, the multicenter randomized controlled trial showed moderate intervention effects on empathy and emotion recognition as primary outcomes, which were no longer detectable three months later. Overall, the dissertation highlights the validity of established models of social information processing and the relevance of using them as a foundation for future research and intervention. By integrating various socio-emotional competencies, the digital intervention Zirkus Empathico seems to strengthen autistic children’s prosocial behavior effectively and feasibly

Psyche: A mobile application connecting students to school counselors

Capstone Project submitted to the Department of Engineering, Ashesi University in partial fulfillment of the requirements for the award of Bachelor of Science degree in Computer Engineering, April 2019This paper explores the problem of depression among university students and provides a means by which the students can be linked to a mental health professional, in order to receive guidance. This system is made up of a mobile application and an electrical system. The mobile application includes a chat system that allows the students to contact available counselors, and also, includes a virtual reality interface that allows the user to relax in a virtual environment of choice while listening to an audio of choice. This could be an instrumental, or natural sounds like the wind blowing, rain drops and othersa. The electrical system takes in the user’s temperature and heart rate and sends these data to the cloud database, to be displayed to the counselor in the mobile application as per the counselor’s request.Ashesi Universit