An action selection architecture for autonomous virtual humans in persistent worlds

Nowadays, virtual humans such as non-player characters in computer games need to have a strong autonomy in order to live their own life in persistent virtual worlds. When designing autonomous virtual humans, the action selection problem needs to be considered, as it is responsible for decision making at each moment in time. Indeed action selection architectures for autonomous virtual humans need to be reactive, proactive, motivational, and emotional to obtain a high degree of autonomy and individuality. The thesis can be divided into three parts. In the first part, we define each word of our title to precise their sense and raise the problematic of this work. We describe also inspirations from several domains that we used to design our model because this thesis is highly multi-disciplinary. Indeed, decision-making is essential for every autonomous entity and is studied in ethology, robotics, computer graphics, computer sciences, and cognitive sciences. However, we have chosen specific techniques to implement our model: hierarchical classifier systems and a free flow hierarchy. The second part of this thesis describes in detail our model of action selection for autonomous virtual humans. We use overlapping hierarchical classifier systems, working in parallel, to generate coherent behavioral plans. They are associated with the functionalities of a free flow hierarchy for the spreading of activation to give reactivity and flexibility to the hierarchical system. Moreover several functionalities are added to enhance and facilitate the choice of the most appropriate action at every time according to the internal and external influences. Finally, in the third part of this thesis, a complex simulated environment is created for testing the model and its functionalities with many conflicting motivations. Results demonstrate that the model is sufficiently efficient, robust and flexible for designing motivational autonomous virtual humans in persistent worlds. Moreover, we have just started to investigate on the emotional level which has to be improved in the future to have more subjective and adaptive behaviors and also manage social interactions with other virtual humans or users. Applied to video games, non player characters are more interesting and believable because they live their own life when people don't interact with them

Evaluation of a virtual agent to train medical students conducting psychiatric interviews for diagnosing major depressive disorders

Background: A psychiatric diagnosis involves the physician's ability to create an empathic interaction with the patient in order to accurately extract semiology (i.e., clinical manifestations). Virtual patients (VPs) can be used to train these skills but need to be evaluated in terms of accuracy, and to be perceived positively by users. Methods: We recruited 35 medical students who interacted in a 35-min psychiatric interview with a VP simulating major depressive disorders. Semiology extraction, verbal and non-verbal empathy were measured objectively during the interaction. The students were then debriefed to collect their experience with the VP. Results: The VP was able to simulate the conduction of a psychiatric interview realistically, and was effective to discriminate students depending on their psychiatric knowledge. Results suggest that students managed to keep an emotional distance during the interview and show the added value of emotion recognition software to measure empathy in psychiatry training. Students provided positive feedback regarding pedagogic usefulness, realism and enjoyment in the interaction. Limitations: Our sample was relatively small. As a first prototype, the measures taken by the VP would need improvement (subtler empathic questions, levels of difficulty). The face-tracking technique might induce errors in detecting non-verbal empathy. Conclusion: This study is the first to simulate a realistic psychiatric interview and to measure both skills needed by future psychiatrists: semiology extraction and empathic communication. Results provide evidence that VPs are acceptable by medical students, and highlight their relevance to complement existing training and evaluation tools in the field of affective disorders.Bordeaux Region Aquitaine Initiative for NeurosciencePhénotypage humain et réalité virtuelleInitiative d'excellence de l'Université de Bordeau

Smartphone-Based Virtual Agents Can Help the General Population Concerned by Sleep Complaints: A Proof-of-Concept Study During COVID-19 Confinement

Background: The COVID-19 crisis induces psychosocial stress and sleep complaints that require early management. KANOPEE is a smartphone-based application, providing an interaction with a virtual agent dedicated to screen and deliver behavioral interventions to fight sleep disorders. This paper describes the feasibility study of this application, during the context of COVID-19 confinement in France. Method: 2,069 users of aged 18 years and over downloaded the app during the inclusion period (between 22 April and 5 May 2020). Users first answered a screening interview based on the insomnia severity index (ISI) that was conducted by the virtual agent. If participants were positive for insomniac complaints (ISI > 14), they could join a two-stage intervention program: a) complete an electronic sleep diary for one week, and b) follow personalized sleep recommendations for 10 days. Measures collected included socio-demographic information, ISI and sleep/wake schedules; and acceptance and trust of the agent.Bordeaux Region Aquitaine Initiative for NeurosciencePhénotypage humain et réalité virtuell

J Med Internet Res

Background: The COVID-19 crisis and consequent confinement restrictions have caused significant psychosocial stress and reports of sleep complaints, which require early management, have increased during recent months. To help individuals concerned about their sleep, we developed a smartphone-based app called KANOPEE that allows users to interact with a virtual agent dedicated to autonomous screening and delivering digital behavioral interventions. Objective: Our objective was to assess the feasibility of this app, in terms of inclusion rate, follow-up rate, perceived trust and acceptance of the virtual agent, and effects of the intervention program, in the context of COVID-19 confinement in France. Methods: The virtual agent is an artificial intelligence program using decision tree architecture and interacting through natural body motion and natural voice. A total of 2069 users aged 18 years and above downloaded the free app during the study period (April 22 to May 5, 2020). These users first completed a screening interview based on the Insomnia Severity Index (ISI) conducted by the virtual agent. If the users were positive for insomnia complaints (ISI score >14), they were eligible to join the 2-stage intervention program: (1) complete an electronic sleep diary for 1 week and (2) follow personalized sleep recommendations for 10 days. We collected and analyzed the following measures: sociodemographic information, ISI scores and sleep/wake schedules, and acceptance and trust of the agent. Results: Approximately 76% (1574/2069) of the app users completed the screening interview with the virtual agent. The virtual agent was well accepted by 27.4% (431/1574) of the users who answered the acceptance and trust questionnaires on its usability, satisfaction, benevolence, and credibility. Of the 773 screened users who reported sleep complaints (ISI score >14), 166 (21.5%) followed Step 1 of the intervention, and only 47 of those (28.3%) followed Step 2. Users who completed Step 1 found that their insomnia complaints (baseline mean ISI score 18.56, mean ISI score after Step 1 15.99; P21) did not respond to either intervention. Conclusions: These preliminary results suggest that the KANOPEE app is a promising solution to screen populations for sleep complaints and that it provides acceptable and practical behavioral advice for individuals reporting moderately severe insomnia.Bordeaux Region Aquitaine Initiative for NeurosciencePhénotypage humain et réalité virtuell

Effectiveness and Acceptance of a Smartphone-Based Virtual Agent Screening for Alcohol and Tobacco Problems and Associated Risk Factors During COVID-19 Pandemic in the General Population

Background: During the current COVID-19 pandemic, alcohol, and tobacco are the most available substances for managing stress and can induce a risk of addiction. KANOPEE is a smartphone application available to the general population using an embodied conversational agent (ECA) to screen for experiences of problems with alcohol/tobacco use and to provide follow-up tools for brief intervention.Objectives: This study aimed to determine if the smartphone KANOPEE application could identify people at risk for alcohol and/or tobacco use disorders in the context of the current COVID-19 pandemic, to assess adherence to a 7-day follow-up use diary, and to evaluate trust and acceptance of the application.Methods: The conversational agent, named Jeanne, interviewed participants about perceived problems with the use of alcohol and tobacco since the pandemic and explored risk for tobacco and alcohol use disorder with the five-item Cigarette Dependence Scale (CDS-5) and “Cut Down, Annoyed, Guilty, Eye-opener” (CAGE) questionnaire and experience of craving for each substance. Descriptive, univariate, and multivariate analyses were performed to specify personalized associations with reporting a problem with alcohol/tobacco use; descriptive analysis reported the experience with the intervention and acceptance and trust in the application.Results: From April 22 to October 26, 2020, 1,588 French participants completed the KANOPEE interview, and 318 answered the acceptance and trust scales. Forty-two percent of tobacco users and 27% of alcohol users reported problem use since the pandemic. Positive screening with CDS-5 and CAGE and craving were associated with reported problem use (p < 0.0001). Lockdown period influenced alcohol (p < 0.0005) but not tobacco use (p > 0.05). Eighty-eight percent of users reported that KANOPEE was easy to use, and 82% found Jeanne to be trustworthy and credible.Conclusion: KANOPEE was able to screen for risk factors for substance use disorder (SUD) and was acceptable to users. Reporting craving and being at risk for SUD seem to be early markers to be identified. Alcohol problem use seems to be more reliant on contextual conditions such as confinement. This method is able to offer acceptable, brief, and early intervention with minimal delay for vulnerable people.Bordeaux Region Aquitaine Initiative for NeurosciencePhénotypage humain et réalité virtuell

Front Psychiatry

The rate of individuals with addiction who are currently treated are low, and this can be explained by barriers such as stigma, desire to cope alone, and difficulty to access treatment. These barriers could be overcome by mobile technologies. EMI (Ecological Momentary Intervention) is a treatment procedure characterized by the delivery of interventions (messages on smartphones) to people in their daily lives. EMI presents opportunities for treatments to be available to people during times and in situations when they are most needed. Craving is a strong predictor of relapse and a key target for addiction treatment. Studies using Ecological Momentary Assessment (EMA) method have revealed that, in daily life, person-specific cues could precipitate craving, that in turn, is associated with a higher probability to report substance use and relapse in the following hours. Assessment and management of these specific situations in daily life could help to decrease addictive use and avoid relapse. The Craving-Manager smartphone app has been designed to diagnose addictive disorders, and assess and manage craving as well as individual predictors of use/relapse. It delivers specific and individualized interventions (counseling messages) composed of evidence-based addiction treatments approaches (cognitive behavioral therapy and mindfulness). The Craving-Manager app can be used for any addiction (substance or behavior). The objective of this protocol is to evaluate the efficacy of the Craving-Manager app in decreasing use (of primary substance(s)/addictive behavior(s)) over 4 weeks, among individuals on a waiting list for outpatient addiction treatment. This multicenter double-blind randomized controlled trial (RCT) will compare two parallel groups: experimental group (full interventional version of the app, 4 weeks, EMA + EMI), versus control group (restricted version of the app, 4 weeks, only EMA). Two hundred and seventy-four participants will be recruited in 6 addiction treatment centers in France. This RCT will provide indication on how the Craving-Manager app will reduce addictive use (e.g., better craving management, better stimulus control) in both substance and behavioral addictions. If its efficacy is confirmed, the app could offer the possibility of an easy to use and personalized intervention accessible to the greatest number of individuals with addiction. ClinicalTrials.gov: NCT04732676

Une architecture de sélection de l'action pour des humains virtuels autonomes dans des mondes persistants

Nowadays, virtual humans such as non-player characters in computer games need to have a strong autonomy in order to live their own life in persistent virtual worlds. When designing autonomous virtual humans, the action selection problem needs to be considered, as it is responsible for decision making at each moment in time. Indeed action selection architectures for autonomous virtual humans need to be reactive, proactive, motivational, and emotional to obtain a high degree of autonomy and individuality. The thesis can be divided into three parts. In the first part, we define each word of our title to precise their sense and raise the problematic of this work. We describe also inspirations from several domains that we used to design our model because this thesis is highly multi-disciplinary. Indeed, decision-making is essential for every autonomous entity and is studied in ethology, robotics, computer graphics, computer sciences, and cognitive sciences. However, we have chosen specific techniques to implement our model: hierarchical classifier systems and a free flow hierarchy. The second part of this thesis describes in detail our model of action selection for autonomous virtual humans. We use overlapping hierarchical classifier systems, working in parallel, to generate coherent behavioral plans. They are associated with the functionalities of a free flow hierarchy for the spreading of activation to give reactivity and flexibility to the hierarchical system. Moreover several functionalities are added to enhance and facilitate the choice of the most appropriate action at every time according to the internal and external influences. Finally, in the third part of this thesis, a complex simulated environment is created for testing the model and its functionalities with many conflicting motivations. Results demonstrate that the model is sufficiently efficient, robust and flexible for designing motivational autonomous virtual humans in persistent worlds. Moreover, we have just started to investigate on the emotional level which has to be improved in the future to have more subjective and adaptive behaviors and also manage social interactions with other virtual humans or users. Applied to video games, non player characters are more interesting and believable because they live their own life when people don't interact with them.De nos jours, les humains virtuels ont besoin d'une grande autonomie pour pouvoir vivre leur propre vie dans des mondes virtuels persistants comme les personnages non joueurs dans les jeux vidéo. Lors de la conception d'humains virtuels autonomes, la problématique de la sélection de l'action doit être prise en compte car elle est responsable de leur prise de décision à chaque instant. En effet, les architectures de sélection de l'action pour les humains virtuels autonomes doivent être réactives, dirigées par des buts, et intégrer des motivations et des émotions pour obtenir un haut niveau d'autonomie et d'individualité. Cette thèse peut être divisée en trois parties. Dans la première partie, nous définissons chaque mot de notre titre pour en préciser leur sens et poser la problématique de ce travail. Nous décrivons ensuite les domaines dont nous nous sommes inspirés pour élaborer notre modèle car le sujet de ce travail est très multidisciplinaire. En effet, la prise de décision est essentielle pour toute entité autonome et est étudiée en éthologie, robotique, infographie, informatique, et dans les sciences cognitives. Cependant nous avons choisi certaines techniques spécifiques pour implémenter notre modèle parmi toutes celles possibles : les systèmes de classeurs hiérarchiques et les hiérarchies à libre flux. La seconde partie de cette thèse décrit en détail notre modèle de sélection de l'action pour des humains virtuels. Nous avons utilisé des systèmes de classeurs hiérarchiques, fonctionnant en parallèle, pour générer des plans de comportements cohérents. Ils sont associés avec les fonctionnalités des hiérarchies à libre flux pour la propagation de l'activité car elles donnent une grande flexibilité et réactivité aux systèmes hiérarchiques. Plusieurs fonctionnalités ont été ajoutées pour améliorer et faciliter le choix de l'action la plus appropriée, à chaque instant, par rapport aux influences internes et externes. Finalement, dans la troisième partie de la thèse, un environnement virtuel complexe est créé avec beaucoup de motivations conflictuelles pour tester notre architecture et ses fonctionnalités. Les résultats démontrent que le modèle est suffisamment efficace, robuste, et flexible pour concevoir des humains virtuels motivés et autonomes dans des mondes virtuels persistants. De plus, nous venons de commencer les investigations au niveau des émotions dans notre modèle et nous projetons de continuer dans le futur pour avoir des comportements plus subjectifs et plus adaptés aux différentes situations ainsi que pour gérer les interactions sociales avec d'autres humains virtuels ou des utilisateurs. Appliquée aux jeux vidéo, notre architecture de sélection de l'action pour des humains virtuels autonomes dans des mondes persistants rendrait les personnages non-joueurs plus intéressants et plus réalistes car ils pourraient vivre leur propre vie lorsqu'ils ne sont pas en interaction avec les joueurs

An affective model of action selection for virtual humans

The goal of our work aims at implementing progressively an action selection affective model for virtual humans that should be in the end autonomous, adaptive and sociable. Affect, traditionally distinguished from "cold " cognition, regroups emotions and motivations which are highly intertwined. We present a bottom-up approach by implementing first a motivational model of action selection to obtain motivationally autonomous virtual humans. For the adaptability of virtual humans and completeness of our affective model of action selection, we will define the interactions between motivations and emotions in order to integrate an emotional layer. In order to understand how they affect decision making in virtual humans, the motivations should represent more quantitative aspect of the decision making whereas emotions should be more qualitative one.

Constructing Virtual Human Life Simulations

This paper describes an approach to construct interactive virtual environments, which are suitable for the development of artificial virtual human life simulations. Our main goal is to have virtual human actors living and working autonomously in virtual environments. In our approach, virtual actors have their own motivations and needs, and by sensing and exploring their environment, an action selection mechanism is able to determine the suitable actions to take. Such actions often involve interaction with the environment and thus a specific technique to define actor-object interactions is used, where pre-defined interaction plans are put inside interactive objects, and just selected during the simulation. We explain in this paper the steps taken in order to construct and animate such environments, and we also present a test simulation example