Collaborative Behaviour Modelling of Virtual Agents using Communication in a Mixed Human-Agent Teamwork

International audience—The coordination is an essential ingredient for the mixed human-agent teamwork. It requires team members to share knowledge to establish common grounding and mutual awareness among them. In this paper, we proposed a collaborative conversational belief-desire-intention (C 2 BDI) behavioural agent architecture that allows to enhance the knowledge sharing using natural language communication between team members. We defined collaborative conversation protocols that provide proactive behaviour to agents for the coordination between team members. Furthermore, to endow the communication capabilities to C 2 BDI agent, we described the information state based approach for the natural language processing of the utterances. We have applied the proposed architecture to a real scenario in a collaborative virtual environment for training. Our solution enables the user to coordinate with other team members

Communicative Capabilities of Agents for the Collaboration in a Human-Agent Team

International audienceThe coordination is an essential ingredient for the human-agent teamwork. It requires team members to share knowledge to establish common grounding and mutual awareness among them. In this paper, we propose a behavioral architecture C 2 BDI that allows to enhance the knowledge sharing using natural language communication between team members. We define collaborative conversation protocols that provide proactive behavior to agents for the coordination between team members. We have applied this architecture to a real scenario in a col-laborative virtual environment for training. Our solution enables users to coordinate with other team members

Collaborative Virtual Training with Physical and Communicative Autonomous Agents

International audienceVirtual agents are a real asset in collaborative virtual environment for training (CVET) as they can replace missing team members. Collaboration between such agents and users, however, is generally limited. We present here a whole integrated model of CVET focusing on the abstraction of the real or virtual nature of the actor to define a homogenous collaboration model. First, we define a new collaborative model of interaction. This model notably allows to abstract the real or virtual nature of a teammate. Moreover, we propose a new role exchange approach so that actors can swap their roles during training. The model also permits the use of physically based objects and characters animation to increase the realism of the world. Second, we design a new communicative agent model, which aims at improving collaboration with other actors using dialog to coordinate their actions and to share their knowledge. Finally, we evaluated the proposed model to estimate the resulting benefits for the users and we show that this is integrated in existing CVET applications

Collaborative Virtual Training with Physical and Communicative Autonomous Agents

International audienceVirtual agents are a real asset in collaborative virtual environment for training (CVET) as they can replace missing team members. Collaboration between such agents and users, however, is generally limited. We present here a whole integrated model of CVET focusing on the abstraction of the real or virtual nature of the actor to define a homogenous collaboration model. First, we define a new collaborative model of interaction. This model notably allows to abstract the real or virtual nature of a teammate. Moreover, we propose a new role exchange approach so that actors can swap their roles during training. The model also permits the use of physically based objects and characters animation to increase the realism of the world. Second, we design a new communicative agent model, which aims at improving collaboration with other actors using dialog to coordinate their actions and to share their knowledge. Finally, we evaluated the proposed model to estimate the resulting benefits for the users and we show that this is integrated in existing CVET applications

Des agents avec des capacités communicatives orientées tâche dans les environnements de réalité virtuelle collaboratifs pour l'apprentissage

Growing needs of educational and training requirements motivate the use of collaborative virtual environments for training (CVET) that allows human users to work together with autonomous agents to perform a collective activity. The vision is inspired by the fact that the effective coordination improves productivity, and reduces the individual and team errors. This work addresses the issue of establishing and maintaining the coordination in a mixed human-agent teamwork in the context of CVET. The objective of this research is to provide human-like conversational behavior of the virtual agents in order to cooperate with a user and other agents to achieve shared goals.We propose a belief-desire-intention (BDI) like Collaborative Conversational agent architecture(C2BDI) that treats both deliberative and conversational behaviors uniformly as guided by the goal-directed shared activity. We put forward an integrated model of coordination which is founded on the shared mental model based approaches to establish coordination in a human-agent teamwork. We argue that natural language interaction between team members can affect and modify the individual and shared mental models of the participants. Finally, we describe the cultivation of coordination in a mixed human-agent teamwork through natural language conversation. In order to establish the strong coupling between decision making and the collaborative conversational behavior of the agent, we propose first, the Mascaret based semantic modeling of human activities and the VE, and second, the information state based context model. This representation allows the treatment of semantic knowledge of the collaborative activity and virtual environment, and information exchanged during the dialogue conversation in a unified manner. This knowledge can be used by the agent for multiparty natural language processing (understanding and generation) in the context of the CEVT. To endow the communicative capabilities to C2BDI agent, we put forward the information state based approach for the natural language processing of the utterances. We define collaborative conversation protocols that ensure the coordination between team members. Finally, in this thesis, we propose a decision making mechanism, which is inspired by the BDI based approach and provides the interleaving between deliberation and conversational behavior of the agent. We have applied the proposed architecture to three different scenarios in the CVET. We found that the multiparty collaborative conversational behavior of C2BDI agent is more constructive and facilitates the user to effectively coordinate with other team members to perform a shared task.Les besoins croissants en formation et en entrainement au travail d’équipe ont motivé l’utilisationd’Environnements de réalité Virtuelle Collaboratifs de Formation (EVCF) qui permettent aux utilisateurs de travailler avec des agents autonomes pour réaliser une activité collective. L’idée directrice est que la coordination efficace entre les membres d’une équipe améliore la productivité et réduit les erreurs individuelles et collectives. Cette thèse traite de la mise en place et du maintien de la coordination au sein d’une équipe de travail composée d’agents et d’humains interagissant dans un EVCF.L’objectif de ces recherches est de doter les agents virtuels de comportements conversationnels permettant la coopération entre agents et avec l’utilisateur dans le but de réaliser un but commun.Nous proposons une architecture d’agents Collaboratifs et Conversationnels, dérivée de l’architecture Belief-Desire-Intention (C2-BDI), qui gère uniformément les comportements délibératifs et conversationnels comme deux comportements dirigés vers les buts de l’activité collective. Nous proposons un modèle intégré de la coordination fondé sur l’approche des modèles mentaux partagés, afin d’établir la coordination au sein de l’équipe de travail composée d’humains et d’agents. Nous soutenons que les interactions en langage naturel entre les membres d’une équipe modifient les modèles mentaux individuels et partagés des participants. Enfin, nous décrivons comment les agents mettent en place et maintiennent la coordination au sein de l’équipe par le biais de conversations en langage naturel. Afin d’établir un couplage fort entre la prise de décision et le comportement conversationnel collaboratif d’un agent, nous proposons tout d’abord une approche fondée sur la modélisation sémantique des activités humaines et de l’environnement virtuel via le modèle mascaret puis, dans un second temps, une modélisation du contexte basée sur l’approche Information State. Ces représentations permettent de traiter de manière unifiée les connaissances sémantiques des agents sur l’activité collective et sur l’environnement virtuel ainsi que des informations qu’ils échangent lors de dialogues.Ces informations sont utilisées par les agents pour la génération et la compréhension du langage naturel multipartite. L’approche Information State nous permet de doter les agents C2BDI de capacités communicatives leur permettant de s’engager pro-activement dans des interactions en langue naturelle en vue de coordonner efficacement leur activité avec les autres membres de l’équipe. De plus, nous définissons les protocoles conversationnels collaboratifs favorisant la coordination entre les membres de l’équipe. Enfin, nous proposons dans cette thèse un mécanisme de prise de décision s’inspirant de l’approche BDI qui lie les comportements de délibération et de conversation des agents. Nous avons mis en oeuvre notre architecture dans trois différents scénarios se déroulant dans des EVCF. Nous montrons que les comportements conversationnels collaboratifs multipartites des agents C2BDI facilitent la coordination effective de l’utilisateur avec les autres membres de l’équipe lors de la réalisation d’une tâche partagée

Pedagogical Agents as Team Members: Impact of Proactive and Pedagogical Behavior on the User

International audienceno abstrac

Interactions multiparties pour la coordination d'équipes mixtes humains-agents

National audienc

Towards Generic Multimodal Interaction Systems based on machine learning and context awareness

International audienc

A methodology for the design of pedagogically adaptable learning environments.

International audienceIn the last decades, the industry hasprofoundly integrated the use of digital resources intheir production process. However, these assets arerarely re-used for the training of the users, operatorsand technicians that have to interact with these objects.Furthermore, although training and learning environmentsare classical applications of virtual reality, thedesign of these environments is generally ad hoc, i.e.dedicated to specific operations on specific objects, hencerequiring the intervention of programmers whenevera modification of the pedagogical scenario is required.In this article, we propose a methodology to designadaptable virtual environments, by separating the roleof the different protagonists that play a part in thecreation of learning environments. In particular, itsgoal is to allow the teachers to implement differentscenarios according to the level of the trainees and tothe pedagogical objectives without the intervention ofcomputer scientists. An example of adaptable wind turbineenvironment is shown, with three different learningsituations: simulator, safety training and preventivemaintenance training

Using Multimodal Information to Enhance Addressee Detection in Multiparty Interaction

International audienceAddressee detection is an important challenge to tackle in order to improve dialogical interactions between humans and agents. This detection, essential for turn-taking models, is a hard task in multiparty conditions. Rule based as well as statistical approaches have been explored. Statistical approaches, particularly deep learning approaches, require a huge amount of data to train. However, smart feature selection can help improve addressee detection on small datasets, particularly if multimodal information is available. In this article, we propose a statistical approach based on smart feature selection that exploits contextual and multimodal information for addressee detection. The results show that our model outperforms an existing baseline