Fully generated scripted dialogue for embodied agents

This paper presents the NECA approach to the generation of dialogues between Embodied Conversational Agents (ECAs). This approach consist of the automated construction of an abstract script for an entire dialogue (cast in terms of dialogue acts), which is incrementally enhanced by a series of modules and finally ''performed'' by means of text, speech and body language, by a cast of ECAs. The approach makes it possible to automatically produce a large variety of highly expressive dialogues, some of whose essential properties are under the control of a user. The paper discusses the advantages and disadvantages of NECA's approach to Fully Generated Scripted Dialogue (FGSD), and explains the main techniques used in the two demonstrators that were built. The paper can be read as a survey of issues and techniques in the construction of ECAs, focusing on the generation of behaviour (i.e., focusing on information presentation) rather than on interpretation

The un/real duet : intimacy & agency through interaction with a virtual character

Thesis (M.S.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1998.Includes bibliographical references (leaves 101-103) and index.by Freedom Baird.M.S

Toward a second-person neuroscience

LS & BT : equal contributions (shared first-authorship)Peer reviewedPreprin

Human emotion simulation in a dynamic environment

The aim of this work is to contribute to the believability of the simulated emotions for virtual entities to allow them display human like features. Endowing virtual entities with such features requires an appropriate architecture and model. For that, a study of emotional models from different perspective is undertaken. The fields include Psychology, Organic Components, Attention study and Computing. Two contributions are provided to reach the aim. The first one is a computational emotional model based on Scherer’s theory (K. Scherer, 2001). This contribution allows to generate a series of modifications in the affective state from one event by contrast to the existing solutions where one emotion is mapped to one single event. Several theories are used to make the model concrete. The second contribution make use of attention theories to build a paradigm in the execution of tasks in parallel. An algorithm is proposed to assess the available resources and allocate them to tasks for their execution. The algorithm is based on the multiple resources theory by Wickens (Wickens, 2008). The two contributions are combined into one architecture to produce a dynamic emotional system that allows its components to work in parallel. The first contribution was evaluated using a questionnaire. The results showed that mapping one event into a series of modifications in the affective state can enhance the believability of the simulation. The results also showed that people who develop more variations in the affective state are more perceived to be feminine

Gotta face ‘em all: Pokémon, Japanese animated characters, and the emergence of playful visual animism

As a result of technological innovations and new cultural practices, the contemporary mediasphere is increasingly populated by digital(ized) faces. The phenomenon is not limited to human faces, but includes a vast universe of fictional animated faces, variously called ‘characters’, ‘mascots’ or ‘kyara’. In Japan, while certainly not new, kyara have been spreading thanks to globalization, digitalization and media-mix strategies. Through the connection between visual design, fictional narratives and socio-cultural consumption, kyara can be considered semiotic figures of in-betweenness, key symbolic mediators in the Japanese mediascape. Their anthropomorphic face design mediates the cultural boundaries between the human and the non-human, the animate and the inanimate, nature and culture. Furthermore, their post-modern narratives mix inspiration from the past and the present, from myths to science fiction. Lastly, they involve an encyclopedic reworking between fiction and reality, mythical references and secularization, between the domains of seriousness and playful make-believe. The article aims to explore the semiotic dimensions of kyara in contemporary Japan, with emphasis on their logic of representation and cultural outcomes. These will be investigated through the analysis of the Pokémon franchise, which will make it possible to describe the emergence of new semiotic patterns of ‘playful visual animism’ in the process of media facialization of everyday life.    &nbsp

Procedural-Reasoning Architecture for Applied Behavior Analysis-based Instructions

Autism Spectrum Disorder (ASD) is a complex developmental disability affecting as many as 1 in every 88 children. While there is no known cure for ASD, there are known behavioral and developmental interventions, based on demonstrated efficacy, that have become the predominant treatments for improving social, adaptive, and behavioral functions in children. Applied Behavioral Analysis (ABA)-based early childhood interventions are evidence based, efficacious therapies for autism that are widely recognized as effective approaches to remediation of the symptoms of ASD. They are, however, labor intensive and consequently often inaccessible at the recommended levels. Recent advancements in socially assistive robotics and applications of virtual intelligent agents have shown that children with ASD accept intelligent agents as effective and often preferred substitutes for human therapists. This research is nascent and highly experimental with no unifying, interdisciplinary, and integral approach to development of intelligent agents based therapies, especially not in the area of behavioral interventions. Motivated by the absence of the unifying framework, we developed a conceptual procedural-reasoning agent architecture (PRA-ABA) that, we propose, could serve as a foundation for ABA-based assistive technologies involving virtual, mixed or embodied agents, including robots. This architecture and related research presented in this disser- tation encompass two main areas: (a) knowledge representation and computational model of the behavioral aspects of ABA as applicable to autism intervention practices, and (b) abstract architecture for multi-modal, agent-mediated implementation of these practices

Multiliteracies meaning-making: How four boys’ video gaming experiences influence their cultural knowledge—Two ethnographic cases

Scholars have acknowledged the potential contribution of video gaming to complex forms of learning, identifying links between gaming and engagement, experiential learning spaces, problem-solving, strategies, transliteracy reflectivity, critical literacy, and metacognitive thinking. Despite this movement toward the inclusion of video gaming in literacy teaching, concerns about certain risks raised by scholars have slowed the adoption of using video games to foster learning. Using a multiliteracies lens, this multi-case study examined the experiences of four boys engaged with video gaming in two different contexts: a community centre and an after-school video club. By drawing on Feminist Post-Structural Theory, Vygotskian, and video gaming technology, I have gained an understanding of the nature of boys’ behavior and learning in social settings while they engage in video game play. Studying the ways in which boys make meanings through multimodal ways of learning can offer insights into strategies that can potentially reinvent traditional literacy pedagogical boundaries and establish new ways and practices for building knowledge. These ethnographic cases, along with their naturalistic aspects, strengthened the authenticity of the social-contextual-cultural experiences of the four, adolescent-aged boys and allowed an understanding of their everyday experiences. Interpretations of the cultural meanings made by each of the boys, based on their individual unique experiences engaging with video games, can provide readers with insights into how to approach adolescent aged boys’ literacy development. This study describes how these four boys developed their multimodal ways of learning by engaging with visual perspectives of video games. My methodological approach documented what boys are saying, as much as possible, which is currently understudied in the literature surrounding boys and their video gaming practices. There were a number of findings emanating from this study, including the following: (i) boys use their video gaming practices for meaning-making and collaborative efforts in order to gain an understanding of several knowledge processes (such as decision-making, predicting, analyzing, strategizing, etc.), (ii) boys extend and apply their cultural knowledge as creative innovators, producing and publishing YouTube instructional videos for video game players and designing video games for a history project, (iii) boys demonstrate peer mentoring through storytelling, face-to-face interactions or in their online community of practice, (iv) boys make meanings using metacognitive literacy skills in a variety of ways, and (v) boys focus on cultural preservation and narrative storytelling. While acknowledging concerns related to video gaming, such as negative identity construction, violence, distraction, and time commitment for integration, this study seeks to contribute to the scholarly discussion about the use of video games in classrooms by explicitly considering the ways in which gaming may support boys’ meaning-making and cultural knowledge. Keywords Available designs, boys, community of practice, cultural meaning-making systems, literacy, multiliteracies, multimodal meanings, video gamin

Environnements virtuels émotionnellement intelligents

Les émotions ont été étudiées sous différents angles dans le domaine de l'interaction homme-machine y compris les systèmes tutoriel intelligents, les réseaux sociaux, les plateformes d’apprentissage en ligne et le e-commerce. Beaucoup d’efforts en informatique affective sont investis pour intégrer la dimension émotionnelle dans les environnements virtuels (tel que les jeux vidéo, les jeux sérieux et les environnements de réalité virtuelle ou de réalité augmenté). Toutefois, les stratégies utilisées dans les jeux sont encore empiriques et se basent sur des modèles psychologiques et sociologiques du joueur : Courbe d’apprentissage, gestion de la difficulté, degré d’efficience dans l’évaluation des performances et de la motivation du joueur. Or cette analyse peut malmener le système dans la mesure où les critères sont parfois trop vagues ou ne représentent pas les réelles compétences du joueur, ni ses vraies difficultés. Étant donné que la stratégie d’intervention est très influencée par la précision de l’analyse et l’évaluation du joueur, de nouveaux moyens sont nécessaires afin d’améliorer les processus décisionnels dans les jeux et d’organiser les stratégies d’adaptation de façon optimale. Ce travail de recherche vise à construire une nouvelle approche pour l’évaluation et le suivi du joueur. L’approche permet une modélisation du joueur plus efficace et moins intrusive par l’intégration des états mentaux et affectifs obtenus à partir de senseurs physiologiques (signaux cérébraux, Activité électrodermale, …) ou/et instruments optiques (Webcam, traceur de regard, …). Les états affectifs et mentaux tels que les émotions de base (basées sur les expressions faciales), l’état d’engagement, de motivation et d’attention sont les plus visés dans cette recherche. Afin de soutenir l’adaptation dans les jeux, des modèles des émotions et de la motivation du joueur basé sur ces indicateurs mentaux et affectifs, ont été développés. Nous avons implémenté cette approche en développant un système sous forme d’une architecture modulaire qui permet l’adaptation dans les environnements virtuels selon les paramètres affectifs du joueur détectés en temps-réel par des techniques d’intelligence artificielle.Emotions were studied from different angles in the field of human-machine interaction including intelligent tutorial systems, social networks, online learning platforms and e-commerce. Much effort in affective computing are invested to integrate the emotional dimension in virtual environments (such as video games, serious games and virtual reality environments or augmented reality). However, the strategies used in games are still empirical and are based on psychological and sociological models of the player: Learning Curve, trouble management, degree of efficiency in the evaluation of performance and motivation of the player. But this analysis can mislead the system to the extent that the criteria are sometimes too vague and do not represent the actual skills of the player, nor his real difficulties. Since the intervention strategy is influenced by the accuracy of the analysis and evaluation of the player, new ways are needed to improve decision-making in games and organizing adaptation strategies in optimal way. This research aims to build a new approach to the evaluation and monitoring of the player. The approach enables more effective and less intrusive player modeling through the integration of mental and emotional states obtained from physiological sensors (brain signals, electro-dermal activity, ...) or/and optical instruments (Webcam, eye-tracker, ...). The emotional and mental states such as basic emotions (based on facial expressions), the states of engagement, motivation and attention are the most targeted in this research. In order to support adaptation in games, models of emotions and motivation of the player based on these mental and emotional indicators, have been developed. We have implemented this approach by developing a system in the form of a modular architecture that allows adaptation in virtual environments according to the player's emotional parameters detected in real time by artificial intelligence methods