Evaluating humanoid embodied conversational agents in mobile guide applications

Evolution in the area of mobile computing has been phenomenal in the last few years. The exploding increase in hardware power has enabled multimodal mobile interfaces to be developed. These interfaces differ from the traditional graphical user interface (GUI), in that they enable a more “natural” communication with mobile devices, through the use of multiple communication channels (e.g., multi-touch, speech recognition, etc.). As a result, a new generation of applications has emerged that provide human-like assistance in the user interface (e.g., the Siri conversational assistant (Siri Inc., visited 2010)). These conversational agents are currently designed to automate a number of tedious mobile tasks (e.g., to call a taxi), but the possible applications are endless. A domain of particular interest is that of Cultural Heritage, where conversational agents can act as personalized tour guides in, for example, archaeological attractions. The visitors to historical places have a diverse range of information needs. For example, casual visitors have different information needs from those with a deeper interest in an attraction (e.g., - holiday learners versus students). A personalized conversational agent can access a cultural heritage database, and effectively translate data into a natural language form that is adapted to the visitor’s personal needs and interests. The present research aims to investigate the information needs of a specific type of visitors, those for whom retention of cultural content is important (e.g., students of history, cultural experts, history hobbyists, educators, etc.). Embodying a conversational agent enables the agent to use additional modalities to communicate this content (e.g., through facial expressions, deictic gestures, etc.) to the user. Simulating the social norms that guide the real-world human-to-human interaction (e.g., adapting the story based on the reactions of the users), should at least theoretically optimize the cognitive accessibility of the content. Although a number of projects have attempted to build embodied conversational agents (ECAs) for cultural heritage, little is known about their impact on the users’ perceived cognitive accessibility of the cultural heritage content, and the usability of the interfaces they support. In particular, there is a general disagreement on the advantages of multimodal ECAs in terms of users’ task performance and satisfaction over nonanthropomorphised interfaces. Further, little is known about what features influence what aspects of the cognitive accessibility of the content and/or usability of the interface. To address these questions I studied the user experiences with ECA interfaces in six user studies across three countries (Greece, UK and USA). To support these studies, I introduced: a) a conceptual framework based on well-established theoretical models of human cognition, and previous frameworks from the literature. The framework offers a holistic view of the design space of ECA systems b) a research technique for evaluating the cognitive accessibility of ECA-based information presentation systems that combine data from eye tracking and facial expression recognition. In addition, I designed a toolkit, from which I partially developed its natural language processing component, to facilitate rapid development of mobile guide applications using ECAs. Results from these studies provide evidence that an ECA, capable of displaying some of the communication strategies (e.g., non-verbal behaviours to accompany linguistic information etc.) found in the real-world human guidance scenario, is not affecting and effective in enhancing the user’s ability to retain cultural content. The findings from the first two studies, suggest than an ECA has no negative/positive impact on users experiencing content that is similar (but not the same) across different locations (see experiment one, in Chapter 7), and content of variable difficulty (see experiment two, in Chapter 7). However, my results also suggest that improving the degree of content personalization and the quality of the modalities used by the ECA can result in both effective and affecting human-ECA interactions. Effectiveness is the degree to which an ECA facilitates a user in accomplishing the navigation and information tasks. Similarly, affecting is the degree to which the ECA changes the quality of the user’s experience while accomplishing the navigation and information tasks. By adhering to the above rules, I gradually improved my designs and built ECAs that are affecting. In particular, I found that an ECA can affect the quality of the user’s navigation experience (see experiment three in Chapter 7), as well as how a user experiences narrations of cultural value (see experiment five, in Chapter 8). In terms of navigation, I found sound evidence that the strongest impact of the ECAs nonverbal behaviours is on the ability of users to correctly disambiguate the navigation of an ECA instructions provided by a tour guide system. However, my ECAs failed to become effective, and to elicit enhanced navigation or retention performances. Given the positive impact of ECAs on the disambiguation of navigation instructions, the lack of ECA-effectiveness in navigation could be attributed to the simulated mobile conditions. In a real outdoor environment, where users would have to actually walk around the castle, an ECA could have elicited better navigation performance, than a system without it. With regards to retention performance, my results suggest that a designer should not solely consider the impact of an ECA, but also the style and effectiveness of the question-answering (Q&A) with the ECA, and the type of user interacting with the ECA (see experiments four and six, in Chapter 8). I found that that there is a correlation between how many questions participants asked per location for a tour, and the information they retained after the completion of the tour. When participants were requested to ask the systems a specific number of questions per location, they could retain more information than when they were allowed to freely ask questions. However, the constrained style of interaction decreased their overall satisfaction with the systems. Therefore, when enhanced retention performance is needed, a designer should consider strategies that should direct users to ask a specific number of questions per location for a tour. On the other hand, when maintaining the positive levels of user experiences is the desired outcome of an interaction, users should be allowed to freely ask questions. Then, the effectiveness of the Q&A session is of importance to the success/failure of the user’s interaction with the ECA. In a natural-language question-answering system, the system often fails to understand the user’s question and, by default, it asks the user to rephrase again. A problem arises when the system fails to understand a question repeatedly. I found that a repetitive request to rephrase the same question annoys participants and affects their retention performance. Therefore, in order to ensure effective human-ECA Q&A, the repeat messages should be built in a way to allow users to figure out how to ask the system questions to avoid improper responses. Then, I found strong evidence that an ECA may be effective for some type of users, while for some others it may be not. I found that an ECA with an attention-grabbing mechanism (see experiment six, in Chapter 8), had an inverse effect on the retention performance of participants with different gender. In particular, it enhanced the retention performance of the male participants, while it degraded the retention performance of the female participants. Finally, a series of tentative design recommendations for the design of both affecting and effective ECAs in mobile guide applications in derived from the work undertaken. These are aimed at ECA researchers and mobile guide designers

Relational agents : effecting change through human-computer relationships

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2003.Includes bibliographical references (p. 205-219).What kinds of social relationships can people have with computers? Are there activities that computers can engage in that actively draw people into relationships with them? What are the potential benefits to the people who participate in these human-computer relationships? To address these questions this work introduces a theory of Relational Agents, which are computational artifacts designed to build and maintain long-term, social-emotional relationships with their users. These can be purely software humanoid animated agents--as developed in this work--but they can also be non-humanoid or embodied in various physical forms, from robots, to pets, to jewelry, clothing, hand-helds, and other interactive devices. Central to the notion of relationship is that it is a persistent construct, spanning multiple interactions; thus, Relational Agents are explicitly designed to remember past history and manage future expectations in their interactions with users. Finally, relationships are fundamentally social and emotional, and detailed knowledge of human social psychology--with a particular emphasis on the role of affect--must be incorporated into these agents if they are to effectively leverage the mechanisms of human social cognition in order to build relationships in the most natural manner possible. People build relationships primarily through the use of language, and primarily within the context of face-to-face conversation. Embodied Conversational Agents--anthropomorphic computer characters that emulate the experience of face-to-face conversation--thus provide the substrate for this work, and so the relational activities provided by the theory will primarily be specific types of verbal and nonverbal conversational behaviors used by people to negotiate and maintain relationships.(cont.) This work also provides an analysis of the types of applications in which having a human-computer relationship is advantageous to the human participant. In addition to applications in which the relationship is an end in itself (e.g., in entertainment systems), human-computer relationships are important in tasks in which the human is attempting to undergo some change in behavior or cognitive or emotional state. One such application is explored here: a system for assisting the user through a month-long health behavior change program in the area of exercise adoption. This application involves the research, design and implementation of relational agents as well as empirical evaluation of their ability to build relationships and effect change over a series of interactions with users.by Timothy Wallace Bickmore.Ph.D

Collaborative learning with affective artificial study companions in a virtual learning environment

This research has been carried out in conjunction with Chapeltown and Harehills Assisted Learning Computer School (CHALCS) and local schools. CHALCS is an 'out-of-hours' school in a deprived inner-city community where unemployment is high and many children are failing to meet their educational potential. As the name implies CHALCS provides students with access to computers to support their learning. CHALCS relies on many volunteer tutors and specialist tutors are in short supply. This is especially true for subjects such as Advanced Level Physics with low numbers of students. This research aimed to investigate the feasibility of providing online study skills support to pupils at CHALCS and a local school. Research suggests that collaborative learning that prompts students to explain and justify their understanding can encourage deeper learning. As a potentially effective way of motivating deeper learning from hypertext course notes in a Virtual Learning Environment (VLE), this research investigates the feasibility of designing an artificial Agent capable of collaborating with the learner to jointly construct summary notes. Hypertext course notes covering a portion of the Advanced Level Physics curriculum were designed and uploaded into a WebCT based VLE. A specialist tutor validated the content of the course notes before the ease of use of the VLE was tested with target students. A study was then conducted to develop a model of the kinds of help students required in writing summary notes from the course-notes. Based on the derived process model of summarisation and an analysis of the content structure of the course notes, strategies for summarising the text were devised. An Animated Pedagogical Agent was designed incorporating these strategies. Two versions of the agent with opposing 'Affectations' (giving the appearance of different characters) were evaluated with users. It was therefore possible to test which artificial 'character' students preferred. From the evaluation study some conclusions are made concerning the effect of the two opposite characterisations on student perceptions of the agent and the degree to which it was helpful as a learning companion. Some recommendations for future work are then made

Analysing user's reactions in advice-giving dialogues with a socially intelligent ECA

n this paper, we investigate the user's reactions to received suggestion by an Embodied Conversational Agent playing the role of artificial therapist in the healthy eating domain. Specifically, we analyse the behaviour of people who voluntarily requested to receive information from the agent, and we compare it with the results of a previous evaluation experiment in which subjects were not properly motivated to interact with the agent because they were selected for evaluating the system. This study is part of an ongoing research aimed at developing an intelligent virtual agent that applies natural argumentation techniques to persuade the users to improve their eating habits

Towards an Emotionally Intelligent Interaction Strategy for Multimodal Embodied Conversational Agents acting as Companions

Existing Human Computer Interaction (HCI) strategies are seriously limited by current technologies. These are neither sensitive nor accurate enough to respond to users’ emotional states, the fundamental basis for effective communication in real time. This offered the challenge of investigating factors that would impact on the designing of effective and more emotionally intelligent interaction strategies for Companions. These were applied to a conceptual tool, the Affective Channel (AC), to endow Companions with emotional capabilities. This was implemented in the Wizard of Oz (WoZ) platform to evaluate Companions in real time. The WoZ is an experimental setup where existing immature technologies and a human operator combine to simulate Companion interaction with end users. In these aspects of my work is my original contribution to the HCI knowledge base.Experiments, focus groups and face to face interviews were carried out to ascertain users’ perception and expectations of virtual agents. ‘Descriptors’ thus identified formed the bases for the designing of user friendly Companions. Verbal and facial expressions data and other affective elements of effective human-companion interactionwere collected for use in the AC and the WoZ as stated above.Companion evaluations yielded the subsidiary contribution that Companions are perceived as empathetic, useful and trustworthy entities. Further, that they arouse positive emotions in children and also that they promote their learning improvement.These findings were the result of two experiments, one within subjects and one between subjects, conducted with thirty grade four pupils in a rural school in the poor Oaxaca region of Mexico

“Gear is the Next Weed”:A Qualitative Exploration of the Beliefs, Attitudes and Behaviours of Performance and Image Enhancing Drug Using Subcultures in the South-West of England

Performance-enhancing drugs until relatively recently have been seen to be the preserve of sport-focussed athletes, but in recent years there has been an apparent increase in use amongst the general population, with individuals now using PIEDs not only to increase athletic prowess, but for image-conscious reasons entirely divorced from such ‘competitive’ notions. This research explores the different types of PIED user training in gym environments today, identifying differences in ‘ethnopharmacologies’ between these groups, allowing them to be categorised by their beliefs, attitudes, and patterns of use, based on qualitative data gathered ‘in the field’ from a total of 27 respondents, including 14 in-depth interviews. This exploration further evidences the extent to which a ‘normalisation’ of PIED use is occurring.Results suggest PIED users can be split into three categories, ‘sport-oriented’, ‘image-oriented’ and ‘hedonic’, with sport-oriented users conducting the most research, and having the most rigid cultural ‘disciplines’, and ‘hedonic’ users the least. This is evidenced through exploration of participants’ ‘decision to begin using’, their processes of ‘learning to use’ and their ‘longer term use’ of PIEDs, all of which suggest that patterns of use exist on a spectrum, from informed and cautious use employed by the most serious sport-focussed PIED users, to high-risk, high time-preference use associated with ‘hedonic’ users. This divergence in ethnopharmacologies and behaviours between groups evidences the need for such a categorisation of users in future research and policy, particularly for harm-minimisation purposes, as well as offering in-depth qualitative contributions to findings reported in recent longitudinal studies. Further, these elements of use evidence an increasing normalisation of PIEDs, which appears to have been largely achieved, excepting a perception of ‘stigmatisation’ still faced by users, principally stemming from media portrayals of ‘roid rage’. This limitation to cultural acceptance is therefore addressed, with evidence suggesting ‘roid rage’ is a ‘myth’, and further that this stigmatisation is likely to decline as knowledge is transferred from using populations to their non-using peers, indicating ‘normalisation’ is occurring.<br/