Usefulness, localizability, humanness, and language-benefit: additional evaluation criteria for natural language dialogue systems

Human–computer dialogue systems interact with human users using natural language. We used the ALICE/AIML chatbot architecture as a platform to develop a range of chatbots covering different languages, genres, text-types, and user-groups, to illustrate qualitative aspects of natural language dialogue system evaluation. We present some of the different evaluation techniques used in natural language dialogue systems, including black box and glass box, comparative, quantitative, and qualitative evaluation. Four aspects of NLP dialogue system evaluation are often overlooked: “usefulness” in terms of a user’s qualitative needs, “localizability” to new genres and languages, “humanness” or “naturalness” compared to human–human dialogues, and “language benefit” compared to alternative interfaces. We illustrated these aspects with respect to our work on machine-learnt chatbot dialogue systems; we believe these aspects are worthwhile in impressing potential new users and customers

Feel, Don\u27t Think Review of the Application of Neuroscience Methods for Conversational Agent Research

Conversational agents (CAs) equipped with human-like features (e.g., name, avatar) have been reported to induce the perception of humanness and social presence in users, which can also increase other aspects of users’ affection, cognition, and behavior. However, current research is primarily based on self-reported measurements, leaving the door open for errors related to the self-serving bias, socially desired responding, negativity bias and others. In this context, applying neuroscience methods (e.g., EEG or MRI) could provide a means to supplement current research. However, it is unclear to what extent such methods have already been applied and what future directions for their application might be. Against this background, we conducted a comprehensive and transdisciplinary review. Based on our sample of 37 articles, we find an increased interest in the topic after 2017, with neural signal and trust/decision-making as upcoming areas of research and five separate research clusters, describing current research trends

Advancing Multimedia: Application Sharing, Latency Measurements and User-Created Services

Online collaboration tools exist and have been used since the early days of the Internet. Asynchronous tools such as wikis and discussion boards and real-time tools such as instant messaging and voice conferencing have been the only viable collaboration solutions up until recently, due to the low bandwidth between participants. With the increasing bandwidth in computer networks, multimedia collaboration such as application sharing and video conferencing have become feasible. Application and desktop sharing allows sharing of any application with one or more people over the Internet. The participants receive the screen-view of the shared application from the server. Their mouse and keyboard events are delivered and regenerated at the server. Application and desktop sharing enables collaborative work, software tutoring, and e-learning over the Internet. I have developed a high performance application and desktop sharing system called BASS which is efficient, reliable, independent of the operating system, scales well via heterogeneous multicast, supports all applications, and features true application sharing. Most of the time an application sharing session requires audio and video conferencing to be more useful. High quality video conferencing requires a fair amount of bandwidth and unfortunately Internet bandwidth of home users is still limited and shared by more than one application and user. Therefore, I measured the performance of popular video conferencing applications under congestion to understand whether they are flexible enough to adapt to fluctuating and limited bandwidth conditions. In particular, I analyzed how Skype, Windows Live Messenger, Eyebeam and X-Lite react to changes in available bandwidth, presence of HTTP and BitTorrent traffic and wireless packet losses. To perform these measurements more effectively, I have also developed vDelay, a novel tool for measuring the capture-to-display latency (CDL) and frame rate of real-time video conferencing sessions. vDelay enables developers and testers to measure the CDL and frame rate of any video conferencing application without modifying the source code. Further, it does not require any specialized hardware. I have used vDelay to measure the CDL and frame rate of popular video chat applications including Skype, Windows Live Messenger, and GMail video chat. vDelay can also be used to measure the CDL and frame rate of these applications in the presence of bandwidth variations. The results from the performance study showed that existing products, such as Skype, adapt to bandwidth fluctuations fairly well and can differentiate wireless and congestion-based packet losses. Therefore, rather than trying to improve video conferencing tools, I changed my focus to end-user created communication-related services to increase the utility of existing stand alone Internet services, devices in the physical world, communication and online social networks. I have developed SECE (Sense Everything, Control Everything), a new language and its supporting software infrastructure for user created services. SECE allows non-technical end-users to create services that combine communication, social networks, presence, calendaring, location and devices in the physical world. SECE is an event-driven system that uses a natural-English-like language to trigger action scripts. Users associate actions with events and when an event happens its associated action is executed. Presence updates, social network updates, incoming calls, email, calendar and time events, sensor inputs and location updates can trigger rules. SECE retrieves all this information from multiple sources to personalize services and to adapt them to changes in the user's context and preferences. Actions can control the delivery of email, change the handling of phone calls, update social network status and set the state of actuators such as lights, thermostats and electrical appliances

Affective reactions towards socially interactive agents and their computational modeling

Over the past 30 years, researchers have studied human reactions towards machines applying the Computers Are Social Actors paradigm, which contrasts reactions towards computers with reactions towards humans. The last 30 years have also seen improvements in technology that have led to tremendous changes in computer interfaces and the development of Socially Interactive Agents. This raises the question of how humans react to Socially Interactive Agents. To answer these questions, knowledge from several disciplines is required, which is why this interdisciplinary dissertation is positioned within psychology and computer science. It aims to investigate affective reactions to Socially Interactive Agents and how these can be modeled computationally. Therefore, after a general introduction and background, this thesis first provides an overview of the Socially Interactive Agent system used in this work. Second, it presents a study comparing a human and a virtual job interviewer, which shows that both interviewers induce shame in participants to the same extent. Thirdly, it reports on a study investigating obedience towards Socially Interactive Agents. The results indicate that participants obey human and virtual instructors in similar ways. Furthermore, both types of instructors evoke feelings of stress and shame to the same extent. Fourth, a stress management training using biofeedback with a Socially Interactive Agent is presented. The study shows that a virtual trainer can teach coping techniques for emotionally challenging social situations. Fifth, it introduces MARSSI, a computational model of user affect. The evaluation of the model shows that it is possible to relate sequences of social signals to affective reactions, taking into account emotion regulation processes. Finally, the Deep method is proposed as a starting point for deeper computational modeling of internal emotions. The method combines social signals, verbalized introspection information, context information, and theory-driven knowledge. An exemplary application to the emotion shame and a schematic dynamic Bayesian network for its modeling are illustrated. Overall, this thesis provides evidence that human reactions towards Socially Interactive Agents are very similar to those towards humans, and that it is possible to model these reactions computationally.In den letzten 30 Jahren haben Forschende menschliche Reaktionen auf Maschinen untersucht und dabei das “Computer sind soziale Akteure”-Paradigma genutzt, in dem Reaktionen auf Computer mit denen auf Menschen verglichen werden. In den letzten 30 Jahren hat sich ebenfalls die Technologie weiterentwickelt, was zu einer enormen Veränderung der Computerschnittstellen und der Entwicklung von sozial interaktiven Agenten geführt hat. Dies wirft Fragen zu menschlichen Reaktionen auf sozial interaktive Agenten auf. Um diese Fragen zu beantworten, ist Wissen aus mehreren Disziplinen erforderlich, weshalb diese interdisziplinäre Dissertation innerhalb der Psychologie und Informatik angesiedelt ist. Sie zielt darauf ab, affektive Reaktionen auf sozial interaktive Agenten zu untersuchen und zu erforschen, wie diese computational modelliert werden können. Nach einer allgemeinen Einführung in das Thema gibt diese Arbeit daher, erstens, einen Überblick über das Agentensystem, das in der Arbeit verwendet wird. Zweitens wird eine Studie vorgestellt, in der eine menschliche und eine virtuelle Jobinterviewerin miteinander verglichen werden, wobei sich zeigt, dass beide Interviewerinnen bei den Versuchsteilnehmenden Schamgefühle in gleichem Maße auslösen. Drittens wird eine Studie berichtet, in der Gehorsam gegenüber sozial interaktiven Agenten untersucht wird. Die Ergebnisse deuten darauf hin, dass Versuchsteilnehmende sowohl menschlichen als auch virtuellen Anleiterinnen ähnlich gehorchen. Darüber hinaus werden durch beide Instruktorinnen gleiche Maße von Stress und Scham hervorgerufen. Viertens wird ein Biofeedback-Stressmanagementtraining mit einer sozial interaktiven Agentin vorgestellt. Die Studie zeigt, dass die virtuelle Trainerin Techniken zur Bewältigung von emotional herausfordernden sozialen Situationen vermitteln kann. Fünftens wird MARSSI, ein computergestütztes Modell des Nutzeraffekts, vorgestellt. Die Evaluation des Modells zeigt, dass es möglich ist, Sequenzen von sozialen Signalen mit affektiven Reaktionen unter Berücksichtigung von Emotionsregulationsprozessen in Beziehung zu setzen. Als letztes wird die Deep-Methode als Ausgangspunkt für eine tiefer gehende computergestützte Modellierung von internen Emotionen vorgestellt. Die Methode kombiniert soziale Signale, verbalisierte Introspektion, Kontextinformationen und theoriegeleitetes Wissen. Eine beispielhafte Anwendung auf die Emotion Scham und ein schematisches dynamisches Bayes’sches Netz zu deren Modellierung werden dargestellt. Insgesamt liefert diese Arbeit Hinweise darauf, dass menschliche Reaktionen auf sozial interaktive Agenten den Reaktionen auf Menschen sehr ähnlich sind und dass es möglich ist diese menschlichen Reaktion computational zu modellieren.Deutsche Forschungsgesellschaf

Foodbot: A goal-oriented just-in-time healthy eating interventions chatbot

National Research Foundation (NRF) Singapore under International Research Centres in Singapore Funding Initiativ

Factors That Enhance Consumer Trust in Human-Computer Interaction: An Examination of Interface Factors and Moderating Influences

The Internet coupled with agent technology presents a unique setting to examine consumer trust. Since the Internet is a relatively new, technically complex environment where human-computer interaction (HCI) is the basic communication modality, there is greater perception of risk facing consumers and hence a greater need for trust. In this dissertation, the notion of consumer trust was revisited and conceptually redefined adopting an integrative perspective. A critical test of trust theory revealed its cognitive (i.e., competence, information credibility), affective (i.e., benevolence), and intentional (i.e., trusting intention) constructs. The theoretical relationships among these trust constructs were confirmed through confirmatory factor analysis and structural equation modeling. The primary purpose of this dissertation was to investigate antecedent and moderating factors affecting consumer trust in HCI. This dissertation focused on interface-based antecedents of trust in the agent-assisted shopping context aiming at discovering potential interface strategies as a means to enhance consumer trust in the computer agent. The effects of certain interface design factors including face human-likeliness, script social presence, information richness, and price increase associated with upgrade recommendation by the computer agent were examined for their usefulness in enhancing the affective and cognitive bases for consumer trust. In addition, the role of individual difference factors and situational factors in moderating the relationship between specific types of computer interfaces and consumer trust perceptions was examined. Two experiments were conducted employing a computer agent, Agent John, which was created using MacroMedia Authorware. The results of the two experiments showed that certain interface factors including face and script could affect the affective trust perception. Information richness did not enhance consumers’ cognitive trust perceptions; instead, the percentage of price increase associated with Agent John’s upgrade recommendation affected individuals’ cognitive trust perceptions. Interestingly, the moderating influence of consumer personality (especially feminine orientation) on trust perceptions was significant. The consequences of enhanced consumer trust included increased conversion behavior, satisfaction and retention, and to a lesser extent, self-disclosure behavior. Finally, theoretical and managerial implications as well as future research directions were discussed

Adaptive Emotional Personality Model based on Fuzzy Logic Interpretation of Five Factor Theory

In recent years, emotional personality has found an important application in the field of human machine interaction. Interesting examples of this domain are computer games, interface agents, human-robot interaction, etc. However, few systems in this area include a model of personality, although it plays an important role in differentiating and determining the way they experience emotions and the way they behave. Personality simulation has always been a complex issue due to the complexity of the human personality itself, and the difficulty to model human psychology on electronic basis. Current efforts for emotion simulation are rather based on predefined set or inputs and its responses or on classical models which are simple approximate and have proven flaws. In this paper an emotional simulation system was presented. It utilizes the latest psychological theories to design a complex dynamic system that reacts to any environment, without being pre-programmed on sets of input. The design was relying on fuzzy logic to simulate human emotional reaction, thus increasing the accuracy by further emulating human brain and removing the pre-defined set of input and its matched output