The SEMAINE API : a component integration framework for a naturally interacting and emotionally competent embodied conversational agent

The present thesis addresses the topic area of Embodied Conversational Agents (ECAs) with capabilities for natural interaction with a human user and emotional competence with respect to the perception and generation of emotional expressivity. The focus is on the technological underpinnings that facilitate the implementation of a real-time system with these capabilities, built from re-usable components. The thesis comprises three main contributions. First, it describes a new component integration framework, the SEMAINE API, which makes it easy to build emotion-oriented systems from components which interact with one another using standard and pre-standard XML representations. Second, it presents a prepare-and-trigger system architecture which substantially speeds up the time to animation for system utterances that can be pre-planned. Third, it reports on the W3C Emotion Markup Language, an upcoming web standard for representing emotions in technological systems. We assess critical aspects of system performance, showing that the framework provides a good basis for implementing real-time interactive ECA systems, and illustrate by means of three examples that the SEMAINE API makes it is easy to build new emotion-oriented systems from new and existing components.Die vorliegende Dissertation behandelt das Thema der virtuellen Agenten mit Fähigkeiten zur natürlichen Benutzer-Interaktion sowie emotionaler Kompetenz bzgl. der Wahrnehmung und Generierung emotionalen Ausdrucks. Der Schwerpunkt der Arbeit liegt auf den technologischen Grundlagen für die Implementierung eines echtzeitfähigen Systems mit diesen Fähigkeiten, das aus wiederverwendbaren Komponenten erstellt werden kann. Die Arbeit umfasst drei Kernaspekte. Zum Einen beschreibt sie ein neues Framework zur Komponenten-Integration, die SEMAINE API: Diese erleichtert die Erstellung von Emotions-orientierten Systemen aus Komponenten, die untereinander mittels Standard- oder Prä-Standard-Repräsentationen kommunizieren. Zweitens wird eine Systemarchitektur vorgestellt, welche Vorbereitung und Auslösung von Systemverhalten entkoppelt und so zu einer substanziellen Beschleunigung der Generierungszeit führt, wenn Systemäußerungen im Voraus geplant werden können. Drittens beschreibt die Arbeit die W3C Emotion Markup Language, einen werdenden Web-Standard zur Repräsentation von Emotionen in technologischen Systemen. Es werden kritische Aspekte der Systemperformanz untersucht, wodurch gezeigt wird, dass das Framework eine gute Basis für die Implementierung echtzeitfähiger interaktiver Agentensysteme darstellt. Anhand von drei Beispielen wird illustriert, dass mit der SEMAINE API leicht neue Emotions-orientierte Systeme aus neuen und existierenden Komponenten erstellt werden können

Detecção e tratamento de cliques naturais em bancos de fala visando síntese concatenativa de alta qualidade

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro Tecnológico. Programa de Pós-Graduação em Engenharia Elétrica

Software Usability

This volume delivers a collection of high-quality contributions to help broaden developers’ and non-developers’ minds alike when it comes to considering software usability. It presents novel research and experiences and disseminates new ideas accessible to people who might not be software makers but who are undoubtedly software users

Multimodal access to social media services

Tese de mestrado integrado. Engenharia Informática e Computação. Faculdade de Engenharia. Universidade do Porto, Microsoft Language Development Center. 201

User interfaces and discrete event simulation models.

A user interface is critical to the success of any computer-based system. Numerous studies have shown that interface design has a significant influence on factors such as learning time, performance speed, error rates, and user satisfaction. Computer-based simulation modelling is one of the domains that is particularly demanding in terms of user interfaces. It is also an area that often pioneers new technologies that are not necessarily previously researched in terms of human-computer interaction. The dissertation describes research into user interfaces for discrete event simulation. Issues that influence the 'usability' of such systems are examined. Several representative systems were investigated in order to generate some general assumptions with respect to those characteristics of user interfaces employed in simulation systems. A case study was carried out to gain practical experience and to identify possible problems that can be encountered in user interface development. There is a need for simulation systems that can support the developments of simulation models in many domains, which are not supported by contemporary simulation software. Many user interface deficiencies are discovered and reported. On the basis of findings in this research, proposals are made on how user interfaces for simulation systems can be enhanced to match better the needs specific to the domain of simulation modelling, and on how better to support users in simulation model developments. Such improvements in user interfaces that better support users in simulation model developments could achieve a reduction in the amount of time needed to learn simulation systems, support retention of learned concepts over time, reduce the number of errors during interaction, reduce the amount of time and effort needed for model development, and provide greater user satisfaction

Designing Embodied Interactive Software Agents for E-Learning: Principles, Components, and Roles

Embodied interactive software agents are complex autonomous, adaptive, and social software systems with a digital embodiment that enables them to act on and react to other entities (users, objects, and other agents) in their environment through bodily actions, which include the use of verbal and non-verbal communicative behaviors in face-to-face interactions with the user. These agents have been developed for various roles in different application domains, in which they perform tasks that have been assigned to them by their developers or delegated to them by their users or by other agents. In computer-assisted learning, embodied interactive pedagogical software agents have the general task to promote human learning by working with students (and other agents) in computer-based learning environments, among them e-learning platforms based on Internet technologies, such as the Virtual Linguistics Campus (www.linguistics-online.com). In these environments, pedagogical agents provide contextualized, qualified, personalized, and timely assistance, cooperation, instruction, motivation, and services for both individual learners and groups of learners. This thesis develops a comprehensive, multidisciplinary, and user-oriented view of the design of embodied interactive pedagogical software agents, which integrates theoretical and practical insights from various academic and other fields. The research intends to contribute to the scientific understanding of issues, methods, theories, and technologies that are involved in the design, implementation, and evaluation of embodied interactive software agents for different roles in e-learning and other areas. For developers, the thesis provides sixteen basic principles (Added Value, Perceptible Qualities, Balanced Design, Coherence, Consistency, Completeness, Comprehensibility, Individuality, Variability, Communicative Ability, Modularity, Teamwork, Participatory Design, Role Awareness, Cultural Awareness, and Relationship Building) plus a large number of specific guidelines for the design of embodied interactive software agents and their components. Furthermore, it offers critical reviews of theories, concepts, approaches, and technologies from different areas and disciplines that are relevant to agent design. Finally, it discusses three pedagogical agent roles (virtual native speaker, coach, and peer) in the scenario of the linguistic fieldwork classes on the Virtual Linguistics Campus and presents detailed considerations for the design of an agent for one of these roles (the virtual native speaker)

Haptic augmentation of the cursor: transforming virtual actions into physical actions

This thesis demonstrates, through the exploration of two very different examples, the general claim that haptic feedback relating to a user's representation in a computer system (typically a cursor) can lead to increases in objective performance and subjective experience. Design guidelines covering each of these two topics are also presented, to ensure that the research described here can be readily adopted by other researchers, designers and system developers. The first topic to be investigated was desktop user interfaces. This thesis describes the design of a variety of different forms of haptic feedback for use with number of different Graphical User Interface (GUI) widgets, or widget groups. Two empirical evaluations of these designs are also described in some depth. The results of these studies indicate that although haptic feedback can provide improvements in objective performance, it can also reduce performance and increase subjective workload if inappropriately applied. From these results, and from the previous literature, detailed guidelines were drawn up covering the addition of haptic feedback to GUIs. The goal of these guidelines is to support the creation of performance-enhancing haptic feedback. The second topic examined was communication in interactive collaborative systems. The design of a suite of haptic communication is presented in detail, before two studies investigating different aspects of its use. The first study focuses on the subjective influence of the haptic communication as a whole, while the second is a more thorough look at one particular form of the feedback and includes objective measurements. The combined results of these studies indicate that haptic feedback has a valuable potential for increasing the quality of a user's subjective experience. Observations from these studies also reveal insights into the role of haptic feedback in communication. A set of guidelines summing up this research and the previous literature relevant to this topic are then presented. As research on this domain is in its infancy, the goal of these guidelines is to concisely present the main issues and potential benefits that respectively restrict and drive this work