Towards the Use of Dialog Systems to Facilitate Inclusive Education

Continuous advances in the development of information technologies have currently led to the possibility of accessing learning contents from anywhere, at anytime, and almost instantaneously. However, accessibility is not always the main objective in the design of educative applications, specifically to facilitate their adoption by disabled people. Different technologies have recently emerged to foster the accessibility of computers and new mobile devices, favoring a more natural communication between the student and the developed educative systems. This chapter describes innovative uses of multimodal dialog systems in education, with special emphasis in the advantages that they provide for creating inclusive applications and learning activities

The Prosody of Uncertainty for Spoken Dialogue Intelligent Tutoring Systems

The speech medium is more than an audio conveyance of word strings. It contains meta information about the content of the speech. The prosody of speech, pauses and intonation, adds an extra dimension of diagnostic information about the quality of a speaker\u27s answers, suggesting an important avenue of research for spoken dialogue tutoring systems. Tutoring systems that are sensitive to such cues may employ different tutoring strategies based on detected student uncertainty, and they may be able to perform more precise assessment of the area of student difficulty. However, properly identifying the cues can be challenging, typically requiring thousands of hand labeled utterances for training in machine learning. This study proposes and explores means of exploiting alternate automatically generated information, utterance correctness and the amount of practice a student has had, as indicators of student uncertainty. It finds correlations with various prosodic features and these automatic indicators and compares the result with a small set of annotated utterances, and finally demonstrates a Bayesian classifier based on correctness scores as class labels

Towards Integration of Cognitive Models in Dialogue Management: Designing the Virtual Negotiation Coach Application

This paper presents an approach to flexible and adaptive dialogue management driven by cognitive modelling of human dialogue behaviour. Artificial intelligent agents, based on the ACT-R cognitive architecture, together with human actors are participating in a (meta)cognitive skills training within a negotiation scenario. The agent  employs instance-based learning to decide about its own actions and to reflect on the behaviour of the opponent. We show that task-related actions can be handled by a cognitive agent who is a plausible dialogue partner.  Separating task-related and dialogue control actions enables the application of sophisticated models along with a flexible architecture  in which  various alternative modelling methods can be combined. We evaluated the proposed approach with users assessing  the relative contribution of various factors to the overall usability of a dialogue system. Subjective perception of effectiveness, efficiency and satisfaction were correlated with various objective performance metrics, e.g. number of (in)appropriate system responses, recovery strategies, and interaction pace. It was observed that the dialogue system usability is determined most by the quality of agreements reached in terms of estimated Pareto optimality, by the user's negotiation strategies selected, and by the quality of system recognition, interpretation and responses. We compared human-human and human-agent performance with respect to the number and quality of agreements reached, estimated cooperativeness level, and frequency of accepted negative outcomes. Evaluation experiments showed promising, consistently positive results throughout the range of the relevant scales

Robust Modeling of Epistemic Mental States

This work identifies and advances some research challenges in the analysis of facial features and their temporal dynamics with epistemic mental states in dyadic conversations. Epistemic states are: Agreement, Concentration, Thoughtful, Certain, and Interest. In this paper, we perform a number of statistical analyses and simulations to identify the relationship between facial features and epistemic states. Non-linear relations are found to be more prevalent, while temporal features derived from original facial features have demonstrated a strong correlation with intensity changes. Then, we propose a novel prediction framework that takes facial features and their nonlinear relation scores as input and predict different epistemic states in videos. The prediction of epistemic states is boosted when the classification of emotion changing regions such as rising, falling, or steady-state are incorporated with the temporal features. The proposed predictive models can predict the epistemic states with significantly improved accuracy: correlation coefficient (CoERR) for Agreement is 0.827, for Concentration 0.901, for Thoughtful 0.794, for Certain 0.854, and for Interest 0.913.Comment: Accepted for Publication in Multimedia Tools and Application, Special Issue: Socio-Affective Technologie

Hacia una educación inclusiva y personalizada mediante el uso de los sistemas de diálogo multimodal

Los continuos avances en el desarrollo de tecnologías de la información han dado lugar actualmente a la posibilidad de acceder a los contenidos educativos desde cualquier lugar, en cualquier momento y de forma casi instantánea. Sin embargo, la accesibilidad no es siempre considerada como criterio principal en el diseño de aplicaciones educativas, especialmente para facilitar su utilización por parte de personas con discapacidad. Diferentes tecnologías han surgido recientemente para fomentar la accesibilidad a las nuevas tecnologías y dispositivos móviles, favoreciendo una comunicación más natural con los sistemas educativos. En este artículo se describe el uso innovador de los sistemas de diálogo multimodales en el campo de la educación, con un especial énfasis en la descripción de las ventajas que ofrecen para la creación de aplicaciones educativas inclusivas y adaptadas a la evolución de los estudiantes.Continuous advances in the development of information technologies have currently led to the possibility of accessing learning contents from anywhere, at anytime and almost instantaneously. However, accessibility is not always the main objective in the design of educative applications, specifically to facilitate their adoption by disabled people. Different technologies have recently emerged to foster the accessibility of computers and new mobile devices favouring a more natural communication between the student and the developed educative systems. This paper describes innovative uses of multimodal dialog systems in education, with special emphasis in the advantages that they provide for creating inclusive applications and adapted to the students specific evolution.Trabajo parcialmente financiado por los proyectos MINECO TEC2012-37832-C02-01, CICYT TEC2011-28626-C02-02, CAM CONTEXTS (S2009/TIC-1485) y TRA2010-20225-C03-01.Publicad

Alexa, Can You Help Me Solve That Problem? – Understanding the Value of Smart Personal Assistants as Tutors for Complex Problem Tasks

In recent decades, the number of students per lecturer at universities has constantly risen. In these learning scenarios, individual lecturer support for helping students actively acquiring new knowledge is hardly possible. However, active student behavior is necessary for successful learning. Smart Personal Assistants such as Amazon’s Alexa or Google’s Home promise to fill this gap by being students’ individual tutors. In order to understand what students expect from Smart Personal Assistants as tutors and how they interact with them, we will carry out an experiment. In this research in progress paper, we present our experiment design, where we observe the individual interaction between students and a Smart Personal Assistant tutor and between students and a human tutor applying the same methods in both cases. Drawing on the concepts of parasocial interaction and trust, we derive hypotheses, present the Smart Personal Assistant development and explain the experiment process in detail

I Probe, Therefore I Am: Designing a Virtual Journalist with Human Emotions

By utilizing different communication channels, such as verbal language, gestures or facial expressions, virtually embodied interactive humans hold a unique potential to bridge the gap between human-computer interaction and actual interhuman communication. The use of virtual humans is consequently becoming increasingly popular in a wide range of areas where such a natural communication might be beneficial, including entertainment, education, mental health research and beyond. Behind this development lies a series of technological advances in a multitude of disciplines, most notably natural language processing, computer vision, and speech synthesis. In this paper we discuss a Virtual Human Journalist, a project employing a number of novel solutions from these disciplines with the goal to demonstrate their viability by producing a humanoid conversational agent capable of naturally eliciting and reacting to information from a human user. A set of qualitative and quantitative evaluation sessions demonstrated the technical feasibility of the system whilst uncovering a number of deficits in its capacity to engage users in a way that would be perceived as natural and emotionally engaging. We argue that naturalness should not always be seen as a desirable goal and suggest that deliberately suppressing the naturalness of virtual human interactions, such as by altering its personality cues, might in some cases yield more desirable results.Comment: eNTERFACE16 proceeding

Usable Interface Design for Everyone

En el diseño de "interfaces para todo el mundo" para los sistemas interactivos, es importante tener en cuenta factores como el costo, el mercado de destino, el estado del medio ambiente,etc. Los interfaces de usuario son fundamentales para el proceso de desarrollo de cualquier aplicación, y su diseño debe estar contemplado desde el principio. De las distintas partes de un sistema (hardware y software), es la interfaz el sistema que permite al usuario el acceso a los recursos informáticos. Los siete principios del "Diseño Universal" o "Diseño para Todos" se centran en un diseño utilizable universal, pero al mismo tiempo reconocer la influencia de factores internos y externos. Los cambios estructurales en los servicios sociales y de salud podrían proporcionar un aumento en el bienestar de los ciudadanos de un país a través del uso de la programación de auto-cuidado y la gestión proactiva / prevención de la enfermedad. Plataformas automatizadas en el hogar pueden actuar como un instrumento que permitan a los usuarios evitar, compensar, mitigar o neutralizar las deficiencias y las dependencias causada por el envejecimiento.When designing “interfaces for everyone” for interactive systems, it is important to consider factors such as cost, the intended market, the state of the environment, etc. User interfaces are fundamental for the developmental process in any application, and its design must be contemplated from the start. Of the distinct parts of a system (hardware and software), it is the interface that permits the user access to computer resources. The seven principles of “Universal Design” or “Design for Everyone” focus on a universal usable design, but at the same time acknowledge the influences of internal and external factors. Structural changes in social and health services could provide an increase in the well-being of a country’s citizens through the use of self-care programming and proactive management/prevention of disease. Automated home platforms can act as an accessibility instrument which permits users to avoid, compensate, mitigate, or neutralize the deficiencies and dependencies caused by living alon