Modeling the user state for context-aware spoken interaction in ambient assisted living

Ambient Assisted Living (AAL) systems must provide adapted services easily accessible by a wide variety of users. This can only be possible if the communication between the user and the system is carried out through an interface that is simple, rapid, effective, and robust. Natural language interfaces such as dialog systems fulfill these requisites, as they are based on a spoken conversation that resembles human communication. In this paper, we enhance systems interacting in AAL domains by means of incorporating context-aware conversational agents that consider the external context of the interaction and predict the user's state. The user's state is built on the basis of their emotional state and intention, and it is recognized by means of a module conceived as an intermediate phase between natural language understanding and dialog management in the architecture of the conversational agent. This prediction, carried out for each user turn in the dialog, makes it possible to adapt the system dynamically to the user's needs. We have evaluated our proposal developing a context-aware system adapted to patients suffering from chronic pulmonary diseases, and provide a detailed discussion of the positive influence of our proposal in the success of the interaction, the information and services provided, as well as the perceived quality.This work was supported in part by Projects MINECO TEC2012-37832-C02-01, CICYT TEC2011-28626-C02- 02, CAM CONTEXTS (S2009/TIC-1485

Emotions in context: examining pervasive affective sensing systems, applications, and analyses

Pervasive sensing has opened up new opportunities for measuring our feelings and understanding our behavior by monitoring our affective states while mobile. This review paper surveys pervasive affect sensing by examining and considering three major elements of affective pervasive systems, namely; “sensing”, “analysis”, and “application”. Sensing investigates the different sensing modalities that are used in existing real-time affective applications, Analysis explores different approaches to emotion recognition and visualization based on different types of collected data, and Application investigates different leading areas of affective applications. For each of the three aspects, the paper includes an extensive survey of the literature and finally outlines some of challenges and future research opportunities of affective sensing in the context of pervasive computing

On the Development of Adaptive and User-Centred Interactive Multimodal Interfaces

Multimodal systems have attained increased attention in recent years, which has made possible important improvements in the technologies for recognition, processing, and generation of multimodal information. However, there are still many issues related to multimodality which are not clear, for example, the principles that make it possible to resemble human-human multimodal communication. This chapter focuses on some of the most important challenges that researchers have recently envisioned for future multimodal interfaces. It also describes current efforts to develop intelligent, adaptive, proactive, portable and affective multimodal interfaces

Combining heterogeneous inputs for the development of adaptive and multimodal interaction systems

In this paper we present a novel framework for the integration of visual sensor networks and speech-based interfaces. Our proposal follows the standard reference architecture in fusion systems (JDL), and combines different techniques related to Artificial Intelligence, Natural Language Processing and User Modeling to provide an enhanced interaction with their users. Firstly, the framework integrates a Cooperative Surveillance Multi-Agent System (CS-MAS), which includes several types of autonomous agents working in a coalition to track and make inferences on the positions of the targets. Secondly, enhanced conversational agents facilitate human-computer interaction by means of speech interaction. Thirdly, a statistical methodology allows modeling the user conversational behavior, which is learned from an initial corpus and improved with the knowledge acquired from the successive interactions. A technique is proposed to facilitate the multimodal fusion of these information sources and consider the result for the decision of the next system action.This work was supported in part by Projects MEyC TEC2012-37832-C02-01, CICYT TEC2011-28626-C02-02, CAM CONTEXTS S2009/TIC-1485Publicad

Proceedings of the 1st joint workshop on Smart Connected and Wearable Things 2016

These are the Proceedings of the 1st joint workshop on Smart Connected and Wearable Things (SCWT'2016, Co-located with IUI 2016). The SCWT workshop integrates the SmartObjects and IoWT workshops. It focusses on the advanced interactions with smart objects in the context of the Internet-of-Things (IoT), and on the increasing popularity of wearables as advanced means to facilitate such interactions

Vocal Interactivity in-and-between Humans, Animals, and Robots

Almost all animals exploit vocal signals for a range of ecologically motivated purposes: detecting predators/prey and marking territory, expressing emotions, establishing social relations, and sharing information. Whether it is a bird raising an alarm, a whale calling to potential partners, a dog responding to human commands, a parent reading a story with a child, or a business-person accessing stock prices using Siri, vocalization provides a valuable communication channel through which behavior may be coordinated and controlled, and information may be distributed and acquired. Indeed, the ubiquity of vocal interaction has led to research across an extremely diverse array of fields, from assessing animal welfare, to understanding the precursors of human language, to developing voice-based human–machine interaction. Opportunities for cross-fertilization between these fields abound; for example, using artificial cognitive agents to investigate contemporary theories of language grounding, using machine learning to analyze different habitats or adding vocal expressivity to the next generation of language-enabled autonomous social agents. However, much of the research is conducted within well-defined disciplinary boundaries, and many fundamental issues remain. This paper attempts to redress the balance by presenting a comparative review of vocal interaction within-and-between humans, animals, and artificial agents (such as robots), and it identifies a rich set of open research questions that may benefit from an interdisciplinary analysis

An aesthetics of touch: investigating the language of design relating to form

How well can designers communicate qualities of touch? This paper presents evidence that they have some capability to do so, much of which appears to have been learned, but at present make limited use of such language. Interviews with graduate designer-makers suggest that they are aware of and value the importance of touch and materiality in their work, but lack a vocabulary to fully relate to their detailed explanations of other aspects such as their intent or selection of materials. We believe that more attention should be paid to the verbal dialogue that happens in the design process, particularly as other researchers show that even making-based learning also has a strong verbal element to it. However, verbal language alone does not appear to be adequate for a comprehensive language of touch. Graduate designers-makers’ descriptive practices combined non-verbal manipulation within verbal accounts. We thus argue that haptic vocabularies do not simply describe material qualities, but rather are situated competences that physically demonstrate the presence of haptic qualities. Such competencies are more important than groups of verbal vocabularies in isolation. Design support for developing and extending haptic competences must take this wide range of considerations into account to comprehensively improve designers’ capabilities

Experiencing the Ancient Agora of Athens through Emotionally-led Interactive Stories with Tangibles

Οι χώροι πολιτιστικής κληρονομιάς είναι δημοφιλή τουριστικά αξιοθέατα, αλλά αποτελεί σύνηθες φαινόμενο για τους επισκέπτες να φεύγουν από τους χώρους αυτούς με ένα αίσθημα απεμπλοκής λόγω του μεγάλου όγκου των πληροφοριών που παρέχονται και μη συνδεδεμένοι συναισθηματικά με την κληρονομία. Οι επιμελητές των μουσείων καταβάλλουν σημαντικές προσπάθειες για να ικανοποιήσουν τις ανάγκες των επισκεπτών τους εξερευνώντας τους τρόπους εξυπηρέτησης της εκπαιδευτικής αποστολής των θεσμικών οργάνων τους. Τα τελευταία χρόνια η ανάπτυξη έργων ψηφιακής τεχνολογίας στοχεύει στην άμβλυνση τέτοιων προβλημάτων, καθιστώντας την επίσκεψη του μουσείου πιο ελκυστική. Παρόλα αυτά, οι ψηφιακές εφαρμογές σε πολλές περιπτώσεις υποστηρίζουν προηγούμενες διδακτικές προσεγγίσεις και υπολείπονται από τον επαναπροσδιορισμό του τρόπου με τον οποίο οι επισκέπτες γνωρίζουν την πολιτιστική κληρονομιά. Αυτή η πτυχιακή εξετάζει πώς η συσχέτιση της ενσωματωμένης και απτής αλληλεπίδρασης με την αφήγηση που προκαλείται από συναισθήματα μπορεί να επηρεάσει τον τρόπο με τον οποίο οι άνθρωποι βιώνουν τους χώρους πολιτιστικής κληρονομιάς και τα μουσεία. Οι ψηφιακές εμπειρίες με βάση την αφήγηση σχεδιάστηκαν και υλοποιήθηκαν τόσο για τον υπαίθριο αρχαιολογικό χώρο της Αρχαίας Αγοράς της Αθήνας όσο και για το εσωτερικό μουσείο. Η υπαίθρια εμπειρία στόχευε στη διερεύνηση της χρήσης μιας ενιαίας ιστορίας που εκτείνεται σε ένα μεγάλο ανοικτό χώρο και αξιολογήθηκε επί τόπου από μια μικρή αλλά διεπιστημονική ομάδα χρηστών. Η εσωτερική εμπειρία χρησιμοποιεί ένα απτό αντικείμενο και μια εφαρμογή που λαμβάνει γνώση της θέσης για να καθοδηγήσει την αλληλεπίδραση μέσω σύντομων ιστοριών βασισμένων στο συναίσθημα και αξιολογήθηκε από 12 χρήστες. Τα αποτελέσματα της έρευνας μας δίνουν πληροφορίες για τα στοιχεία των εμπειριών που τις καθιστούν αποτελεσματικές ή όχι στο πλούσιο πλαίσιο ενός πολιτιστικού περιβάλλοντος.Cultural heritage sites are popular tourist attractions, yet it is common for visitors to leave feeling disengaged by the large amount of information provided, and emotionally detached. Curators are making considerable efforts to cater to their visitors’ needs while exploring ways to serve their institutions’ educational mission. Nowadays, the development of digital technology projects aims to alleviate such problems, making the museum visit more engaging. However, even so, digital applications in many cases sustain earlier didactic approaches and fall short from significantly redefining the way that visitors experience cultural heritage. This thesis examines how the association of embodied and tangible interaction with emotion-driven storytelling can affect the way people experience cultural heritage sites and museums. Digital storytelling experiences were designed and implemented for both the outdoor archeological site of the Ancient Agora of Athens and its indoor museum. The outdoor experience aimed at exploring the use of a single story spanning across a large open space and was evaluated on site by a small but multidisciplinary user group. The indoor experience makes use of a custom tangible object and of a location-aware application to guide interaction through short emotionally-led stories, and was evaluated by 12 users. Our findings provide insights on those elements of the experiences that make them effective or not in the rich context of a cultural setting