7 research outputs found
Development Of a Multisensorial System For Emotions Recognition
Automated reading and analysis of human emotion has the potential to be a powerful tool to develop a wide variety of applications, such as human-computer interaction systems, but, at the same time, this is a very difficult issue because the human communication is very complex. Humans employ multiple sensory systems in emotion recognition. At the same way, an emotionally intelligent machine requires multiples sensors to be able to create an affective interaction with users. Thus, this Master thesis proposes the development of a multisensorial system for automatic emotion recognition.
The multisensorial system is composed of three sensors, which allowed exploring different emotional aspects, as the eye tracking, using the IR-PCR technique, helped conducting studies about visual social attention; the Kinect, in conjunction with the FACS-AU system technique, allowed developing a tool for facial expression recognition; and the thermal camera, using the
FT-RoI technique, was employed for detecting facial thermal variation. When performing the multisensorial integration of the system, it was possible to obtain a more complete and varied analysis of the emotional aspects, allowing evaluate focal attention, valence comprehension, valence expressions, facial expression, valence recognition and arousal recognition. Experiments were performed with sixteen healthy adult volunteers and 105 healthy children
volunteers and the results were the developed system, which was able to detect eye gaze, recognize facial expression and estimate the valence and arousal for emotion recognition, This system also presents the potential to analyzed emotions of people by facial features using contactless sensors in semi-structured environments, such as clinics, laboratories, or classrooms. This system also presents the potential to become an embedded tool in robots to
endow these machines with an emotional intelligence for a more natural interaction with humans.
Keywords: emotion recognition, eye tracking, facial expression, facial thermal variation, integration multisensoria
Dynamic Estimation of Rater Reliability using Multi-Armed Bandits
One of the critical success factors for supervised machine learning is the quality of target values, or predictions, associated with training instances. Predictions can be discrete labels (such as a binary variable specifying whether a blog post is positive or negative) or continuous ratings (for instance, how boring a video is on a 10-point scale). In some areas, predictions are readily available, while in others, the eort of human workers has to be involved. For instance, in the task of emotion recognition from speech, a large corpus of speech recordings is usually available, and humans denote which emotions are present in which recordings
Internet and Biometric Web Based Business Management Decision Support
Internet and Biometric Web Based Business Management Decision Support
MICROBE
MOOC material prepared under
IO1/A5 Development of the MICROBE personalized MOOCs content and teaching materials
Prepared by:
A. Kaklauskas, A. Banaitis, I. Ubarte
Vilnius Gediminas Technical University, Lithuania
Project No: 2020-1-LT01-KA203-07810
Evaluating Information Presentation Strategies for Spoken Dialogue Systems
Institute for Communicating and Collaborative SystemsA common task for spoken dialogue systems (SDS) is to help users select a suitable
option (e.g., flight, hotel, restaurant) from the set of options available. When the number
of options is small, they can simply be presented sequentially. However, as the
number of options increases, the system must have strategies for helping users browse
the space of available options.
In this thesis, I compare two approaches to information presentation in SDS: (1)
the summarize and refine (SR) approach (Polifroni et al., 2003; Polifroni, 2008) in
which the summaries are generated by clustering the options based on attributes that
lead to the smallest number of clusters, and (2) the user-model based summarize and
refine (UMSR) approach (Demberg, 2005; Demberg and Moore, 2006) which employs
a user model to cluster options based on attributes that are relevant to the user and
uses coherence markers (e.g., connectives, discourse cues, adverbials) to highlight the
trade-offs among the presented items.
Prior work has shown that users prefer approaches to information presentation that
take the user’s preferences into account (e.g., Komatani et al., 2003;Walker et al., 2004;
Demberg and Moore, 2006). However, due to the complexity of building a working
end-to-end SDS, these studies employed an ”overhearer” evaluation methodology, in
which participants read or listened to pre-prepared dialogues, thus limiting evaluation
criteria to users’ perceptions (e.g., informativeness, overview of options, and so on).
In order to examine whether users prefer presentations based on UMSR when they
were actively interacting with a dialogue system, and to measure the effectiveness and
efficiency of the two approaches, I compared them in a Wizard-of-Oz experiment. I
found that in terms of both task success and dialogue efficiency the UMSR approach
was superior to the SR approach. In addition, I found that users also preferred presentations
based on UMSR in the interactive mode.
SDS are typically developed for situations in which the user’s hands and eyes are
busy. I hypothesized that the benefits of pointing out relationships among options (i.e.,
trade-offs) in information presentation messages outweighs the costs of processing
more complex sentences. To test this hypothesis, I performed two dual task experiments
comparing the two approaches to information presentation in terms of their
effect on cognitive load. Again, participants performed better with presentations based
on the UMSR algorithm in terms of both dialogue efficiency and task success, and I
found no detrimental effect on performance of the primary task.
Finally, I hypothesized that one of the main reasons why UMSR is more efficient
is because it uses coherence markers to highlight relations (e.g., trade-offs) between
options and attributes. To test this hypothesis, I performed an eye-tracking experiment
in which participants read presentations with and without these linguistic devices, and
answered evaluation and comparison questions to measure differences in item recall.
In addition, I used reading times to examine comprehension differences between the
two information presentation strategies. I found that the linguistic devices used in
UMSR indeed facilitated item recall, with no penalty in terms of comprehension cost.
Thus, in this thesis I showed that an approach to information presentation that employs
a user model and uses linguistic devices such as coherence markers to highlight
trade-offs among the presented items improves information browsing. User studies
demonstrated that this finding also applies to situations where users are performing
another demanding task simultaneously
Towards Detecting Cognitive Load and Emotions in Usability Studies using the RealEYES Framework
In this paper, we will discuss some extensions to the RealEYES framework that can help to automatically detect interesting sections in usability studies using additional sensor input and knowledge discovery techniques