18,471 research outputs found
Multi-modal Approach for Affective Computing
Throughout the past decade, many studies have classified human emotions using
only a single sensing modality such as face video, electroencephalogram (EEG),
electrocardiogram (ECG), galvanic skin response (GSR), etc. The results of
these studies are constrained by the limitations of these modalities such as
the absence of physiological biomarkers in the face-video analysis, poor
spatial resolution in EEG, poor temporal resolution of the GSR etc. Scant
research has been conducted to compare the merits of these modalities and
understand how to best use them individually and jointly. Using multi-modal
AMIGOS dataset, this study compares the performance of human emotion
classification using multiple computational approaches applied to face videos
and various bio-sensing modalities. Using a novel method for compensating
physiological baseline we show an increase in the classification accuracy of
various approaches that we use. Finally, we present a multi-modal
emotion-classification approach in the domain of affective computing research.Comment: Published in IEEE 40th International Engineering in Medicine and
Biology Conference (EMBC) 201
Machine Understanding of Human Behavior
A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground. If this prediction is to come true, then next generation computing, which we will call human computing, should be about anticipatory user interfaces that should be human-centered, built for humans based on human models. They should transcend the traditional keyboard and mouse to include natural, human-like interactive functions including understanding and emulating certain human behaviors such as affective and social signaling. This article discusses a number of components of human behavior, how they might be integrated into computers, and how far we are from realizing the front end of human computing, that is, how far are we from enabling computers to understand human behavior
SALSA: A Novel Dataset for Multimodal Group Behavior Analysis
Studying free-standing conversational groups (FCGs) in unstructured social
settings (e.g., cocktail party ) is gratifying due to the wealth of information
available at the group (mining social networks) and individual (recognizing
native behavioral and personality traits) levels. However, analyzing social
scenes involving FCGs is also highly challenging due to the difficulty in
extracting behavioral cues such as target locations, their speaking activity
and head/body pose due to crowdedness and presence of extreme occlusions. To
this end, we propose SALSA, a novel dataset facilitating multimodal and
Synergetic sociAL Scene Analysis, and make two main contributions to research
on automated social interaction analysis: (1) SALSA records social interactions
among 18 participants in a natural, indoor environment for over 60 minutes,
under the poster presentation and cocktail party contexts presenting
difficulties in the form of low-resolution images, lighting variations,
numerous occlusions, reverberations and interfering sound sources; (2) To
alleviate these problems we facilitate multimodal analysis by recording the
social interplay using four static surveillance cameras and sociometric badges
worn by each participant, comprising the microphone, accelerometer, bluetooth
and infrared sensors. In addition to raw data, we also provide annotations
concerning individuals' personality as well as their position, head, body
orientation and F-formation information over the entire event duration. Through
extensive experiments with state-of-the-art approaches, we show (a) the
limitations of current methods and (b) how the recorded multiple cues
synergetically aid automatic analysis of social interactions. SALSA is
available at http://tev.fbk.eu/salsa.Comment: 14 pages, 11 figure
- …