Throughout the past decade, many studies have classified human emotions using
only a single sensing modality such as face video, electroencephalogram (EEG),
electrocardiogram (ECG), galvanic skin response (GSR), etc. The results of
these studies are constrained by the limitations of these modalities such as
the absence of physiological biomarkers in the face-video analysis, poor
spatial resolution in EEG, poor temporal resolution of the GSR etc. Scant
research has been conducted to compare the merits of these modalities and
understand how to best use them individually and jointly. Using multi-modal
AMIGOS dataset, this study compares the performance of human emotion
classification using multiple computational approaches applied to face videos
and various bio-sensing modalities. Using a novel method for compensating
physiological baseline we show an increase in the classification accuracy of
various approaches that we use. Finally, we present a multi-modal
emotion-classification approach in the domain of affective computing research.Comment: Published in IEEE 40th International Engineering in Medicine and
Biology Conference (EMBC) 201