4,824 research outputs found
Group-level Emotion Recognition using Transfer Learning from Face Identification
In this paper, we describe our algorithmic approach, which was used for
submissions in the fifth Emotion Recognition in the Wild (EmotiW 2017)
group-level emotion recognition sub-challenge. We extracted feature vectors of
detected faces using the Convolutional Neural Network trained for face
identification task, rather than traditional pre-training on emotion
recognition problems. In the final pipeline an ensemble of Random Forest
classifiers was learned to predict emotion score using available training set.
In case when the faces have not been detected, one member of our ensemble
extracts features from the whole image. During our experimental study, the
proposed approach showed the lowest error rate when compared to other explored
techniques. In particular, we achieved 75.4% accuracy on the validation data,
which is 20% higher than the handcrafted feature-based baseline. The source
code using Keras framework is publicly available.Comment: 5 pages, 3 figures, accepted for publication at ICMI17 (EmotiW Grand
Challenge
High-Level Concepts for Affective Understanding of Images
This paper aims to bridge the affective gap between image content and the
emotional response of the viewer it elicits by using High-Level Concepts
(HLCs). In contrast to previous work that relied solely on low-level features
or used convolutional neural network (CNN) as a black-box, we use HLCs
generated by pretrained CNNs in an explicit way to investigate the
relations/associations between these HLCs and a (small) set of Ekman's
emotional classes. As a proof-of-concept, we first propose a linear admixture
model for modeling these relations, and the resulting computational framework
allows us to determine the associations between each emotion class and certain
HLCs (objects and places). This linear model is further extended to a nonlinear
model using support vector regression (SVR) that aims to predict the viewer's
emotional response using both low-level image features and HLCs extracted from
images. These class-specific regressors are then assembled into a regressor
ensemble that provide a flexible and effective predictor for predicting
viewer's emotional responses from images. Experimental results have
demonstrated that our results are comparable to existing methods, with a clear
view of the association between HLCs and emotional classes that is ostensibly
missing in most existing work
Hierarchical Multi-resolution Mesh Networks for Brain Decoding
We propose a new framework, called Hierarchical Multi-resolution Mesh
Networks (HMMNs), which establishes a set of brain networks at multiple time
resolutions of fMRI signal to represent the underlying cognitive process. The
suggested framework, first, decomposes the fMRI signal into various frequency
subbands using wavelet transforms. Then, a brain network, called mesh network,
is formed at each subband by ensembling a set of local meshes. The locality
around each anatomic region is defined with respect to a neighborhood system
based on functional connectivity. The arc weights of a mesh are estimated by
ridge regression formed among the average region time series. In the final
step, the adjacency matrices of mesh networks obtained at different subbands
are ensembled for brain decoding under a hierarchical learning architecture,
called, fuzzy stacked generalization (FSG). Our results on Human Connectome
Project task-fMRI dataset reflect that the suggested HMMN model can
successfully discriminate tasks by extracting complementary information
obtained from mesh arc weights of multiple subbands. We study the topological
properties of the mesh networks at different resolutions using the network
measures, namely, node degree, node strength, betweenness centrality and global
efficiency; and investigate the connectivity of anatomic regions, during a
cognitive task. We observe significant variations among the network topologies
obtained for different subbands. We, also, analyze the diversity properties of
classifier ensemble, trained by the mesh networks in multiple subbands and
observe that the classifiers in the ensemble collaborate with each other to
fuse the complementary information freed at each subband. We conclude that the
fMRI data, recorded during a cognitive task, embed diverse information across
the anatomic regions at each resolution.Comment: 18 page
Triaging Content Severity in Online Mental Health Forums
Mental health forums are online communities where people express their issues
and seek help from moderators and other users. In such forums, there are often
posts with severe content indicating that the user is in acute distress and
there is a risk of attempted self-harm. Moderators need to respond to these
severe posts in a timely manner to prevent potential self-harm. However, the
large volume of daily posted content makes it difficult for the moderators to
locate and respond to these critical posts. We present a framework for triaging
user content into four severity categories which are defined based on
indications of self-harm ideation. Our models are based on a feature-rich
classification framework which includes lexical, psycholinguistic, contextual
and topic modeling features. Our approaches improve the state of the art in
triaging the content severity in mental health forums by large margins (up to
17% improvement over the F-1 scores). Using the proposed model, we analyze the
mental state of users and we show that overall, long-term users of the forum
demonstrate a decreased severity of risk over time. Our analysis on the
interaction of the moderators with the users further indicates that without an
automatic way to identify critical content, it is indeed challenging for the
moderators to provide timely response to the users in need.Comment: Accepted for publication in Journal of the Association for
Information Science and Technology (2017
CentralNet: a Multilayer Approach for Multimodal Fusion
This paper proposes a novel multimodal fusion approach, aiming to produce
best possible decisions by integrating information coming from multiple media.
While most of the past multimodal approaches either work by projecting the
features of different modalities into the same space, or by coordinating the
representations of each modality through the use of constraints, our approach
borrows from both visions. More specifically, assuming each modality can be
processed by a separated deep convolutional network, allowing to take decisions
independently from each modality, we introduce a central network linking the
modality specific networks. This central network not only provides a common
feature embedding but also regularizes the modality specific networks through
the use of multi-task learning. The proposed approach is validated on 4
different computer vision tasks on which it consistently improves the accuracy
of existing multimodal fusion approaches
- …