1 research outputs found
Visual sentiment prediction based on automatic discovery of affective regions
Automatic assessment of sentiment from visual
content has gained considerable attention with the increasing
tendency of expressing opinions via images and videos online.
This paper investigates the problem of visual sentiment analysis,
which involves a high-level abstraction in the recognition process.
While most of the current methods focus on improving holistic
representations, we aim to utilize the local information, which is
inspired by the observation that both the whole image and local
regions convey significant sentiment information. We propose
a framework to leverage affective regions, where we first use
an off-the-shelf objectness tool to generate the candidates, and
employ a candidate selection method to remove redundant and
noisy proposals. Then a convolutional neural network (CNN) is
connected with each candidate to compute the sentiment scores,
and the affective regions are automatically discovered, taking the
objectness score as well as the sentiment score into consideration.
Finally, the CNN outputs from local regions are aggregated with
the whole images to produce the final predictions. Our framework
only requires image-level labels, thereby significantly reducing
the annotation burden otherwise required for training. This is
especially important for sentiment analysis as sentiment can be
abstract, and labeling affective regions is too subjective and
labor-consuming. Extensive experiments show that the proposed
algorithm outperforms the state-of-the-art approaches on eight
popular benchmark datasets