Visual Affect Around the World: A Large-scale Multilingual Visual Sentiment Ontology

Every culture and language is unique. Our work expressly focuses on the uniqueness of culture and language in relation to human affect, specifically sentiment and emotion semantics, and how they manifest in social multimedia. We develop sets of sentiment- and emotion-polarized visual concepts by adapting semantic structures called adjective-noun pairs, originally introduced by Borth et al. (2013), but in a multilingual context. We propose a new language-dependent method for automatic discovery of these adjective-noun constructs. We show how this pipeline can be applied on a social multimedia platform for the creation of a large-scale multilingual visual sentiment concept ontology (MVSO). Unlike the flat structure in Borth et al. (2013), our unified ontology is organized hierarchically by multilingual clusters of visually detectable nouns and subclusters of emotionally biased versions of these nouns. In addition, we present an image-based prediction task to show how generalizable language-specific models are in a multilingual context. A new, publicly available dataset of >15.6K sentiment-biased visual concepts across 12 languages with language-specific detector banks, >7.36M images and their metadata is also released.Comment: 11 pages, to appear at ACM MM'1

Effects of Genre Tag Complexity on Popular Music Perception and Enjoyment

The popular online streaming platform Spotify added over 1400 genre tags in the last two years. Despite that numerous artists and composition competitions claim to seek projects that “transcend the traditional notion of genre,” the industry has only added more complex and mystifying genre labels. This dichotomy between artists and industry ignores the effects these labels have on consumers. Do more complex genre tags enhance the listening experience for the average consumer by providing additional information about what they are about to hear? The current research seeks to examine the effects of the granularity of genre tags on popular music perception by identifying whether more nuanced subgenre genre tags increase enjoyment and understanding of popular music excerpts. Participants heard four 20-second excerpts of popular music from four broad genre categories—including pop, country, rap/hip-hop, rock—as defined in Gjerdingen & Perrott, 2008 and Mace et al., 2011. Excerpts were presented simultaneously with two or three corresponding broad genre category tags or nuanced subgenre category tags in a randomized order. Participants used Likert-type scales to rate how well the genre tags matched the excerpt with which they were presented, how much they enjoyed the excerpt, and were asked to self-label each excerpt with a genre tag. Results showed that ratings were significantly higher for the broad genre categories than the subgenre categories for both enjoyment and matching, (F(1, 2109.67) = 19.07, p \u3c .001; F(1, 2109.38) = 56.47, p \u3c .001), respectively. Further, participants did not self-label any of the excerpts with genre categories that were not previously attached to the respective stimuli. These results have practical implications for how music producers market popular music since broad genre categories appear to be adequate for conveying expectations for popular music

“I can haz emoshuns?”: understanding anthropomorphosis of cats among internet users

The attribution of human-like traits to non-human animals, termed anthropomorphism, can lead to misunderstandings of animal behaviour, which can result in risks to both human and animal wellbeing and welfare. In this paper, we, during an inter-disciplinary collaboration between social computing and animal behaviour researchers, investigated whether a simple image-tagging application could improve the understanding of how people ascribe intentions and emotions to the behaviour of their domestic cats. A web-based application, Tagpuss, was developed to present casual users with photographs drawn from a database of 1631 images of domestic cats and asked them to ascribe an emotion to the cat portrayed in the image. Over five thousand people actively participated in the study in the space of four weeks, generating over 50,000 tags. Results indicate Tagpuss can be used to identify cat behaviours that lay-people find difficult to distinguish. This highlights further expert scientific exploration that focuses on educating cat owners to identify possible problems with their cat’s welfare