Diving Deep into Sentiment: Understanding Fine-tuned CNNs for Visual Sentiment Prediction

Visual media are powerful means of expressing emotions and sentiments. The constant generation of new content in social networks highlights the need of automated visual sentiment analysis tools. While Convolutional Neural Networks (CNNs) have established a new state-of-the-art in several vision problems, their application to the task of sentiment analysis is mostly unexplored and there are few studies regarding how to design CNNs for this purpose. In this work, we study the suitability of fine-tuning a CNN for visual sentiment prediction as well as explore performance boosting techniques within this deep learning setting. Finally, we provide a deep-dive analysis into a benchmark, state-of-the-art network architecture to gain insight about how to design patterns for CNNs on the task of visual sentiment prediction.Comment: Preprint of the paper accepted at the 1st Workshop on Affect and Sentiment in Multimedia (ASM), in ACM MultiMedia 2015. Brisbane, Australi

Mining Valence, arousal, and Dominance - Possibilities for detecting burnout and productivity?

Similar to other industries, the software engineering domain is plagued by psychological diseases such as burnout, which lead developers to lose interest, exhibit lower activity and/or feel powerless. Prevention is essential for such diseases, which in turn requires early identification of symptoms. The emotional dimensions of Valence, Arousal and Dominance (VAD) are able to derive a person's interest (attraction), level of activation and perceived level of control for a particular situation from textual communication, such as emails. As an initial step towards identifying symptoms of productivity loss in software engineering, this paper explores the VAD metrics and their properties on 700,000 Jira issue reports containing over 2,000,000 comments, since issue reports keep track of a developer's progress on addressing bugs or new features. Using a general-purpose lexicon of 14,000 English words with known VAD scores, our results show that issue reports of different type (e.g., Feature Request vs. Bug) have a fair variation of Valence, while increase in issue priority (e.g., from Minor to Critical) typically increases Arousal. Furthermore, we show that as an issue's resolution time increases, so does the arousal of the individual the issue is assigned to. Finally, the resolution of an issue increases valence, especially for the issue Reporter and for quickly addressed issues. The existence of such relations between VAD and issue report activities shows promise that text mining in the future could offer an alternative way for work health assessment surveys

A sentiment analysis approach to increase authorship identification

Writing style is considered the manner in which an author expresses his thoughts, influenced by language characteristics, period, school, or nation. Often, this writing style can identify the author. One of the most famous examples comes from 1914 in Portuguese literature. With Fernando Pessoa and his heteronyms Alberto Caeiro, alvaro de Campos, and Ricardo Reis, who had completely different writing styles, led people to believe that they were different individuals. Currently, the discussion of authorship identification is more relevant because of the considerable amount of widespread fake news in social media, in which it is hard to identify who authored a text and even a simple quote can impact the public image of an author, especially if these texts or quotes are from politicians. This paper presents a process to analyse the emotion contained in social media messages such as Facebook to identify the author's emotional profile and use it to improve the ability to predict the author of the message. Using preprocessing techniques, lexicon-based approaches, and machine learning, we achieved an authorship identification improvement of approximately 5% in the whole dataset and more than 50% in specific authors when considering the emotional profile on the writing style, thus increasing the ability to identify the author of a text by considering only the author's emotional profile, previously detected from prior texts.FCT has supported this work – Fundação para a Ciência e Tecnologia within the Project Scope: UID/CEC/00319/2019

Visual Affect Around the World: A Large-scale Multilingual Visual Sentiment Ontology

Every culture and language is unique. Our work expressly focuses on the uniqueness of culture and language in relation to human affect, specifically sentiment and emotion semantics, and how they manifest in social multimedia. We develop sets of sentiment- and emotion-polarized visual concepts by adapting semantic structures called adjective-noun pairs, originally introduced by Borth et al. (2013), but in a multilingual context. We propose a new language-dependent method for automatic discovery of these adjective-noun constructs. We show how this pipeline can be applied on a social multimedia platform for the creation of a large-scale multilingual visual sentiment concept ontology (MVSO). Unlike the flat structure in Borth et al. (2013), our unified ontology is organized hierarchically by multilingual clusters of visually detectable nouns and subclusters of emotionally biased versions of these nouns. In addition, we present an image-based prediction task to show how generalizable language-specific models are in a multilingual context. A new, publicly available dataset of >15.6K sentiment-biased visual concepts across 12 languages with language-specific detector banks, >7.36M images and their metadata is also released.Comment: 11 pages, to appear at ACM MM'1

Criminal narrative experience: relating emotions to offence narrative roles during crime commission

A neglected area of research within criminality has been that of the experience of the offence for the offender. The present study investigates the emotions and narrative roles that are experienced by an offender while committing a broad range of crimes and proposes a model of Criminal Narrative Experience (CNE). Hypotheses were derived from the Circumplex of Emotions (Russell, 1997), Frye (1957), Narrative Theory (McAdams, 1988) and its link with Investigative Psychology (Canter, 1994). The analysis was based on 120 cases. Convicted for a variety of crimes, incarcerated criminals were interviewed and the data were subjected to Smallest Space Analysis (SSA). Four themes of Criminal Narrative Experience (CNE) were identified: Elated Hero, Calm Professional, Distressed Revenger and Depressed Victim in line with the recent theoretical framework posited for Narrative Offence Roles (Youngs & Canter, 2012). The theoretical implications for understanding crime on the basis of the Criminal Narrative Experience (CNE) as well as practical implications are discussed

Multimodal database of emotional speech, video and gestures

People express emotions through different modalities. Integration of verbal and non-verbal communication channels creates a system in which the message is easier to understand. Expanding the focus to several expression forms can facilitate research on emotion recognition as well as human-machine interaction. In this article, the authors present a Polish emotional database composed of three modalities: facial expressions, body movement and gestures, and speech. The corpora contains recordings registered in studio conditions, acted out by 16 professional actors (8 male and 8 female). The data is labeled with six basic emotions categories, according to Ekman’s emotion categories. To check the quality of performance, all recordings are evaluated by experts and volunteers. The database is available to academic community and might be useful in the study on audio-visual emotion recognition

Understanding the Relationships between Tourists’ Emotional Experiences, Perceived Overall Image, Satisfaction, and Intention to Recommend

The purpose of this study is to empirically test an integrative model linking tourists' emotional experiences, perceived overall image, satisfaction, and intention to recommend. The model was tested using data collected from domestic tourists visiting Sardinia, Italy. Results show that tourists' emotional experiences act as antecedents of perceived overall image and satisfaction evaluations. In addition, overall image has a positive influence on tourist satisfaction and intention to recommend. The study expands current theorizations by examining the merits of emotions in tourist behavior models. From a practical perspective, the study offers important implications for destination marketers

SentiBench - a benchmark comparison of state-of-the-practice sentiment analysis methods

In the last few years thousands of scientific papers have investigated sentiment analysis, several startups that measure opinions on real data have emerged and a number of innovative products related to this theme have been developed. There are multiple methods for measuring sentiments, including lexical-based and supervised machine learning methods. Despite the vast interest on the theme and wide popularity of some methods, it is unclear which one is better for identifying the polarity (i.e., positive or negative) of a message. Accordingly, there is a strong need to conduct a thorough apple-to-apple comparison of sentiment analysis methods, \textit{as they are used in practice}, across multiple datasets originated from different data sources. Such a comparison is key for understanding the potential limitations, advantages, and disadvantages of popular methods. This article aims at filling this gap by presenting a benchmark comparison of twenty-four popular sentiment analysis methods (which we call the state-of-the-practice methods). Our evaluation is based on a benchmark of eighteen labeled datasets, covering messages posted on social networks, movie and product reviews, as well as opinions and comments in news articles. Our results highlight the extent to which the prediction performance of these methods varies considerably across datasets. Aiming at boosting the development of this research area, we open the methods' codes and datasets used in this article, deploying them in a benchmark system, which provides an open API for accessing and comparing sentence-level sentiment analysis methods

The clean conscience at work: Emotions, intuitions and morality

How do people decide what is right and wrong, and to what extent are their actions guided by such moral considerations? Inspired by philosophical traditions, early approaches to morality focused on rationality, and assumed that people arrive at moral standards by logical thought. More recently, however, psychologists have explored the influence of emotions and intuitions on morality, and evidence has been accumulating that moral decisions and behaviors are far from rational, but instead, are guided by intuitions and situational considerations. For example, seemingly irrelevant concerns such as keeping one’s mind and spirit clean and pure can change people’s moral judgment. Emotions can also influence behavior, and positive, uplifting emotions such as elevation and gratitude can be harnessed to produce beneficial outcomes for individuals and organizations alike. Furthermore, people appear to aspire to an equilibrium of moral self-worth, and engage in more or less ethical behavior depending on their currently perceived moral integrity. Thus, morality and ethical behavior is less likely to reside in the person than in the context, and thus, for the study of spirituality, it might be beneficial to focus on people’s situational constraints in the workplace rather than their stable dispositions. Further, because of their potential to inspire positive action, organizations might aim to make positive moral emotions, such as gratitude, elevation, and awe part of everyday work contexts. Overall, in organizations and the workplace, the goal shifts from trying to identify the moral individual to providing the contextual conditions that appeal to spiritual concerns in order to foster moral behavior.</jats:p