Diving Deep into Sentiment: Understanding Fine-tuned CNNs for Visual Sentiment Prediction

Visual media are powerful means of expressing emotions and sentiments. The constant generation of new content in social networks highlights the need of automated visual sentiment analysis tools. While Convolutional Neural Networks (CNNs) have established a new state-of-the-art in several vision problems, their application to the task of sentiment analysis is mostly unexplored and there are few studies regarding how to design CNNs for this purpose. In this work, we study the suitability of fine-tuning a CNN for visual sentiment prediction as well as explore performance boosting techniques within this deep learning setting. Finally, we provide a deep-dive analysis into a benchmark, state-of-the-art network architecture to gain insight about how to design patterns for CNNs on the task of visual sentiment prediction.Comment: Preprint of the paper accepted at the 1st Workshop on Affect and Sentiment in Multimedia (ASM), in ACM MultiMedia 2015. Brisbane, Australi

Discovering conversational topics and emotions associated with Demonetization tweets in India

Social media platforms contain great wealth of information which provides us opportunities explore hidden patterns or unknown correlations, and understand people's satisfaction with what they are discussing. As one showcase, in this paper, we summarize the data set of Twitter messages related to recent demonetization of all Rs. 500 and Rs. 1000 notes in India and explore insights from Twitter's data. Our proposed system automatically extracts the popular latent topics in conversations regarding demonetization discussed in Twitter via the Latent Dirichlet Allocation (LDA) based topic model and also identifies the correlated topics across different categories. Additionally, it also discovers people's opinions expressed through their tweets related to the event under consideration via the emotion analyzer. The system also employs an intuitive and informative visualization to show the uncovered insight. Furthermore, we use an evaluation measure, Normalized Mutual Information (NMI), to select the best LDA models. The obtained LDA results show that the tool can be effectively used to extract discussion topics and summarize them for further manual analysis.Comment: 6 pages, 11 figures. arXiv admin note: substantial text overlap with arXiv:1608.02519 by other authors; text overlap with arXiv:1705.08094 by other author

Pitch Patterns in Vocal Expression of “Happiness” and “Sadness” in the Reading Aloud of Prose on the Basis of Selected Audiobooks

The primary focus of this paper is to examine the way the emotional categories of “happiness” and “sadness” are expressed vocally in the reading aloud of prose. In particular, the two semantic categories were analysed in terms of the pitch level and the pitch variability on a corpus based on 28 works written by Charles Dickens. passages with the intended emotional colouring were selected and the fragments found in the corresponding audiobooks. They were then analysed acoustically in terms of the mean F0 and the standard deviation of F0. The results for individual emotional passages were compared with a particular reader’s mean pitch and standard deviation of pitch. The differences obtained in this way supported the initial assumptions that the pitch level and its standard deviation would raise in “happy” extracts but lower in “sad” ones. Nevertheless, not all of these tendencies could be statistically validated and additional examples taken from a selection of random novels by other nineteenth century writers were added. The statistical analysis of the larger samples confirmed the assumed tendencies but also indicated that the two semantic domains may utilise the acoustic parameters under discussion to varying degrees. While “happiness” tends to be signalled primarily by raising F0, “sadness” is communicated mostly by lowering the variability of F0. Changes in the variability of F0 seem to be of less importance in the former case, and shifts in the F0 level less significant in the latter

A sentiment analysis approach to increase authorship identification

Writing style is considered the manner in which an author expresses his thoughts, influenced by language characteristics, period, school, or nation. Often, this writing style can identify the author. One of the most famous examples comes from 1914 in Portuguese literature. With Fernando Pessoa and his heteronyms Alberto Caeiro, alvaro de Campos, and Ricardo Reis, who had completely different writing styles, led people to believe that they were different individuals. Currently, the discussion of authorship identification is more relevant because of the considerable amount of widespread fake news in social media, in which it is hard to identify who authored a text and even a simple quote can impact the public image of an author, especially if these texts or quotes are from politicians. This paper presents a process to analyse the emotion contained in social media messages such as Facebook to identify the author's emotional profile and use it to improve the ability to predict the author of the message. Using preprocessing techniques, lexicon-based approaches, and machine learning, we achieved an authorship identification improvement of approximately 5% in the whole dataset and more than 50% in specific authors when considering the emotional profile on the writing style, thus increasing the ability to identify the author of a text by considering only the author's emotional profile, previously detected from prior texts.FCT has supported this work – Fundação para a Ciência e Tecnologia within the Project Scope: UID/CEC/00319/2019

Visual Affect Around the World: A Large-scale Multilingual Visual Sentiment Ontology

Every culture and language is unique. Our work expressly focuses on the uniqueness of culture and language in relation to human affect, specifically sentiment and emotion semantics, and how they manifest in social multimedia. We develop sets of sentiment- and emotion-polarized visual concepts by adapting semantic structures called adjective-noun pairs, originally introduced by Borth et al. (2013), but in a multilingual context. We propose a new language-dependent method for automatic discovery of these adjective-noun constructs. We show how this pipeline can be applied on a social multimedia platform for the creation of a large-scale multilingual visual sentiment concept ontology (MVSO). Unlike the flat structure in Borth et al. (2013), our unified ontology is organized hierarchically by multilingual clusters of visually detectable nouns and subclusters of emotionally biased versions of these nouns. In addition, we present an image-based prediction task to show how generalizable language-specific models are in a multilingual context. A new, publicly available dataset of >15.6K sentiment-biased visual concepts across 12 languages with language-specific detector banks, >7.36M images and their metadata is also released.Comment: 11 pages, to appear at ACM MM'1

Mining Valence, arousal, and Dominance - Possibilities for detecting burnout and productivity?

Similar to other industries, the software engineering domain is plagued by psychological diseases such as burnout, which lead developers to lose interest, exhibit lower activity and/or feel powerless. Prevention is essential for such diseases, which in turn requires early identification of symptoms. The emotional dimensions of Valence, Arousal and Dominance (VAD) are able to derive a person's interest (attraction), level of activation and perceived level of control for a particular situation from textual communication, such as emails. As an initial step towards identifying symptoms of productivity loss in software engineering, this paper explores the VAD metrics and their properties on 700,000 Jira issue reports containing over 2,000,000 comments, since issue reports keep track of a developer's progress on addressing bugs or new features. Using a general-purpose lexicon of 14,000 English words with known VAD scores, our results show that issue reports of different type (e.g., Feature Request vs. Bug) have a fair variation of Valence, while increase in issue priority (e.g., from Minor to Critical) typically increases Arousal. Furthermore, we show that as an issue's resolution time increases, so does the arousal of the individual the issue is assigned to. Finally, the resolution of an issue increases valence, especially for the issue Reporter and for quickly addressed issues. The existence of such relations between VAD and issue report activities shows promise that text mining in the future could offer an alternative way for work health assessment surveys

Dual harm: an exploration of the presence and characteristics for dual violence and self-harm behaviour in prison

Objective: The study aimed to quantify the rate of dual-harm behaviour in comparison with sole self-harm or assault rates; with an analysis of the distinguishing features. Method: Official data on in-prison incidents, demographic and offending information was analysed for two prisons in England. Results: Proportions of up to 42% of offenders who assault others in prison will also engage in self-harm and vice versa. Dual harm prisoners will engage in a broader and greater frequency of prison incidents than either sole group; with dual-harm prisoners reflecting greater proportions of damage to property and fire setting. Connectedly, dual harm prisoners receive a far higher rate of adjudication. There were no differences in their time in prison, presence of serious violent offences or for the dual harm prisoners whether the first incident was self-harm or violence. An index offence of drug supply was less likely in the dual-harm group, with minor violence slightly more likely in longer sentence prisoners. Implications: In-prison behaviour can assist in the identification of prisoners at dual-risk of harm. Greater inclusion of in-prison behaviour and awareness of dual-harm in research methodologies may assist in improving risk management. A wider use of joint risk assessment and single case management approach is suggested for prisoners with dual-harm profile

Movies emotional analysis using textual contents

In this paper, we use movies and series subtitles and applied text mining and Natural Language Processing methods to evaluate emotions in videos. Three different word lexicons were used and one of the outcomes of this research is the generation of a secondary dataset with more than 3658 records which can be used for other data analysis and data mining research. We used our secondary dataset to find and display correlations between different emotions on the videos and the correlation between emotions on the movies and users’ scores on IMDb using the Pearson correlation method and found some statistically significant correlations

Team Players and Collective Performance: How Agreeableness Affects Team Performance Over Time

Previous research on teams has found that agreeableness is one of the strongest personality predictors of team performance, yet one of the weakest personality predictors of individual-level job performance. In this study, we examined why teams with more agreeable members perform better. Data were collected across 4 months at 5 points in time from 107 project teams. We found that agreeableness affects performance through communication and cohesion and that communication precedes cohesion in time. Furthermore, we found that virtualness moderated the relationships between agreeableness and communication, as well as between agreeableness and team performance, such that teams only benefitted from high levels of agreeableness when interacting face-to-face.Yeshttps://us.sagepub.com/en-us/nam/manuscript-submission-guideline