Multimodal Sentiment Analysis: Perceived vs Induced Sentiments

Social media has created a global network where people can easily access and exchange vast information. This information gives rise to a variety of opinions, reflecting both positive and negative viewpoints. GIFs stand out as a multimedia format offering a visually engaging way for users to communicate. In this research, we propose a multimodal framework that integrates visual and textual features to predict the GIF sentiment. It also incorporates attributes including face emotion detection and OCR generated captions to capture the semantic aspects of the GIF. The developed classifier achieves an accuracy of 82.7% on Twitter GIFs, which is an improvement over state-of-the-art models. Moreover, we have based our research on the ReactionGIF dataset, analysing the variance in sentiment perceived by the author and sentiment induced in the reade

Predicting Perceived Emotions in Animated GIFs with 3D Convolutional Neural Networks

© 2016 IEEE. Animated GIFs are widely used on the Internet to express emotions, but their automatic analysis is largely unexplored before. To help with the search and recommendation of GIFs, we aim to predict their emotions perceived by humans based on their contents. Since previous solutions to this problem only utilize image-based features and lose all the motion information, we propose to use 3D convolutional neural networks (CNNs) to extract spatiotemporal features from GIFs. We evaluate our methodology on a crowd-sourcing platform called GIFGIF with more than 6000 animated GIFs, and achieve a better accuracy then any previous approach in predicting crowd-sourced intensity scores of 17 emotions. It is also found that our trained model can be used to distinguish and cluster emotions in terms of valence and risk perception

The Phenomenological Exploration of Animated GIF Use in Computer-Mediated Communication

The current study seeks to remedy the lack of scholarly investigation into the use of animated GIFs in computer-mediated communication (CMC). Through phenomenological analysis of in-depth 1-on-1 interviews with individuals engaging in the behavior, one over-arching theme was found with the four underlying sub-themes of: Choice, Meaning, Use and Gratification. Individuals using animated GIFs in their CMC seem to formulate a mental image of an expression they wish to demonstrate and select a GIF that fits a particular context, within a specific conversation, with a specific person. Individuals seem to construct meaning of animated GIFs by reading social cues such as facial expressions and body language presented by the actors in the GIF and combining it with the context of the conversation and the person or persons they are communicating with. Individuals seem to use animated GIFs to actively compensate for the lack of social cue transmission in CMC, and seem do so for the purpose of humor, clarification of message, and to increase saliency. Lastly, this whole process seems to be lubricated by a feedback loop of gratification where in individuals feel their communication is improved and more enjoyable than with just words. The current findings are relevant to theories of communication as well as to online education. Recommendations for future research into their effectiveness for educational purposes are provided