Sentiment Recognition in Egocentric Photostreams

Lifelogging is a process of collecting rich source of information about daily life of people. In this paper, we introduce the problem of sentiment analysis in egocentric events focusing on the moments that compose the images recalling positive, neutral or negative feelings to the observer. We propose a method for the classification of the sentiments in egocentric pictures based on global and semantic image features extracted by Convolutional Neural Networks. We carried out experiments on an egocentric dataset, which we organized in 3 classes on the basis of the sentiment that is recalled to the user (positive, negative or neutral)

Research on multi-modal sentiment feature learning of social media content

社交媒体已成为现代社会舆论交流和信息传递的主要平台。针对社交媒体的情感分析对于舆论监控、商业产品导向和股市预测等都具有重大应用价值。但社交媒体内容的多模态性（文本、图片等）让传统的单模态情感分析方法面临许多局限，多模态情感分析技术对跨媒体内容的理解与分析具有重大的理论价值。 多模态情感分析区别于单模态方法的关键问题在于，如何综合利用形态各异的多模态情感信息，来获取整体的情感倾向性，同时考虑单个模态本身在情感表达上的性质。针对该问题，利用社交媒体上的多模态内容在情感表达上所具有的关联性、抽象层级性的特点，提出了一套面向社交媒体的多模态情感特征学习与融合方法，实现多模态情感分析，主要内容和创新点...Social media has become a main platform of public communication and information transmission. Therefore, social media sentiment analysis has great application values in many fields, such as public opinion monitoring, production marking, stock forecasting and so on. But the multi-modal characteristic of social media content (e.g. texts and images) significantly challenges traditional text-based sen...学位：工学硕士院系专业：信息科学与技术学院_模式识别与智能系统学号：3152013115327

A multimodal feature learning approach for sentiment analysis of social network multimedia

Investigation of the use of a multimodal feature learning approach, using neural network based models such as Skip-gram and Denoising Autoencoders, to address sentiment analysis of micro-blogging content, such as Twitter short messages, that are composed by a short text and, possibly, an image

Sensing Human Sentiment via Social Media Images: Methodologies and Applications

abstract: Social media refers computer-based technology that allows the sharing of information and building the virtual networks and communities. With the development of internet based services and applications, user can engage with social media via computer and smart mobile devices. In recent years, social media has taken the form of different activities such as social network, business network, text sharing, photo sharing, blogging, etc. With the increasing popularity of social media, it has accumulated a large amount of data which enables understanding the human behavior possible. Compared with traditional survey based methods, the analysis of social media provides us a golden opportunity to understand individuals at scale and in turn allows us to design better services that can tailor to individuals’ needs. From this perspective, we can view social media as sensors, which provides online signals from a virtual world that has no geographical boundaries for the real world individual's activity. One of the key features for social media is social, where social media users actively interact to each via generating content and expressing the opinions, such as post and comment in Facebook. As a result, sentiment analysis, which refers a computational model to identify, extract or characterize subjective information expressed in a given piece of text, has successfully employs user signals and brings many real world applications in different domains such as e-commerce, politics, marketing, etc. The goal of sentiment analysis is to classify a user’s attitude towards various topics into positive, negative or neutral categories based on textual data in social media. However, recently, there is an increasing number of people start to use photos to express their daily life on social media platforms like Flickr and Instagram. Therefore, analyzing the sentiment from visual data is poise to have great improvement for user understanding. In this dissertation, I study the problem of understanding human sentiments from large scale collection of social images based on both image features and contextual social network features. We show that neither visual features nor the textual features are by themselves sufficient for accurate sentiment prediction. Therefore, we provide a way of using both of them, and formulate sentiment prediction problem in two scenarios: supervised and unsupervised. We first show that the proposed framework has flexibility to incorporate multiple modalities of information and has the capability to learn from heterogeneous features jointly with sufficient training data. Secondly, we observe that negative sentiment may related to human mental health issues. Based on this observation, we aim to understand the negative social media posts, especially the post related to depression e.g., self-harm content. Our analysis, the first of its kind, reveals a number of important findings. Thirdly, we extend the proposed sentiment prediction task to a general multi-label visual recognition task to demonstrate the methodology flexibility behind our sentiment analysis model.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Microblog sentiment analysis based on cross-media bag-of-words model

Conference Name:6th International Conference on Internet Multimedia Computing and Service, ICIMCS 2014. Conference Address: Xiamen, China. Time:July 10, 2014 - July 12, 2014.National Natural Foundation of China; SIGMM China Chapter; Xiamen UniversitySentiment analysis on social networks has attracted ever increasing attention in recent years. To this end, most existing methods mainly focus on analyzing the textual information. This has faced huge difficulty as nowadays users are more likely to express their feelings in a hybrid manner not only with texts, but also with images. It is therefore essential to take images into account. In this paper, we propose a novel Cross-media Bag-of-words Model (CBM) for Microblog sentiment analysis. In this model, we represent the text and image of a Weibo tweet as a unified Bag-of-words representation. Based on this model, we use Logistic Regression to classify the Microblog sentiment. It performs well in the sentiment classification task since it doesn't require the conditional dependence assumption. We also use SVM and Naive Bayes to make a comparison. Experiments on 5,000 Microblog messages demonstrate that our CBM model performs better than text-based methods. The sentiment classification accuracy on Microblog messages of our model is 80%, improved by 4% than the text-based methods. Copyright 2014 ACM

Multimodal Sentiment Analysis of Instagram Using Cross-media Bag-of-words Model

Instagram, one of social media sharing services has increasing growth of use and popularity during recent years. Photos or videos shared by Instagram users are challenging to be mined and analyzed for some purposes. One type of studies can be applied to Instagram data is sentiment analysis, a field of study that learn and analyze people opinion, sentiment, and (or) evaluation about something. Sentiment analysis applied to Instagram can be used as analytics tool for some business purposes such as user behavior, market intelligence and user evaluation. This research aimed to analyze sentiment contained on Instagrams post by considering two modalities: images and English text on its caption. The Cross-media Bag-of-Words Model (CBM) was applied for analyzing the sentiment contained on Instagrams post. CBM treated text and image features as a unit of vector representation. These cross-media features then classified using logistic regression to predict sentiment values which categorized into three classes: positive, negative and neutral. Simulation results showed that the combination of unigram text features and 56-length images features achieves the highest accuracy. The accuracy achieved is 87.2%. Keywords : Instagram, sentiment analysis, Cross-media Bag-of-Words Model (CBM), logistic regression, classification Bibliography [1] D. Borth, R. Ji, T. Chen, T. Breuel, and S.-F. Chang, “Large-scale visual sentiment ontology and detectors using adjective noun pairs,” in Proceedings of the 21st ACM International Conference on Multimedia, ser. MM '13. New York, NY, USA: ACM, 2013, pp. 223–232. [2] R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin, “Liblinear: A library for large linear classification,” J. Mach. Learn. Res., vol. 9, pp. 1871– 1874, Jun. 2008. [3] E. Ferrara, R. Interdonato, and A. Tagarelli, “Online popularity and topical interests through the lens of instagram,” in Proceedings of the 25th ACM Conference on Hypertext and Social Media, ser. HT '14. New York, NY, USA: ACM, 2014, pp. 24–34. [4] N. Gunawardena, J. Plumb, N. Xiao, and H. Zhang, “Instagram hashtag sentiment analysis,” in University of Utah CS530/CS630 Conference of Machine Learning 2013. Universiti of Utah CS530/CS630 Conference of Machine Learning 2013, 2013. [5] J. Han, Data Mining: Concepts and Techniques. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2005. [6] M. Hu and B. Liu, “Mining and summarizing customer reviews,” inProceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2004, pp. 168–177.[7] Y. Hu, L. Manikonda, and S. Kambhampati, “What we instagram: A first analysis of instagram photo content and user types,” International AAAI Conference on Weblogs and Social Media, 2014. [8] L. S. Huey and R. Yazdanifard, “How instagram can be used as a tool in social networking marketing,” Help College of Art and Technology Malaysia, Tech. Rep., 2014. [9] D. Jurafsky and J. H. Martin, Speech and Language Processing: An introduction to natural language processing, computational linguistics, and speech recognition(2nd Edition). Upper Saddle River, NJ, USA: Prentice-Hall, Inc., 2009. [10] S. G. K and S. Joseph, “Text classification by augmenting bag of words ( bow ) representation with co-occurrence feature,” IOSR Journal of Computer Engineering (IOSR-JCE), vol. 16, pp. 34–38, 1 2014. [11] B. B. Kachru, The Alchemy of English: The Spread, Functions and Models of Non-native Englishes. Champaign: University of Illinois Press, 1990. [12] S. S. Keerthi and C.-J. Lin, “Asymptotic behaviors of support vector machines with gaussian kernel,” Neural Comput., vol. 15, no. 7, pp. 1667–1689, Jul. 2003. [13] M. Koppel and J. Schler, “The importance of neutral examples for learning sentiment,” in In Workshop on the Analysis of Informal and Formal Information Exchange during Negotiations, 2005.[14] A. Kowcika, A. Gupta, K. Sondhi, N. Shivhre, and R. Kumar, “Sentiment analysis for social media,” International Journal of Advanced Research in Computer Science and Software Engineering, vol. 3, no. 7, 7 2013. [15] F.-F. Li and P. Perona, “A bayesian hierarchical model for learning natural scene categories,” in Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02, ser. CVPR '05. Washington, DC, USA: IEEE Computer Society, 2005, pp. 524–531. [16] B. Liu, Sentiment Analysis and Opinion Mining. Morgan and Claypool Publisher, 2012. [17] W.-Y. Ma and K.-J. Chen, “A bottom-up merging algorithm for chinese unknown word extraction,” in Proceedings of the Second SIGHAN Workshop on Chinese Language Processing - Volume 17, ser. SIGHAN '03. Stroudsburg, PA, USA: Association for Computational Linguistics, 2003, pp. 31–38. [18] Z. McCune, “Consumer production in social media networks: A case study of the instagram iphone app,” Ph.D. dissertation, Dr. John Thompson, 2011. [19] W. Medhata, A. Hassanb, and H. Korashyb, “Sentiment analysis algorithms and applications: A survey,” Ain Shams Engineering Journal, 2014. [20] L.-P. Morency, R. Mihalcea, and P. Doshi, “Toward multimodal sentiment analysis: Harvesting opinion from the web,” in International Conference on Multimodal Interface, 2011. [21] A. Pak and P. Paroubek, “Twitter as a corpus for sentiment analysis and opinion mining,” in Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10), N. C. C. Chair), K. Choukri, B. Maegaard, J. Mariani, J. Odijk, S. Piperidis, M. Rosner, and D. Tapias, Eds. Valletta, Malta: European Language Resources Association (ELRA), may 2010.[22] B. Pang and L. Lee, “Opinion mining and sentiment analysis,” Foundation and Trends in Information Retrieval, vol. 2, no. 1-2, p. 4, 2008. [23] B. Pang, L. Lee, and S. Vaithyanathan, “Thumbs up?: Sentiment classification using machine learning techniques,” in Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing - Volume 10, ser. EMNLP '02. Stroudsburg, PA, USA: Association for Computational Linguistics, 2002, pp. 79–86. [24] C.-Y. J. Peng, K. L. Lee, and G. M. Ingersol, “An introduction to logistic regression analysis and reporting,” The Journal of Educational Research, vol. 96, no. 1, September/October 2002. [25] V. V. Piyush Bansal, Romil Bansal, “Towards deep semantic analysis of hashtags,” 37th European Conference on Information Retrieval, 2015. [26] R. Plutchik, Emotion: A Psycho-evolutionary Synthesis. Harper and Row, 1980. [27] S. Poria, A. Hussain, and E. Cambria, “Beyond text based sentiment analysis: Towards multi-modal systems,” University of Stirling, Stirling FK9 4LA, UK, Tech. Rep., 2013. [Online]. Available: http://www.cs.stir.ac.uk/~spo/publication/resources/cogcomp.pdf [28] E. Praseyto, Data Mining Konsep dan Aplikasi Menggunakan Matlab. Yogyakarta: Andi, 2012.[29] A. Qazi, R. G. Raj, M. Tahir, E. Cambria, and K. B. S. Syed, “Enhancing business intelligence by means of suggestive reviews,” The Scientific World Journal, vol. 2014, June 2014. [30] R. Schapire, “Machine learning algorithms for classification,” Princeton University, Tech. Rep. [31] S. Siersdorfer, E. Minack, F. Deng, and J. Hare, “Analyzing and predicting sentiment of images on the social web,” in Proceedings of the International Conference on Multimedia, ser. MM '10. New York, NY, USA: ACM, 2010, pp. 715–718. [32] T. H. Silva, P. O. S. V. de Melo, J. M. Almeida, J. Salles, and A. A. F. Loureiro, “A comparison of foursquare and instagram to the study of city dynamics and urban social behavior,” in Proceedings of the 2Nd ACM SIGKDD International Workshop on Urban Computing, ser. UrbComp '13. New York, NY, USA: ACM, 2013, pp. 4:1–4:8. [33] P. N. Stuart Russell, Artificial Intelligence A Modern Approach, M. Hirsch, Ed. New Jersey: Pearson Education, 2010. [34] M. Taboada, J. Brooke, M. Tofiloski, K. Voll, and M. Stede, “Lexicon-based methods for sentiment analysis,” Comput. Linguist., vol. 37, no. 2, pp. 267– 307, Jun. 2011. [35] C.-F. Tsai, “Bag-of-words representation in image annotation: A review,” ISRN Artificial Intelligence, p. 19, 2012. [36] A. J. Viera and J. M. Garrett, “Understanding interobserver agreement: The kappa statistic,” Family Medicine, vol. 37, no. 5, pp. 360–363, May 2005. [37] M. Wang, D. Cao, L. Li, S. Li, and R. Ji, “Microblog sentiment analysis based on cross-media bag-of-words model,” in Proceedings of International Conference on Internet Multimedia Computing and Service, ser. ICIMCS '14. New York, NY, USA: ACM, 2014, pp. 76:76–76:80. [38] A. Westerski, “Sentiment analysis: Introduction and the state of the art overview,” Universidad Politecnica de Madrid, Spain, Tech. Rep., 2009. [39] F. Yu, L. Cao, R. Feris, J. Smith, and S.-F. Chang, “Designing category-level attributes for discriminative visual recognition,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Portland, OR, June 2013. [40] L. Yu and H. Liu, “Efficiently handling feature redundancy in high-dimensional data,” in Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ser. KDD '03. New York, NY, USA: ACM, 2003, pp. 685–690. [41] J. Yuan, S. Mcdonough, Q. You, and J. Luo, “Sentribute: Image sentiment analysis from a mid-level perspective,” in Proceedings of the Second International Workshop on Issues of Sentiment Discovery and Opinion Mining, ser. WISDOM '13. New York, NY, USA: ACM, 2013, pp. 10:1– 10:8