Multimodal Sentiment Analysis of Instagram Using Cross-media Bag-of-words Model

Abstract

Instagram, one of social media sharing services has increasing growth of use and popularity during recent years. Photos or videos shared by Instagram users are challenging to be mined and analyzed for some purposes. One type of studies can be applied to Instagram data is sentiment analysis, a field of study that learn and analyze people opinion, sentiment, and (or) evaluation about something. Sentiment analysis applied to Instagram can be used as analytics tool for some business purposes such as user behavior, market intelligence and user evaluation. This research aimed to analyze sentiment contained on Instagrams post by considering two modalities: images and English text on its caption. The Cross-media Bag-of-Words Model (CBM) was applied for analyzing the sentiment contained on Instagrams post. CBM treated text and image features as a unit of vector representation. These cross-media features then classified using logistic regression to predict sentiment values which categorized into three classes: positive, negative and neutral. Simulation results showed that the combination of unigram text features and 56-length images features achieves the highest accuracy. The accuracy achieved is 87.2%. Keywords : Instagram, sentiment analysis, Cross-media Bag-of-Words Model (CBM), logistic regression, classification Bibliography [1] D. Borth, R. Ji, T. Chen, T. Breuel, and S.-F. Chang, “Large-scale visual sentiment ontology and detectors using adjective noun pairs,” in Proceedings of the 21st ACM International Conference on Multimedia, ser. MM '13. New York, NY, USA: ACM, 2013, pp. 223–232. [2] R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin, “Liblinear: A library for large linear classification,” J. Mach. Learn. Res., vol. 9, pp. 1871– 1874, Jun. 2008. [3] E. Ferrara, R. Interdonato, and A. Tagarelli, “Online popularity and topical interests through the lens of instagram,” in Proceedings of the 25th ACM Conference on Hypertext and Social Media, ser. HT '14. New York, NY, USA: ACM, 2014, pp. 24–34. [4] N. Gunawardena, J. Plumb, N. Xiao, and H. Zhang, “Instagram hashtag sentiment analysis,” in University of Utah CS530/CS630 Conference of Machine Learning 2013. Universiti of Utah CS530/CS630 Conference of Machine Learning 2013, 2013. [5] J. Han, Data Mining: Concepts and Techniques. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2005. [6] M. Hu and B. Liu, “Mining and summarizing customer reviews,” inProceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2004, pp. 168–177.[7] Y. Hu, L. Manikonda, and S. Kambhampati, “What we instagram: A first analysis of instagram photo content and user types,” International AAAI Conference on Weblogs and Social Media, 2014. [8] L. S. Huey and R. Yazdanifard, “How instagram can be used as a tool in social networking marketing,” Help College of Art and Technology Malaysia, Tech. Rep., 2014. [9] D. Jurafsky and J. H. Martin, Speech and Language Processing: An introduction to natural language processing, computational linguistics, and speech recognition(2nd Edition). Upper Saddle River, NJ, USA: Prentice-Hall, Inc., 2009. [10] S. G. K and S. Joseph, “Text classification by augmenting bag of words ( bow ) representation with co-occurrence feature,” IOSR Journal of Computer Engineering (IOSR-JCE), vol. 16, pp. 34–38, 1 2014. [11] B. B. Kachru, The Alchemy of English: The Spread, Functions and Models of Non-native Englishes. Champaign: University of Illinois Press, 1990. [12] S. S. Keerthi and C.-J. Lin, “Asymptotic behaviors of support vector machines with gaussian kernel,” Neural Comput., vol. 15, no. 7, pp. 1667–1689, Jul. 2003. [13] M. Koppel and J. Schler, “The importance of neutral examples for learning sentiment,” in In Workshop on the Analysis of Informal and Formal Information Exchange during Negotiations, 2005.[14] A. Kowcika, A. Gupta, K. Sondhi, N. Shivhre, and R. Kumar, “Sentiment analysis for social media,” International Journal of Advanced Research in Computer Science and Software Engineering, vol. 3, no. 7, 7 2013. [15] F.-F. Li and P. Perona, “A bayesian hierarchical model for learning natural scene categories,” in Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02, ser. CVPR '05. Washington, DC, USA: IEEE Computer Society, 2005, pp. 524–531. [16] B. Liu, Sentiment Analysis and Opinion Mining. Morgan and Claypool Publisher, 2012. [17] W.-Y. Ma and K.-J. Chen, “A bottom-up merging algorithm for chinese unknown word extraction,” in Proceedings of the Second SIGHAN Workshop on Chinese Language Processing - Volume 17, ser. SIGHAN '03. Stroudsburg, PA, USA: Association for Computational Linguistics, 2003, pp. 31–38. [18] Z. McCune, “Consumer production in social media networks: A case study of the instagram iphone app,” Ph.D. dissertation, Dr. John Thompson, 2011. [19] W. Medhata, A. Hassanb, and H. Korashyb, “Sentiment analysis algorithms and applications: A survey,” Ain Shams Engineering Journal, 2014. [20] L.-P. Morency, R. Mihalcea, and P. Doshi, “Toward multimodal sentiment analysis: Harvesting opinion from the web,” in International Conference on Multimodal Interface, 2011. [21] A. Pak and P. Paroubek, “Twitter as a corpus for sentiment analysis and opinion mining,” in Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10), N. C. C. Chair), K. Choukri, B. Maegaard, J. Mariani, J. Odijk, S. Piperidis, M. Rosner, and D. Tapias, Eds. Valletta, Malta: European Language Resources Association (ELRA), may 2010.[22] B. Pang and L. Lee, “Opinion mining and sentiment analysis,” Foundation and Trends in Information Retrieval, vol. 2, no. 1-2, p. 4, 2008. [23] B. Pang, L. Lee, and S. Vaithyanathan, “Thumbs up?: Sentiment classification using machine learning techniques,” in Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing - Volume 10, ser. EMNLP '02. Stroudsburg, PA, USA: Association for Computational Linguistics, 2002, pp. 79–86. [24] C.-Y. J. Peng, K. L. Lee, and G. M. Ingersol, “An introduction to logistic regression analysis and reporting,” The Journal of Educational Research, vol. 96, no. 1, September/October 2002. [25] V. V. Piyush Bansal, Romil Bansal, “Towards deep semantic analysis of hashtags,” 37th European Conference on Information Retrieval, 2015. [26] R. Plutchik, Emotion: A Psycho-evolutionary Synthesis. Harper and Row, 1980. [27] S. Poria, A. Hussain, and E. Cambria, “Beyond text based sentiment analysis: Towards multi-modal systems,” University of Stirling, Stirling FK9 4LA, UK, Tech. Rep., 2013. [Online]. Available: http://www.cs.stir.ac.uk/~spo/publication/resources/cogcomp.pdf [28] E. Praseyto, Data Mining Konsep dan Aplikasi Menggunakan Matlab. Yogyakarta: Andi, 2012.[29] A. Qazi, R. G. Raj, M. Tahir, E. Cambria, and K. B. S. Syed, “Enhancing business intelligence by means of suggestive reviews,” The Scientific World Journal, vol. 2014, June 2014. [30] R. Schapire, “Machine learning algorithms for classification,” Princeton University, Tech. Rep. [31] S. Siersdorfer, E. Minack, F. Deng, and J. Hare, “Analyzing and predicting sentiment of images on the social web,” in Proceedings of the International Conference on Multimedia, ser. MM '10. New York, NY, USA: ACM, 2010, pp. 715–718. [32] T. H. Silva, P. O. S. V. de Melo, J. M. Almeida, J. Salles, and A. A. F. Loureiro, “A comparison of foursquare and instagram to the study of city dynamics and urban social behavior,” in Proceedings of the 2Nd ACM SIGKDD International Workshop on Urban Computing, ser. UrbComp '13. New York, NY, USA: ACM, 2013, pp. 4:1–4:8. [33] P. N. Stuart Russell, Artificial Intelligence A Modern Approach, M. Hirsch, Ed. New Jersey: Pearson Education, 2010. [34] M. Taboada, J. Brooke, M. Tofiloski, K. Voll, and M. Stede, “Lexicon-based methods for sentiment analysis,” Comput. Linguist., vol. 37, no. 2, pp. 267– 307, Jun. 2011. [35] C.-F. Tsai, “Bag-of-words representation in image annotation: A review,” ISRN Artificial Intelligence, p. 19, 2012. [36] A. J. Viera and J. M. Garrett, “Understanding interobserver agreement: The kappa statistic,” Family Medicine, vol. 37, no. 5, pp. 360–363, May 2005. [37] M. Wang, D. Cao, L. Li, S. Li, and R. Ji, “Microblog sentiment analysis based on cross-media bag-of-words model,” in Proceedings of International Conference on Internet Multimedia Computing and Service, ser. ICIMCS '14. New York, NY, USA: ACM, 2014, pp. 76:76–76:80. [38] A. Westerski, “Sentiment analysis: Introduction and the state of the art overview,” Universidad Politecnica de Madrid, Spain, Tech. Rep., 2009. [39] F. Yu, L. Cao, R. Feris, J. Smith, and S.-F. Chang, “Designing category-level attributes for discriminative visual recognition,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Portland, OR, June 2013. [40] L. Yu and H. Liu, “Efficiently handling feature redundancy in high-dimensional data,” in Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ser. KDD '03. New York, NY, USA: ACM, 2003, pp. 685–690. [41] J. Yuan, S. Mcdonough, Q. You, and J. Luo, “Sentribute: Image sentiment analysis from a mid-level perspective,” in Proceedings of the Second International Workshop on Issues of Sentiment Discovery and Opinion Mining, ser. WISDOM '13. New York, NY, USA: ACM, 2013, pp. 10:1– 10:8