Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks

Sentiment analysis of online user generated content is important for many social media analytics tasks. Researchers have largely relied on textual sentiment analysis to develop systems to predict political elections, measure economic indicators, and so on. Recently, social media users are increasingly using images and videos to express their opinions and share their experiences. Sentiment analysis of such large scale visual content can help better extract user sentiments toward events or topics, such as those in image tweets, so that prediction of sentiment from visual content is complementary to textual sentiment analysis. Motivated by the needs in leveraging large scale yet noisy training data to solve the extremely challenging problem of image sentiment analysis, we employ Convolutional Neural Networks (CNN). We first design a suitable CNN architecture for image sentiment analysis. We obtain half a million training samples by using a baseline sentiment algorithm to label Flickr images. To make use of such noisy machine labeled data, we employ a progressive strategy to fine-tune the deep network. Furthermore, we improve the performance on Twitter images by inducing domain transfer with a small number of manually labeled Twitter images. We have conducted extensive experiments on manually labeled Twitter images. The results show that the proposed CNN can achieve better performance in image sentiment analysis than competing algorithms.Comment: 9 pages, 5 figures, AAAI 201

Sentiment Analysis Twitter Bahasa Indonesia Berbasis WORD2VEC Menggunakan Deep Convolutional Neural Network

Media sosial sebagai media informasi dan komunikasi mulai berkembang pesat sejak internet mudah diakses. Orang dengan mudah menyatakan pendapat, ekspresi, opini, dan informasi melalui tulisan pada media sosial. Opini atau informasi pada media sosial dapat digunakan untuk menilai baik atau buruk suatu brand perusahaan. Orang cenderung jujur dalam mengungkapkan perasaan terhadap sesuatu pada media sosial. Dengan menggunakan sentiment analysis terhadap opini dari pelanggan, analisis opini dapat dilakukan secara otomatis. Perusahaan dapat secara langsung mengetahui tingkat kepuasan pelanggan dan digunakan untuk meningkatkan kualitas pelayanan hingga menaikan brand perusahaan. Penggunaan metode classical machine learning yang sudah banyak diterapkan pada sentiment analysis, tetapi metode tersebut tidak memperhatikan pentingnya urutan kata pada suatu kalimat. Metode deep learning dengan algoritme Deep Convolutional Neural Network ditawarkan untuk menjawab permasalahan tersebut dengan melakukan operasi convolution menggunakan filter sebesar ukuran window untuk mendapatkan fitur berdasarkan urutan kata. Model Word2Vec untuk Bahasa Indonesia digunakan sebagai representasi kata dalam bentuk vektor. Penggunaan Word2Vec juga mempercepat proses pelatihan dan meningkatkan akurasi algoritme Deep Convolutional Neural Network. Data yang digunakan dalam makalah ini adalah data Twitter Bahasa Indonesia dengan jumlah 999 tweet. Hasil percobaan yang telah dilakukan dengan algoritme Deep Convolutional Neural Network memiliki nilai akurasi terbaik sebesar 76,40%. AbstractSocial media as information media and communication is growing rapidly since the internet is easily accessible. People easily express opinions, expressions, and information by writing on social media. Opinion or information on social media can be used to assess how good or bad a companies is. People tend to be honest in expressing feelings towards something on social media. With sentiment analysis, analysis of the opinions of customers can be done automatically. The company will know the level of customer satisfaction and can be used to improve the quality of service to raise the company's brand. The use of classical machine learning methods that have been widely applied to sentiment analysis ignoring the importance of the word order in a sentence. Deep Convolutional Neural Network algorithm is offered to answer these problems by carrying out convolution operations using filters as large as window size to get features based on word order. Word2Vec model for Indonesian is used as a word vector representation. The use of Word2Vec also reduce the training time and improve the accuracy of the Deep Convolutional Neural Network algorithm. The data used in this paper is Indonesian Twitter data with 999 tweets. The results of experiments that have been carried out with the Deep Convolutional Neural Network algorithm have the best accuracy value of 76.40%

From Pixels to Sentiment: Fine-tuning CNNs for Visual Sentiment Prediction

Visual multimedia have become an inseparable part of our digital social lives, and they often capture moments tied with deep affections. Automated visual sentiment analysis tools can provide a means of extracting the rich feelings and latent dispositions embedded in these media. In this work, we explore how Convolutional Neural Networks (CNNs), a now de facto computational machine learning tool particularly in the area of Computer Vision, can be specifically applied to the task of visual sentiment prediction. We accomplish this through fine-tuning experiments using a state-of-the-art CNN and via rigorous architecture analysis, we present several modifications that lead to accuracy improvements over prior art on a dataset of images from a popular social media platform. We additionally present visualizations of local patterns that the network learned to associate with image sentiment for insight into how visual positivity (or negativity) is perceived by the model.Comment: Accepted for publication in Image and Vision Computing. Models and source code available at https://github.com/imatge-upc/sentiment-201

Diving Deep into Sentiment: Understanding Fine-tuned CNNs for Visual Sentiment Prediction

Visual media are powerful means of expressing emotions and sentiments. The constant generation of new content in social networks highlights the need of automated visual sentiment analysis tools. While Convolutional Neural Networks (CNNs) have established a new state-of-the-art in several vision problems, their application to the task of sentiment analysis is mostly unexplored and there are few studies regarding how to design CNNs for this purpose. In this work, we study the suitability of fine-tuning a CNN for visual sentiment prediction as well as explore performance boosting techniques within this deep learning setting. Finally, we provide a deep-dive analysis into a benchmark, state-of-the-art network architecture to gain insight about how to design patterns for CNNs on the task of visual sentiment prediction.Comment: Preprint of the paper accepted at the 1st Workshop on Affect and Sentiment in Multimedia (ASM), in ACM MultiMedia 2015. Brisbane, Australi

Do Convolutional Networks need to be Deep for Text Classification ?

We study in this work the importance of depth in convolutional models for text classification, either when character or word inputs are considered. We show on 5 standard text classification and sentiment analysis tasks that deep models indeed give better performances than shallow networks when the text input is represented as a sequence of characters. However, a simple shallow-and-wide network outperforms deep models such as DenseNet with word inputs. Our shallow word model further establishes new state-of-the-art performances on two datasets: Yelp Binary (95.9\%) and Yelp Full (64.9\%)