A deep learning approach for identifying sarcasm in text

The aim of this work is to evaluate the performance of deep learning, specifically models of Recurrent Neural Networks (RNN) and Convolutional Neural Networks (CNN), on the problem of detecting sarcasm in tweets. This is done partly by comparing our results to current state-of-the art performance, and partly by making a qualitative analysis of network functionality. In addition to this, we also conduct a survey to examine the human ability to detect sarcasm in tweets for result comparison. We examine three models: Two RNNs, one with Long Short Term Memory (LSTM) cells and one with Gated Recurrent Unit (GRU) cells, and also a CNN. Sarcasm detection is done by binary classification on the same datasets used by related works, and our performance is then compared to that of those works’. The main questions we aim to answer by analyzing the network functionality are what features affect the outcome, and how. By comparing our classifications with those of a basic bag-of-words model, scrambling the word content in tweets and looking at repeatedly misclassified tweets we are able to get a deeper understanding of the networks’ decisions. Experimental results suggest that the networks’ predictions mainly are based on word occurrence in the tweets. The best performance reach an F1-score of 0.842 when using the RNN with LSTM-cells. This network performed better overall among our models, indicating it might be the best option for this particular task. When conducting the survey, the model performed with an F1-score of 0.775 whereas humans reached an average score of 0.701. The model also performed better than a basic bag-of-words model, indicating that deep neural networks might be a feasible approach in tackling the problem of sarcasm detection in text