Most research in the area of emotion detection in written text focused on detecting explicit expressions of emotions in text. In this paper, we present a rule-based pipeline approach for detecting implicit emotions in written text without emotion-bearing words based on the OCC Model. We have evaluated our approach on three different datasets with five emotion categories. Our results show that the proposed approach outperforms the lexicon matching method consistently across all the three datasets by a large margin of 17–30% in F-measure and gives competitive performance compared to a supervised classifier. In particular, when dealing with formal text which follows grammatical rules strictly, our approach gives an average F-measure of 82.7% on “Happy”, “Angry-Disgust” and “Sad”, even outperforming the supervised baseline by nearly 17% in F-measure. Our preliminary results show the feasibility of the approach for the task of implicit emotion detection in written text

A Ortony

J Friedenberg

L Polanyi

LK Hansen

P Ekman

S Marsella

English

Orizu Udochukwu

Yulan He

Crossref

A Rule-Based Approach to Implicit Emotion Detection in Text

Orizu, Udochukwu

He, Yulan

Aston Publications Explorer

A Rule-Based Approach to Implicit EmotionDetection in TextOrizu Udochukwu and Yulan HeSchool of Engineering and Applied ScienceAston University,UK{orizuus,y.he9}@aston.ac.ukAbstract. Most research in the area of emotion detection in writtentext focused on detecting explicit expressions of emotions in text. Inthis paper, we present a rule-based pipeline approach for detecting im-plicit emotions in written text without emotion-bearing words based onthe OCC Model. We have evaluated our approach on three differentdatasets with five emotion categories. Our results show that the pro-posed approach outperforms the lexicon matching method consistentlyacross all the three datasets by a large margin of 17-30% in F-measureand gives competitive performance compared to a supervised classifier.In particular, when dealing with formal text which follows grammati-cal rules strictly, our approach gives an average F-measure of 82.7% on“Happy”, “Angry-Disgust” and “Sad”, even outperforming the super-vised baseline by nearly 17% in F-measure. Our preliminary results showthe feasibility of the approach for the task of implicit emotion detectionin written text.Keywords: Implicit emotions, OCC model, emotion detection, rule-based approach1 IntroductionHuman emotions are defined as subjective feelings and thoughts, and is a shortepisode that is coordinated by the brain [4]. Emotions exist in various forms andEkman [2] made a strong compelling case for the six basic emotion categories. InNatural language Processing (NLP), emotion detection focuses on categorisinga piece of text into an emotion category. The expression of emotion in writtentext is through the use of words and most often emotion-bearing words suchas “happy”. However, emotions can be adequately expressed without the use ofemotion-bearing words. For example, given two sentences “The outcome of myexam makes me happy.” and “I passed my exam.” , both sentences express theemotion of happiness, with the first expressing it explicitly and the second imply-ing it. Most research in the area of emotion detection focuses on explicit emotiondetection [9, 6]. Implicit emotion detection is a much more difficult task and theapproaches which rely on emotion lexicons are inapplicable here. Although it ispossible to train supervised classifiers from annotated data, acquiring sufficientannotated data for training requires heavy manual effort.2 Udochukwu and HeWe present in this paper a rule-based approach to sentence-level implicit emo-tion detection based on the OCC model, which was created by Ortony, Clore &Collins in their book “The Cognitive Structure of Emotions” [7] and over theyears has become a widely accepted model for emotion. we demonstrate usingthis resource and with the use of NLP techniques we are able to detect andclassify implicit expressions of emotion in text. As opposed to most existingapproaches, our approach does not rely on any specific knowledge base or anno-tated training data and can still offer a reasonably high precision rate given thecomplex nature of the problem.2 The OCC ModelThe OCC Model provides a clear and convincing structure of the eliciting condi-tions of emotions and the variables that affect their intensities [7]. It describes ahierarchy that classifies 22 emotion types. The hierarchy contains three branches.The first branch is the “consequences of events” branch. The emotions on thisbranch express pleasure or displeasure with event consequences. The secondbranch contains emotions in relation to “actions of agents”. An agent can beself or other and these are the attribution type of emotions. The third branchcontains emotions relating to liking or disliking in regards of “aspects of objects”.The OCC model is dependent on its variables and the rules for implement-ing/identifying emotion. The variables are grouped into emotion-inducing vari-able and emotion intensity variables. For a full list of the emotions and variables,see [7]. The OCC model however is rather complex and full of ambiguity. Ste-unebrink et al. [10] outlined a number of issues of the original OCC model andproposed changes to remove duplications and ambiguities. In our work, we usethe revisited OCC model [10] for emotion detection in text.3 Our ApproachIn order to use the OCC model for emotion detection, we need to first assignvalues to a list of variables defined in OCC and then use a set of pre-defined rulesto identify an emotion for a given text. In this paper, we focus on identifyingemotion in relation to events and actions only and leave the detection of emotionsassociated with objects as future work. The list of rules is shown in Table 1. Forexample, the first row of Table 1(a) can be read asIf Direction = “Self” and Tense = “Future” andOverall Polarity = “Positive”and Event Polarity = “Positive”, then Emotion = “Hope".It is worth noting that emotion-bearing words are different from polarity-bearing words. An emotion-bearing word can be described as words which ontheir own can convey emotions. For example, the word “passionate” can conveyan emotion of Joy. Polarity-bearing words, on the contrary, express positive ornegative polarity in a given context. For example, the word “pass” expressesa positive polarity as in “I passed my exam.”. But the word “pass” does nothave an explicit prior emotion associated with it. Hence, it is more likely thatA Rule-Based Approach to Implicit Emotion Detection in Text 3emotions-bearing words also have a polarity, but not all polarity words conveyspecific emotions.Table 1. Rules for emotion detection.Input Variables OutputDirection Tense Overall polarity Event polarity EmotionSelf Future Positive Positive HopeSelf Future Negative Negative FearSelf Present Positive Positive JoySelf Present Negative Negative DistressSelf Past Positive Positive SatisfactionSelf Past Negative Negative Fears-confirmedSelf Past Positive Negative ReliefSelf Past Negative Positive DisappointmentOther All Positive Positive Happy-forOther All Negative Positive ResentmentOther All Positive Negative GloatingOther All Negative Negative Sorry-for(a) Event-based.Input Variables OutputDirection Polarity EmotionSelf Positive PrideSelf Negative ShameOther Positive AdmirationOther Negative Reproach(b) Action-based.Input Variables OutputEvent Action EmotionJoy Pride GratificationDistress Shame RemorseJoy Admiration GratitudeDistress Reproach Anger(c) Compound emotions.We first perform pre-processing on text in order to be able to assign valuesto the OCC variables which will be discussed subsequently. For pre-processingwe carried out sentence splitting and tokenisation, part-of-speech (POS) tagging,word sense disambiguation (WSD), dependency parsing, sentence tense detectionbased on the POS tags, and polarity detection using majority vote based on thelexicon matching results obtained with three sentiment lexicons, SentiWordNet[3], AFINN [5] and the Subjectivity Lexicon[12].We now describe how we assign values to each OCC variable.Direction: The value for this variable can either be ”Self” or ”Other”. Theformer refers to emotions expressed for oneself while the latter refers to emotionsexpressed for others. This value is assigned based on the dependency relationship(identified by the dependency parser) of a first person pronoun (such as “I” ,“we”) with an action or event. We identify 3 possible scenarios for assigninga value to this variable: 1) When dealing with a simple sentence, we simplyapply the process mentioned above; 2) When dealing with a complex sentencewhere multiple subject(s) are identified by the parser, we assign values basedon respective action/event relations with identified subjects; 3) No subject isidentified or no verbs exist in the text, here we just assign the value ”Other” tothe variable.Tense: The value for this variable can either be ”Present”, ”Past” or ”Future”.The value assignment is determined by the POS tags of the verbs in a sentence4 Udochukwu and Heor by the results obtained from the FrameNet1 where the tense of the verbis identified as future tense if the frame is associated with “desiring”. In thecases where no verbs are used in a sentence, the value of the variable is set to”Present”.Overall Sentence Polarity: The value for this variable can either be ”Neutral”,”Negative” or ”Positive”. It is determined by polarity detection through ma-jority vote.Event Polarity: The event is identified based on the verb-object relations re-vealed by the dependency parser. The noun phrase which contains an identifiedobject is treated as the event for its relative verb. The polarity of an event isthen determined using lexicon-matching.Action Polarity: The action is identified based on the subject-verb relationsrevealed by the dependency parser. The verb phrase which contains the identifiedverb is treated as an action. Similar to event polarity, the action polarity is alsodetermined using lexicon-matching.We have also implemented a contextual valence shifter as described in [8] todetect polarity change in different context. Once the variable values are identi-fied, the rules defined in Table 1 are then applied to detect the presence of emo-tions. The compound emotions are results of the output of the event-based andaction-based emotions. For the ”sorry-for” emotion, we ensure that the sub-ject is of positive valence; otherwise the emotion is identified as “resentment”.The same rule is applied to the “admiration” and “reproach” emotion pairs.4 ExperimentsIn this section, we present the evaluation results of our rule-based emotion de-tection approach on three different datasets, which include:– The International Survey On Emotion Antecedents And Reactions (ISEAR)Dataset2 which was developed by asking nearly 3,000 participants from dif-ferent cultural background about their emotional experiences.– The SemEval-2007 Task 14 Affective Text dataset [11] consists of news head-lines collected from major newspapers.– The Alm’s Dataset [1] comprises sentences taken from 176 fairy tale stories.We use only the data extracted from Grimm’s and Anderson’s tales, whichhave a total number of 1,040 sentences.As our goal is to detect emotions in the absence of emotion-bearing words, wefilter out sentences which contain emotion words as can be found in the emotionlexicon WordNet-Affect3. The total number of sentences before and after filteringof emotion-bearing words in each emotion category for these three datasets areshown in Table 2. We focus specifically on the 5 emotion categories which are1 https://framenet.icsi.berkeley.edu/fndrupal/2 http://www.unige.ch/fapse/emotion/databanks/isear.html3 http://wndomains.fbk.eu/wnaffect.htmlA Rule-Based Approach to Implicit Emotion Detection in Text 5shared across these three datasets and map the OCC-output emotions to thefive emotion categories in the following ways: (Fear, Fear-confirmed) → Fear,(Joy, Happy-For, Satisfaction, Admiration, Pride) → Joy, (Anger, Reproach) →Anger, (Distress, Sorry-For, Disappointment, Shame) → Sadness, Resentment→ Disgust.Table 2. Statistics of the datasets. “Total” denotes the original number of sentencesin each emotion category while “Implicit” denote the number of sentence which do nocontain any emotion words according to WordNet-Affect.Emotion Total ImplicitJoy 1095 537Fear 1095 366Anger 1096 483Sadness 1096 488Disgust 1096 484Shame 1096 581Guilt 1093 482Total 7667 3421(a) ISEAREmotion Total ImplicitJoy 362 317Fear 160 130Anger 66 60Sadness 202 182Disgust 26 24Surprise 184 160Total 1000 873(b) SemEvalEmotion Total ImplicitHappy 406 103Fearful 121 33Angry-Disgusted 174 84Sad 247 90Surprised 92 50Total 1040 360(c) Alm’sWe have developed two baseline models. One is a lexicon matching methodwhich uses the NRC emotion Lexicon4 for sentence-level emotion detection, Wealso train supervised Na¨ıve Bayes (NB) classifiers using the implementation inWeka5 on the three datasets with 5-fold cross validation.We report the resultsin terms of the F-measure scores.Table 3. Performance comparison of F-measure results on the three datasets. Boldface values denote the best results obtained in each dataset.EmotionISEAR SemEval Alm’sLexicon NB Rule Lexicon NB Rule Lexicon NB RuleJoy/Happy 33.4 61.2 69.6 39.7 71.7 59.9 58.8 63.5 81.8Fear/Fearful 0 47.6 18.3 0 52.2 31.8 0 26.7 14.0Anger/Angry-Disgusted 23.0 47.1 61.3 55.8 16.2 61.3 48.9 58.6 86.6Sadness/Sad 25.6 55.4 68.0 47.8 56.0 71.5 61.0 56.0 79.6Disgust 25.6 51.0 39.2 38.5 34.5 61.7 - - -Average 21.5 52.5 51.3 36.4 58.2 57.3 42.2 56.0 65.5Average (− Fear) 27.0 53.7 59.5 45.5 44.6 63.6 56.12 65.8 82.7It can be observed from Table 3 that although we have filtered out sentenceswhich contain emotion words from WordNet-Affect, using other emotion lexicons4 http://www.saifmohammad.com/WebPages/lexicons.html5 http://www.cs.waikato.ac.nz/ml/weka6 Udochukwu and Hesuch as the NRC emotion lexicon can still identify emotions of some sentences.Nevertheless, using the NRC lexicon gives quite low F-measure values acrossthe three datasets and it fails to detect any “Fear” emotion bearing sentences.Despite using no labelled data, our approach achieves similar performance assupervised NB on the ISEAR and SemEval datasets (with 1% difference in F-measure on average) and outperforms NB by 9.5% in F-measure on the Alm’sdataset. The results also show that our approach is largely affected by the qual-ity of text. ISEAR contains personal experience expressed by a wide range ofparticipants and hence might contain lots of informal and ill-grammatical text.SemEval contains news headlines which are often incomplete sentences ignor-ing grammar conventions. The Alm’s dataset, on the other hand, contains fairytales which are formal text following rules of grammar very strictly. As such, theperformance obtained from the Alm’s dataset by our approach are significantbetter than that obtained from the other two datasets.Our approach relies on results generated from a series of NLP tasks such asPOS tagging,word-sense disambiguation, dependency parsing and polarity de-tection in order to be able to assign values to a set of OCC variables for emotiondetection. Thus, any error occurred will be propagated down the pipeline pro-cess. Furthermore, failure in detecting the polarity of text will make it impossiblefor our approach to identify the underlying emotion. Also, we have not consid-ered ironic and sarcastic sentences in our current work. Nevertheless, we haveshown that in the absence of annotated data, the OCC-based approach is ableto identify implicit emotion in text with performance competing to supervisedclassifiers and it even outperforms the supervised approach for formal text (theAlm’s dataset). The emotion detection results generated by the OCC-based ap-proach can be used as seed examples to bootstrap more complicated emotiondetection methods which require large amount of training data. We will leave itas our future work.5 Conclusions and Future WorkIn this paper, we have proposed a OCC-based approach for implicit emotiondetection. Experimental results on three datasets have shown that our approachoutperforms the lexicon matching method consistently across all the three datasetsby a large margin of 17-30% in F-measure and gives competitive performancecompared to a supervised method. Also, when dealing with formal text whichfollows grammatical rules strictly, the approach achieves an average F-measurescore of 82.7% in identifying “Happy”, “Angry-Disgusted” and “Sad” categories.In future, we will investigate methods to improve the performance of ourapproach with informal short text such as tweets and social media posts. Wewill also improve the identification of emotions involving intensity variables andunexpectedness variables by examining how adverbs and adjectives influence theemotion of sentences (for emotions like “Surprise” and “Shock”) and investigateperformance on ironic or sarcastic sentences. We will study possible solutions todeal with the poor performing emotion category such as “Fear”.A Rule-Based Approach to Implicit Emotion Detection in Text 7References1. Alm, C.O., Roth, D., Sproat, R.: Emotions from text: machine learning for text-based emotion prediction. In: Proceedings of the conference on Human LanguageTechnology and Empirical Methods in Natural Language Processing. pp. 579–586(2005)2. Ekman, P.: An argument for basic emotions. Cognition & Emotion 6(3-4), 169–200(1992)3. Esuli, A., Sebastiani, F.: Determining the semantic orientation of terms throughgloss classification. In: Proceedings of the 14th ACM international conference onInformation and knowledge management. pp. 617–624 (2005)4. Friedenberg, J., Silverman, G.: Cognitive science: an introduction to the study ofmind. Sage (2011)5. Hansen, L.K., Arvidsson, A., Nielsen, F.A˚., Colleoni, E., Etter, M.: Good friends,bad news-affect and virality in twitter. In: Future information technology, pp. 34–43. Springer (2011)6. Marsella, S., Gratch, J., Petta, P.: Computational models of emotion. A Blueprintfor Affective Computing-A sourcebook and manual pp. 21–46 (2010)7. Ortony, A.: The cognitive structure of emotions. Cambridge university press (1990)8. Polanyi, L., Zaenen, A.: Contextual valence shifters. In: Computing attitude andaffect in text: Theory and applications, pp. 1–10. Springer (2006)9. Shivhare, S.N., Khethawat, S.: Emotion detection from text. arXiv preprintarXiv:1205.4944 (2012)10. Steunebrink, B.R., Dastani, M., Meyer, J.J.C.: The occ model revisited. In: Pro-ceedings of the 4th Workshop on Emotion and Computing. vol. 65, pp. 2047–2056(2009)11. Strapparava, C., Mihalcea, R.: Semeval-2007 task 14: Affective text. In: Proceed-ings of the 4th International Workshop on Semantic Evaluations. pp. 70–74. Se-mEval ’07 (2007)12. Wilson, T., Wiebe, J., Hoffmann, P.: Recognizing contextual polarity in phrase-level sentiment analysis. In: Proceedings of the conference on human languagetechnology and empirical methods in natural language processing. pp. 347–354.Association for Computational Linguistics (2005)

A rule-based approach to implicit emotion detection in text

https://publications.aston.ac.uk/id/eprint/27397/1/Implicit_emotion_detection_in_text.pdf

A rule-based approach to implicit emotion detection in text

Abstract

Similar works

Full text

Available Versions

Crossref

Aston Publications Explorer