Analysis of Affect Expressed through the Evolving Language of Online Communication

In this paper, we focus on affect recognition from text in order to facilitate sensitive and expressive communication in computer-mediated environments. Our model for analyzing affect conveyed by text is tailored to handle the style and specifics of informal online conversations. The motivation behind our approach is to improve social interactivity and emotional expressiveness of real-time messaging. In order to estimate affect in text, our model processes symbolic cues, such as emoticons, detects and transforms abbreviations, and employs natural language processing techniques for word/phrase/sentence-level analysis, e.g. by considering relations among words in a sentence. As a result of the analysis, text can be categorized into emotional states and communicative functions. A designed graphical representation of a user (avatar) displays emotions and social behaviour driven by text and performs natural idle movements. The proposed system shows promising results on affect recognition in real examples of online conversation. ACM Classification: H5.2 [Information interfaces and presentation]: User Interfaces.- Graphical user interfaces (GUI), interaction styles, natural language, theory and methods; I2.

AffectIM : An Avatar-based Instant Messaging System Employing Rule-based Affect Sensing from Text

Social interactions among people play an important role in the establishment of genuine interpersonal relationships and communities. We convey information through multiple expressive channels, such as natural language, intonation, gaze, facial expressions, gestures, and body language. Recently, computer-mediated communication became a popular way of interaction, especially among young people. However, it lacks the mentioned signals of face-to-face communication. In our work, we concentrate on affect recognition from text in Instant Messaging (IM) to automate expressive channels and to improve social interactivity of this media type. To analyse affect communicated through written language, researchers in the area of natural language processing proposed a variety of approaches, methodologies and techniques. However, the weakness of most affect recognition systems integrated with chat or e-mail browsers is, that they do not take into consideration crucial aspects of informal online conversation, such as evolving language or linguistic and interactional features. IM users are continuously developing their own language, the motivations behind which are speed and less typing. The understanding of this evolving language by people and syntactical parsers is the main problem in messaging. In order to facilitate sensitive and expressive communication in computer-mediated environments, we introduced a novel syntactical rule-based approach to affect recognition from text. The developed Affect Analysis Model was designed to handle not only grammatically and syntactically correct textual input, but also informal messages written in abbreviated or expressive manner. In contrast to machine-learning based approaches, for which detection of emotions on a sentence level still remains a challenge, the proposed rule-based approach processes each sentence in sequential stages, including symbolic cue processing, detection and transformation of abbreviations, sentence parsing, and word/phrase/sentence-level analyses. Since the purpose of affect recognition in an IM system is to relate text to avatar emotional expressions, affect categories were confined to those that can be visually expressed and easily understood by users: ‘anger’, ‘disgust’, ‘fear’, ‘guilt’, ‘interest’, ‘joy’, ‘sadness’, ‘shame’, and ‘surprise’ Additionally, the information related to communicative behaviour (e.g. ‘greeting’, ‘thanks’, ‘posing a question’, ‘congratulation’, and ‘farewell’), directly dependent on context and carrying significant communicative power, can be derived from online conversations. To support the handling of abbreviated language and the interpretation of affective features of linguistic concepts, a special Affect database, containing emoticons and abbreviations, interjections, modifiers, direct and indirect emotion-related words (adjectives, adverbs, nouns, and verbs), and words standing for communicative functions, was created. For accumulation of relevant and most often used emoticons and abbreviations, we employed five online dictionaries dedicated to and describing such data. Words conveying affective content directly or indirectly were taken from the source of affective lexicon, WordNet-Affect. Each database entry was annotated, depending on its role, with the emotion category with intensity, or communicative function category, or modifier coefficient. The main advantage of the proposed rule-based methodology is its flexibility allowing to handle the evolving language of online communications; to represent the affective features of words, phrases, clauses and sentences as a vector; to cover such linguistic aspects as negation, modality, and conditionality; to consider syntactic relations and dependences between words in a sentence, or between clauses in compound, complex, or complex-compound sentences; and to introduce new processing rules to the developed Affect Analysis Model. While affect sensing, each analyzed sentence is automatically annotated with emotion (or neutral) label, and numerical value, which indicates the degree of emotion intensity. Furthermore, the information related to communicative behaviour is identified. The evaluation of the Affect Analysis Model algorithm showed promising results regarding its capability to recognize affective information in text from an existing corpus of informal online communication. In a study based on 160 sentences, the system result agreed with at least two out of three human annotators in 70% of the cases. In order to enrich the user’s experience in online communication, make it enjoyable, exciting and fun, we realized a web-based IM application, AffectIM, and endowed it with the emotional intelligence by integrating with the developed Affect Analysis Model. AffectIM supports online communication, allows users to see the conversation flow in three modes (plain text, transcribed text, or text annotated with emotion), and visualizes the communicated emotions, emotion distribution and emotion dynamics. To support visual reflection of sensed affective information, we have designed and animated two 2D cartoon-like avatars (graphical representations of a user) performing various expressive patterns (emotions, social behavior, and natural idle movements), contributing thus to greater interactivity. During our 20-user experiment with three interfaces of AffectIM system (automatic, manual, and random conditions) we got many valuable results and feedbacks, and investigated new ways of improvements. The data obtained showed that the developed emotion recognition engine worked with good level of reliability, so that there was no significant difference between system providing automatic emotion sensing from text and system with manual control of emotional behavior of avatars (so called “gold standard”). It is evident that AffectIM would benefit from integration of both these functions in one interface, where they can complement each other and provide user with the ability to select between two modes (automatic or manual control of emotion expressions) depending on type and sensitivity of conversation.報告番号: ; 学位授与年月日: 2008-03-24 ; 学位の種別: 修士 ; 学位の種類: 修士(情報理工学) ; 学位記番号: ; 研究科・専攻: 情報理工学系研究科電子情報学専

AffectIM : テキストからのルールに基づく感情抽出を用いるアバタ付きインスタントメセージング・システム

Social interactions among people play an important role in the establishment of genuine interpersonal relationships and communities. We convey information through multiple expressive channels, such as natural language, intonation, gaze, facial expressions, gestures, and body language. Recently, computer-mediated communication became a popular way of interaction, especially among young people. However, it lacks the mentioned signals of face-to-face communication. In our work, we concentrate on affect recognition from text in Instant Messaging (IM) to automate expressive channels and to improve social interactivity of this media type. To analyse affect communicated through written language, researchers in the area of natural language processing proposed a variety of approaches, methodologies and techniques. However, the weakness of most affect recognition systems integrated with chat or e-mail browsers is, that they do not take into consideration crucial aspects of informal online conversation, such as evolving language or linguistic and interactional features. IM users are continuously developing their own language, the motivations behind which are speed and less typing. The understanding of this evolving language by people and syntactical parsers is the main problem in messaging. In order to facilitate sensitive and expressive communication in computer-mediated environments, we introduced a novel syntactical rule-based approach to affect recognition from text. The developed Affect Analysis Model was designed to handle not only grammatically and syntactically correct textual input, but also informal messages written in abbreviated or expressive manner. In contrast to machine-learning based approaches, for which detection of emotions on a sentence level still remains a challenge, the proposed rule-based approach processes each sentence in sequential stages, including symbolic cue processing, detection and transformation of abbreviations, sentence parsing, and word/phrase/sentence-level analyses. Since the purpose of affect recognition in an IM system is to relate text to avatar emotional expressions, affect categories were confined to those that can be visually expressed and easily understood by users: ‘anger’, ‘disgust’, ‘fear’, ‘guilt’, ‘interest’, ‘joy’, ‘sadness’, ‘shame’, and ‘surprise’ Additionally, the information related to communicative behaviour (e.g. ‘greeting’, ‘thanks’, ‘posing a question’, ‘congratulation’, and ‘farewell’), directly dependent on context and carrying significant communicative power, can be derived from online conversations. To support the handling of abbreviated language and the interpretation of affective features of linguistic concepts, a special Affect database, containing emoticons and abbreviations, interjections, modifiers, direct and indirect emotion-related words (adjectives, adverbs, nouns, and verbs), and words standing for communicative functions, was created. For accumulation of relevant and most often used emoticons and abbreviations, we employed five online dictionaries dedicated to and describing such data. Words conveying affective content directly or indirectly were taken from the source of affective lexicon, WordNet-Affect. Each database entry was annotated, depending on its role, with the emotion category with intensity, or communicative function category, or modifier coefficient. The main advantage of the proposed rule-based methodology is its flexibility allowing to handle the evolving language of online communications; to represent the affective features of words, phrases, clauses and sentences as a vector; to cover such linguistic aspects as negation, modality, and conditionality; to consider syntactic relations and dependences between words in a sentence, or between clauses in compound, complex, or complex-compound sentences; and to introduce new processing rules to the developed Affect Analysis Model. While affect sensing, each analyzed sentence is automatically annotated with emotion (or neutral) label, and numerical value, which indicates the degree of emotion intensity. Furthermore, the information related to communicative behaviour is identified. The evaluation of the Affect Analysis Model algorithm showed promising results regarding its capability to recognize affective information in text from an existing corpus of informal online communication. In a study based on 160 sentences, the system result agreed with at least two out of three human annotators in 70% of the cases. In order to enrich the user’s experience in online communication, make it enjoyable, exciting and fun, we realized a web-based IM application, AffectIM, and endowed it with the emotional intelligence by integrating with the developed Affect Analysis Model. AffectIM supports online communication, allows users to see the conversation flow in three modes (plain text, transcribed text, or text annotated with emotion), and visualizes the communicated emotions, emotion distribution and emotion dynamics. To support visual reflection of sensed affective information, we have designed and animated two 2D cartoon-like avatars (graphical representations of a user) performing various expressive patterns (emotions, social behavior, and natural idle movements), contributing thus to greater interactivity. During our 20-user experiment with three interfaces of AffectIM system (automatic, manual, and random conditions) we got many valuable results and feedbacks, and investigated new ways of improvements. The data obtained showed that the developed emotion recognition engine worked with good level of reliability, so that there was no significant difference between system providing automatic emotion sensing from text and system with manual control of emotional behavior of avatars (so called “gold standard”). It is evident that AffectIM would benefit from integration of both these functions in one interface, where they can complement each other and provide user with the ability to select between two modes (automatic or manual control of emotion expressions) depending on type and sensitivity of conversation

Compositionality Principle in Recognition of Fine-Grained Emotions from Text

The recognition of personal emotional state or sentiment conveyed through text is the main task we address in our research. The communication of emotions through text messaging and posts of personal blogs poses the ‘informal style of writing ’ challenge for researchers expecting grammatically correct input. Our Affect Analysis Model was designed to handle the informal messages written in an abbreviated or expressive manner. While constructing our rule-based approach to affect recognition from text, we followed the compositionality principle. Our method is capable of processing sentences of different complexity, including simple, compound, complex (with complement and relative clauses), and complex-compound sentences. The evaluation of the Affect Analysis Model algorithm showed promising results regarding its capability to accurately recognize affective information in text from an existing corpus of personal blog posts