Comparing the utility of different classification schemes for emotive language analysis

In this paper we investigated the utility of different classification schemes for emotive language analysis with the aim of providing experimental justification for the choice of scheme for classifying emotions in free text. We compared six schemes: (1) Ekman's six basic emotions, (2) Plutchik's wheel of emotion, (3) Watson and Tellegen's Circumplex theory of affect, (4) the Emotion Annotation Representation Language (EARL), (5) WordNet–Affect, and (6) free text. To measure their utility, we investigated their ease of use by human annotators as well as the performance of supervised machine learning. We assembled a corpus of 500 emotionally charged text documents. The corpus was annotated manually using an online crowdsourcing platform with five independent annotators per document. Assuming that classification schemes with a better balance between completeness and complexity are easier to interpret and use, we expect such schemes to be associated with higher inter–annotator agreement. We used Krippendorff's alpha coefficient to measure inter–annotator agreement according to which the six classification schemes were ranked as follows: (1) six basic emotions (a = 0.483), (2) wheel of emotion (a = 0.410), (3) Circumplex (a = 0.312), EARL (a = 0.286), (5) free text (a = 0.205), and (6) WordNet–Affect (a = 0.202). However, correspondence analysis of annotations across the schemes highlighted that basic emotions are oversimplified representations of complex phenomena and as such likely to lead to invalid interpretations, which are not necessarily reflected by high inter-annotator agreement. To complement the result of the quantitative analysis, we used semi–structured interviews to gain a qualitative insight into how annotators interacted with and interpreted the chosen schemes. The size of the classification scheme was highlighted as a significant factor affecting annotation. In particular, the scheme of six basic emotions was perceived as having insufficient coverage of the emotion space forcing annotators to often resort to inferior alternatives, e.g. using happiness as a surrogate for love. On the opposite end of the spectrum, large schemes such as WordNet–Affect were linked to choice fatigue, which incurred significant cognitive effort in choosing the best annotation. In the second part of the study, we used the annotated corpus to create six training datasets, one for each scheme. The training data were used in cross–validation experiments to evaluate classification performance in relation to different schemes. According to the F-measure, the classification schemes were ranked as follows: (1) six basic emotions (F = 0.410), (2) Circumplex (F = 0.341), (3) wheel of emotion (F = 0.293), (4) EARL (F = 0.254), (5) free text (F = 0.159) and (6) WordNet–Affect (F = 0.158). Not surprisingly, the smallest scheme was ranked the highest in both criteria. Therefore, out of the six schemes studied here, six basic emotions are best suited for emotive language analysis. However, both quantitative and qualitative analysis highlighted its major shortcoming – oversimplification of positive emotions, which are all conflated into happiness. Further investigation is needed into ways of better balancing positive and negative emotions. Keywords: annotation, crowdsourcing, text classification, sentiment analysis, supervised machine learnin

Consumer trust and willingness to pay for certified animal-friendly products

Increasing animal welfare standards requires changes along the supply chain which involve several stakeholders: scientists, farmers and people involved in transportation and slaughtering. The majority of researchers agree that compliance with these standards increases costs along the livestock value chain, especially for monitoring and certifying animal-friendly products. Knowledge of consumer willingness to pay (WTP) in such a decision context is paramount to understanding the magnitude of market incentives necessary to compensate all involved stakeholders. The market outcome of certification programs is dependent on consumer trust. Particularly, there is a need to understand to what extent consumers believe that stakeholders operating in the animal-friendly supply chain will respect certification standards. We examine these issues using a contingent valuation survey administered in five economically dominant EU countries. The implied WTP estimates are found to be sensitive to robust measures of consumer trust for certified animal-friendly products. Significant differences across countries are discussed

Comparing hierarchical approaches to enhance supervised emotive text classification

The performance of emotive text classification using affective hierarchical schemes (e.g. WordNet-Affect) is often evaluated using the same traditional measures used to evaluate the performance of when a finite set of isolated classes are used. However, applying such measures means the full characteristics and structure of the emotive hierarchical scheme are not considered. Thus, the overall performance of emotive text classification using emotion hierarchical schemes is often inaccurately reported and may lead to ineffective information retrieval and decision making. This paper provides a comparative investigation into how methods used in hierarchical classification problems in other domains, which extend traditional evaluation metrics to consider the characteristics of the hierarchical classification scheme can be applied and subsequently improve the classification of emotive texts. This study investigates the classification performance of three widely used classifiers, Naive Bayes, J48 Decision Tree, and SVM, following the application of the aforementioned methods. The results demonstrated that all methods improved the emotion classification. However, the most notable improvement was recorded when a depth-based method was applied to both the testing and validation data, where the precision, recall, and F1-score were significantly improved by around 70 percentage points for each classifier

Investigating and extending the methods in automated opinion analysis through improvements in phrase based analysis

Opinion analysis is an area of research which deals with the computational treatment of opinion statement and subjectivity in textual data. Opinion analysis has emerged over the past couple of decades as an active area of research, as it provides solutions to the issues raised by information overload. The problem of information overload has emerged with the advancements in communication technologies which gave rise to an exponential growth in user generated subjective data available online. Opinion analysis has a rich set of applications which are used to enable opportunities for organisations such as tracking user opinions about products, social issues in communities through to engagement in political participation etc.The opinion analysis area shows hyperactivity in recent years and research at different levels of granularity has, and is being undertaken. However it is observed that there are limitations in the state-of-the-art, especially as dealing with the level of granularities on their own does not solve current research issues. Therefore a novel sentence level opinion analysis approach utilising clause and phrase level analysis is proposed. This approach uses linguistic and syntactic analysis of sentences to understand the interdependence of words within sentences, and further uses rule based analysis for phrase level analysis to calculate the opinion at each hierarchical structure of a sentence. The proposed opinion analysis approach requires lexical and contextual resources for implementation. In the context of this Thesis the approach is further presented as part of an extended unifying framework for opinion analysis resulting in the design and construction of a novel corpus. The above contributions to the field (approach, framework and corpus) are evaluated within the Thesis and are found to make improvements on existing limitations in the field, particularly with regards to opinion analysis automation. Further work is required in integrating a mechanism for greater word sense disambiguation and in lexical resource development

Functional Text Dimensions for the annotation of web corpora

This paper presents an approach to classifying large web corpora into genres by means of Functional Text Dimensions (FTDs). This offers a topological approach to text typology in which the texts are described in terms of their similarity to prototype genres. The suggested set of categories is designed to be applicable to any text on the web and to be reliable in annotation practice. Interannotator agreement results show that the suggested categories produce Krippendorff's α at above 0.76. In addition to the functional space of eighteen dimensions, similarity between annotated documents can be described visually within a space of reduced dimensions obtained through t-distributed Statistical Neighbour Embedding. Reliably annotated texts also provide the basis for automatic genre classification, which can be done in each FTD, as well as as within the space of reduced dimensions. An example comparing texts from the Brown Corpus, the BNC and ukWac, a large web corpus, is provided

Rearticulating the case for minority language rights

While advocacy of minority language rights (MLR) has become well established in sociolinguistics, language policy and planning and the wider human rights literature, it has also come under increased criticism in recent times for a number of key limitations. In this paper, I address directly three current key criticisms of the MLR movement. The first is a perceived tendency towards essentialism in articulations of language rights. The second is the apparent utopianism and artificiality of 'reversing language shift' in the face of wider social and political 'realities'. And the third is that the individual mobility of minority-language speakers is far better served by shifting to a majority language. While acknowledging the perspicacity of some of these arguments, I aim to rearticulate a defence of minority language rights that effectively addresses these key concerns. This requires, however, a sociohistorical/sociopolitical rather than a biological/ecological analysis of MLR. In addition, I will argue that a sociohistorical/sociopolitical defence of MLR can problematise the positions often adopted by minority language rights' critics themselves, particularly those who defend majoritarian forms of linguistic essentialism and those who sever the instrumental/identity aspects of language. Implications for language policy and planning will also be discussed

FINE-GRAINED EMOTION DETECTION IN MICROBLOG TEXT

Automatic emotion detection in text is concerned with using natural language processing techniques to recognize emotions expressed in written discourse. Endowing computers with the ability to recognize emotions in a particular kind of text, microblogs, has important applications in sentiment analysis and affective computing. In order to build computational models that can recognize the emotions represented in tweets we need to identify a set of suitable emotion categories. Prior work has mainly focused on building computational models for only a small set of six basic emotions (happiness, sadness, fear, anger, disgust, and surprise). This thesis describes a taxonomy of 28 emotion categories, an expansion of these six basic emotions, developed inductively from data. This set of 28 emotion categories represents a set of fine-grained emotion categories that are representative of the range of emotions expressed in tweets, microblog posts on Twitter. The ability of humans to recognize these fine-grained emotion categories is characterized using inter-annotator reliability measures based on annotations provided by expert and novice annotators. A set of 15,553 human-annotated tweets form a gold standard corpus, EmoTweet-28. For each emotion category, we have extracted a set of linguistic cues (i.e., punctuation marks, emoticons, emojis, abbreviated forms, interjections, lemmas, hashtags and collocations) that can serve as salient indicators for that emotion category. We evaluated the performance of automatic classification techniques on the set of 28 emotion categories through a series of experiments using several classifier and feature combinations. Our results shows that it is feasible to extend machine learning classification to fine-grained emotion detection in tweets (i.e., as many as 28 emotion categories) with results that are comparable to state-of-the-art classifiers that detect six to eight basic emotions in text. Classifiers using features extracted from the linguistic cues associated with each category equal or better the performance of conventional corpus-based and lexicon-based features for fine-grained emotion classification. This thesis makes an important theoretical contribution in the development of a taxonomy of emotion in text. In addition, this research also makes several practical contributions, particularly in the creation of language resources (i.e., corpus and lexicon) and machine learning models for fine-grained emotion detection in text

Preaching in the \u27Hear\u27 and Now: Justification, Development, and Assessment of \u27Parabolic Engagement\u27 Pedogogy in French-Speaking Missionary Settings

This thesis argues the utility of ‘parabolic engagement’ method for preachers and listeners in the French Antillean context. The opening chapter defines key terms and clarifies how this imaged sermonic style addresses the listening habits of targeted audiences. It explains that figured delivery is often context-interpretive, involving a more personal, experiential decoding by the listener. Engagement technique increases auditor involvement and creates unique communicative rapport. The chapter points out that the entire experimental process validates the usefulness of the pedagogy. Part One addresses the theological rationale for ‘parabolic engagement’ method. Chapter Two reviews appropriate literature with respect to engagement. Chapter Three argues the biblical basis for creating a method of figured preaching. Chapter Four discusses how precise homiletic situations demand a circumstantial approach to engaging delivery. Part Two attempts to synthesize a broad range of image-creation methodologies and make them suitable for teaching among oral peoples. Chapter Five shows the necessity of a grammar for figured proclamation pedagogy. Chapter Six develops simplified classical methods for finding the illustrative crux of an idea or text. Chapter Seven shows the need to then engage the listener by means of analogous correspondence with the concrete world. Chapter Eight explores how circumstantial factors encourage the transformation of engaging analogies into extended narratives. Part Three validates the thesis within the missionary setting. Chapter Nine describes the suitability of ‘parabolic engagement’ method among Creoles and European French on the island of Martinique. Chapter Ten establishes an experimental design by specifying components, clarifying how the hypotheses were tested, justifying data collection methods, and explaining the use of participatory action research and educational ethnography. Chapter Eleven details the implementation, measurement, and success of engagement strategies. Lastly, Chapter Twelve argues for the utility of ‘parabolic engagement’ and posits generalizations by summarizing the merits, conclusions, and limitations of the model