IOWA & Cross-ratio Uninorm operators as aggregation tools in sentiment analysis and ensemble methods

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.In the field of Sentiment Analysis, a number of different classifiers are utilised to attempt to establish the polarity of a given sentence. As such, there could be a need for aggregating the outputs of the algorithms involved in the classification effort. If the output of every classification algorithm resembles the opinion of an expert in the subject at hand, we are then in the presence of a group decision making problem, which in turn translates into two sub-problems: (a) defining the desired semantic of the aggregation of all opinions, and (b) applying the proper aggregation technique that can achieve the desired semantic chosen in (a). The objective of this article is twofold. Firstly, we present two specific aggregation semantics, namely fuzzy-majority and compensatory, which are based on Induced Ordered Weighted Averaging and Uninorm operators, respectively. Secondly, we show the power of these two techniques by applying them to an existing hybrid method for classification of sentiments at the sentence level. In this case, the proposed aggregation solutions act as a complement in order to improve the performance of the aforementioned hybrid method. In more general terms, the proposed solutions could be used in the creation of semantic-sensitive ensemble methods, instead of the more simple ensemble choices available today in commercial machine learning software offerings

Cross-ratio uninorms as an effective aggregation mechanism in sentiment analysis

There are situations in which lexicon-based methods for Sentiment Analysis (SA) are not able to generate a classification output for specific instances of a given dataset. Most often, the reason for this situation is the absence of specific terms in the sentiment lexicon required in the classification effort. In such cases, there were only two possible paths to follow: (1) add terms to the lexicon (off-line process) by human intervention to guarantee no noise is introduced into the lexicon, which prevents the classification system to provide an immediate answer; or (2) use the services of a word-frequency dictionary (on-line process), which is computationally costly to build. This paper investigates an alternative approach to compensate for the lack of ability of a lexicon-based method to produce a classification output. The method is based on the combination of the classification outputs of non lexicon-based tools. Specifically, firstly the outcome values of applying two or more non-lexicon classification methods are obtained. Secondly, these non-lexicon outcomes are fused using a uninorm based approach, which has been proved to have desirable compensation properties as required in the SA context, to generate the classification output the lexicon based approach is unable to achieve. Experimental results based on the execution of two well-known supervised machine learning algorithms, namely Naïve Bayes and Maximum Entropy, and the application of a cross-ratio uninorm operator are presented. Performance indices associated to options (1) and (2) above are compared against the results obtained using the proposed approach for two different datasets. Additionally, the performance of the proposed cross-ratio uninorm operator based approach is also compared when the aggregation operator used is the arithmetic mean instead. It is shown that the combination of non lexicon-based classification methods with specific uninorm operators improves the classification performance of lexicon-based methods, and it enables the offering of an alternative solution to the SA classification problem when needed. The proposed aggregation method could be used as well as a replacement of ensemble averaging techniques commonly applied when combining the results of several machine learning classifiers’ outputs

Learning Aggregation Functions

Learning on sets is increasingly gaining attention in the machine learning community, due to its widespread applicability. Typically, representations over sets are computed by using fixed aggregation functions such as sum or maximum. However, recent results showed that universal function representation by sum- (or max-) decomposition requires either highly discontinuous (and thus poorly learnable) mappings, or a latent dimension equal to the maximum number of elements in the set. To mitigate this problem, we introduce a learnable aggregation function (LAF) for sets of arbitrary cardinality. LAF can approximate several extensively used aggregators (such as average, sum, maximum) as well as more complex functions (e.g., variance and skewness). We report experiments on semi-synthetic and real data showing that LAF outperforms state-of-the-art sum- (max-) decomposition architectures such as DeepSets and library-based architectures like Principal Neighborhood Aggregation, and can be effectively combined with attention-based architectures.Comment: Extended version (with proof appendix) of paper that is to appear in Proceedings of IJCAI 202

Extraction of similarity based fuzzy rules from artificial neural networks

A method to extract a fuzzy rule based system from a trained artificial neural network for classification is presented. The fuzzy system obtained is equivalent to the corresponding neural network. In the antecedents of the fuzzy rules, it uses the similarity between the input datum and the weight vectors. This implies rules highly understandable. Thus, both the fuzzy system and a simple analysis of the weight vectors are enough to discern the hidden knowledge learnt by the neural network. Several classification problems are presented to illustrate this method of knowledge discovery by using artificial neural networks

UNFIS: A Novel Neuro-Fuzzy Inference System with Unstructured Fuzzy Rules for Classification

An important constraint of Fuzzy Inference Systems (FIS) is their structured rules defined based on evaluating all input variables. Indeed, the length of all fuzzy rules and the number of input variables are equal. However, in many decision-making problems evaluating some conditions on a limited set of input variables is sufficient to decide properly (unstructured rules). Therefore, this constraint limits the performance, generalization, and interpretability of the FIS. To address this issue, this paper presents a neuro-fuzzy inference system for classification applications that can select different sets of input variables for constructing each fuzzy rule. To realize this capability, a new fuzzy selector neuron with an adaptive parameter is proposed that can select input variables in the antecedent part of each fuzzy rule. Moreover, in this paper, the consequent part of the Takagi-Sugeno-Kang FIS is also changed properly to consider only the selected set of input variables. To learn the parameters of the proposed architecture, a trust-region-based learning method (General quasi-Levenberg-Marquardt (GqLM)) is proposed to minimize cross-entropy in multiclass problems. The performance of the proposed method is compared with some related previous approaches in some real-world classification problems. Based on these comparisons the proposed method has better or very close performance with a parsimonious structure consisting of unstructured fuzzy

Fuzzy Aggregators - an Overview

The article deals with mathematical formalism of the process of combining several inputs into a single output in fuzzy inteligent systems, the process known as aggregation. We are interested in logic aggregation operators. Such aggregators are present in most decision problems and in fuzzy expert systems. Fuzzy inteligent systems are equipped with aggregation operators (aggregators) with which reasoning models adapt well to human reasoning. A brief overview of the field of fuzzy aggregators is given. Attention is devoted to so called graded logic aggregators.. The role of fuzzy agregators in modelling reasoning and the way they are chosen in modelling are pointed out. The conclusions are given and research in the field is pointed out

Using Choquet integrals for kNN approximation and classification

k-nearest neighbors (kNN) is a popular method for function approximation and classification. One drawback of this method is that the nearest neighbors can be all located on one side of the point in question x. An alternative natural neighbors method is expensive for more than three variables. In this paper we propose the use of the discrete Choquet integral for combining the values of the nearest neighbors so that redundant information is canceled out. We design a fuzzy measure based on location of the nearest neighbors, which favors neighbors located all around x. <br /