Evaluating the Prediction Bias Induced by Label Imbalance in Multi-label Classification

Prediction bias is a well-known problem in classification algorithms, which tend to be skewed towards more represented classes. This phenomenon is even more remarkable in multi-label scenarios, where the number of underrepresented classes is usually larger. In light of this, we hereby present the Prediction Bias Coefficient (PBC), a novel measure that aims to assess the bias induced by label imbalance in multi-label classification. The approach leverages Spearman's rank correlation coefficient between the label frequencies and the F-scores obtained for each label individually. After describing the theoretical properties of the proposed indicator, we illustrate its behaviour on a classification task performed with state-of-the-art methods on two real-world datasets, and we compare it experimentally with other metrics described in the literature

Robust reputation independence in ranking systems for multiple sensitive attributes

Ranking systems have an unprecedented influence on how and what information people access, and their impact on our society is being analyzed from different perspectives, such as users’ discrimination. A notable example is represented by reputation-based ranking systems, a class of systems that rely on users’ reputation to generate a non-personalized item-ranking, proved to be biased against certain demographic classes. To safeguard that a given sensitive user’s attribute does not systematically affect the reputation of that user, prior work has operationalized a reputation independence constraint on this class of systems. In this paper, we uncover that guaranteeing reputation independence for a single sensitive attribute is not enough. When mitigating biases based on one sensitive attribute (e.g., gender), the final ranking might still be biased against certain demographic groups formed based on another attribute (e.g., age). Hence, we propose a novel approach to introduce reputation independence for multiple sensitive attributes simultaneously. We then analyze the extent to which our approach impacts on discrimination and other important properties of the ranking system, such as its quality and robustness against attacks. Experiments on two real-world datasets show that our approach leads to less biased rankings with respect to multiple users’ sensitive attributes, without affecting the system’s quality and robustness

Combining mitigation treatments against biases in personalized rankings: Use case on item popularity

Historical interactions leveraged by recommender systems are often non-uniformly distributed across items. Though they are of interest for consumers, certain items end up therefore being biasedly under-recommended. Existing treatments for mitigating these biases act at a single step of the pipeline (either pre-, in-, or post-processing), and it remains unanswered whether simultaneously introducing treatments throughout the pipeline leads to a better mitigation. In this paper, we analyze the impact of bias treatments along the steps of the pipeline under a use case on popularity bias. Experiments show that, with small losses in accuracy, the combination of treatments leads to better trade-offs than treatments applied separately. Our findings call for treatments rooting out bias at different steps simultaneously

Espacialização e geração de mapas temáticos das condições de cobertura dos solos para a região Norte de MG, a partir de análises espectrais de imagens do satélite Landsat-5 TM.

Este trabalho teve como objetivo realizar uma análise do comportamento espectral das classes de cobertura vegetal do Norte de Minas Gerais, na região semi-árida do médio São Francisco, onde estão inseridos os municípios de Nova Porteirinha (Sítio de Fenotipagem de seleção de genótipos de cereais para a Tolerância à Seca - Embrapa Milho e Sorgo) e Janaúba e os Perímetros Irrigados de Gorutuba e Jaíba, por meio de técnicas de sensoriamento remoto (imagens de um satélite) e de geoprocessamento (SIG), visando espacializar as condições de cobertura dos solos, utilizando como metodologia o cálculo do NDVI (?Normalized Difference Vegetation Index?) e Kc (coeficiente de cultura) para gerar mapas temáticos das condições da cobertura do solo e do uso da água. O NDVI calculado com base na imagem Landsat-5 TM foi eficiente na detecção de áreas com diferentes coberturas vegetais e a equação linear utilizada na espacialização do Kc apresentou resultados satisfatórios

Post Processing Recommender Systems with Knowledge Graphs for Recency, Popularity, and Diversity of Explanations

Existing explainable recommender systems have mainly modeled relationships between recommended and already experienced products, and shaped explanation types accordingly (e.g., movie "x"starred by actress "y"recommended to a user because that user watched other movies with "y"as an actress). However, none of these systems has investigated the extent to which properties of a single explanation (e.g., the recency of interaction with that actress) and of a group of explanations for a recommended list (e.g., the diversity of the explanation types) can influence the perceived explaination quality. In this paper, we conceptualized three novel properties that model the quality of the explanations (linking interaction recency, shared entity popularity, and explanation type diversity) and proposed re-ranking approaches able to optimize for these properties. Experiments on two public data sets showed that our approaches can increase explanation quality according to the proposed properties, fairly across demographic groups, while preserving recommendation utility. The source code and data are available at https: //github.com/giacoballoccu/explanation-quality-recsys

Recency, Popularity, and Diversity of Explanations in Knowledge-based Recommendation

Modern knowledge-based recommender systems enable the end-to-end generation of textual explanations. These explanations are created from learnt paths between an already experience product and a recommended product in a knowledge graph, for a given user. However, none of the existing studies has investigated the extent to which properties of a single explanation (e.g., the recency of interaction with the already experience product) and of a group of explanations for a recommended list (e.g., the diversity of the explanation types) can influence the perceived explanation quality. In this paper, we summarize our previous work on conceptualizing three novel properties that model the quality of the explanations (linking interaction recency, shared entity popularity, and explanation type diversity) and proposing re-ranking approaches able to optimize for these properties. Experiments on two public data sets showed that our approaches can increase explanation quality according to the proposed properties, while preserving recommendation utility. Source code and data: https://github.com/giacoballoccu/explanation-quality-recsys

Do Graph Neural Networks Build Fair User Models? Assessing Disparate Impact and Mistreatment in Behavioural User Profiling

Recent approaches to behavioural user profiling employ Graph Neural Networks (GNNs) to turn users' interactions with a platform into actionable knowledge. The effectiveness of an approach is usually assessed with accuracy-based perspectives, where the capability to predict user features (such as gender or age) is evaluated. In this work, we perform a beyond-accuracy analysis of the state-of-the-art approaches to assess the presence of disparate impact and disparate mistreatment, meaning that users characterised by a given sensitive feature are unintentionally, but systematically, classified worse than their counterparts. Our analysis on two real-world datasets shows that different user profiling paradigms can impact fairness results. The source code and the preprocessed datasets are available at: https://github.com/erasmopurif/do_gnns_build_fair_models

XRecSys: A framework for path reasoning quality in explainable recommendation

There is increasing evidence that recommendations accompanied by explanations positively impact on businesses in terms of trust, guidance, and persuasion. This advance has been made possible by traditional models representing user–product interactions augmented with external knowledge modeled as knowledge graphs. However, these models produce textual explanations on top of reasoning paths extracted from the knowledge graph without considering relevant properties of the path entities. In this paper, we present XRecSys, a Python framework for the optimization of the reasoning path selection process according to properties deemed relevant by users (e.g., time relevance of the linking interaction or popularity of the entity linked to the explanation). Our framework leads to a higher reasoning path quality in terms of the considered properties and, consequently, textual explanations more relevant for the users