1,089 research outputs found
Causal Interpretation of Self-Attention in Pre-Trained Transformers
We propose a causal interpretation of self-attention in the Transformer
neural network architecture. We interpret self-attention as a mechanism that
estimates a structural equation model for a given input sequence of symbols
(tokens). The structural equation model can be interpreted, in turn, as a
causal structure over the input symbols under the specific context of the input
sequence. Importantly, this interpretation remains valid in the presence of
latent confounders. Following this interpretation, we estimate conditional
independence relations between input symbols by calculating partial
correlations between their corresponding representations in the deepest
attention layer. This enables learning the causal structure over an input
sequence using existing constraint-based algorithms. In this sense, existing
pre-trained Transformers can be utilized for zero-shot causal-discovery. We
demonstrate this method by providing causal explanations for the outcomes of
Transformers in two tasks: sentiment classification (NLP) and recommendation.Comment: 37th Conference on Neural Information Processing Systems (NeurIPS
2023). arXiv admin note: text overlap with arXiv:2210.1062
Enhancing explainability and scrutability of recommender systems
Our increasing reliance on complex algorithms for recommendations calls for models and methods for explainable, scrutable, and trustworthy AI. While explainability is required for understanding the relationships between model inputs and outputs, a scrutable system allows us to modify its behavior as desired. These properties help bridge the gap between our expectations and the algorithmâs behavior and accordingly boost our trust in AI. Aiming to cope with information overload, recommender systems play a crucial role in ďŹltering content (such as products, news, songs, and movies) and shaping a personalized experience for their users. Consequently, there has been a growing demand from the information consumers to receive proper explanations for their personalized recommendations. These explanations aim at helping users understand why certain items are recommended to them and how their previous inputs to the system relate to the generation of such recommendations. Besides, in the event of receiving undesirable content, explanations could possibly contain valuable information as to how the systemâs behavior can be modiďŹed accordingly. In this thesis, we present our contributions towards explainability and scrutability of recommender systems: ⢠We introduce a user-centric framework, FAIRY, for discovering and ranking post-hoc explanations for the social feeds generated by black-box platforms. These explanations reveal relationships between usersâ proďŹles and their feed items and are extracted from the local interaction graphs of users. FAIRY employs a learning-to-rank (LTR) method to score candidate explanations based on their relevance and surprisal. ⢠We propose a method, PRINCE, to facilitate provider-side explainability in graph-based recommender systems that use personalized PageRank at their core. PRINCE explanations are comprehensible for users, because they present subsets of the userâs prior actions responsible for the received recommendations. PRINCE operates in a counterfactual setup and builds on a polynomial-time algorithm for ďŹnding the smallest counterfactual explanations. ⢠We propose a human-in-the-loop framework, ELIXIR, for enhancing scrutability and subsequently the recommendation models by leveraging user feedback on explanations. ELIXIR enables recommender systems to collect user feedback on pairs of recommendations and explanations. The feedback is incorporated into the model by imposing a soft constraint for learning user-speciďŹc item representations. We evaluate all proposed models and methods with real user studies and demonstrate their beneďŹts at achieving explainability and scrutability in recommender systems.Unsere zunehmende Abhängigkeit von komplexen Algorithmen fĂźr maschinelle Empfehlungen erfordert Modelle und Methoden fĂźr erklärbare, nachvollziehbare und vertrauenswĂźrdige KI. Zum Verstehen der Beziehungen zwischen Modellein- und ausgaben muss KI erklärbar sein. MĂśchten wir das Verhalten des Systems hingegen nach unseren Vorstellungen ändern, muss dessen Entscheidungsprozess nachvollziehbar sein. Erklärbarkeit und Nachvollziehbarkeit von KI helfen uns dabei, die LĂźcke zwischen dem von uns erwarteten und dem tatsächlichen Verhalten der Algorithmen zu schlieĂen und unser Vertrauen in KI-Systeme entsprechend zu stärken. Um ein ĂbermaĂ an Informationen zu verhindern, spielen Empfehlungsdienste eine entscheidende Rolle um Inhalte (z.B. Produkten, Nachrichten, Musik und Filmen) zu ďŹltern und deren Benutzern eine personalisierte Erfahrung zu bieten. Infolgedessen erheben immer mehr In- formationskonsumenten Anspruch auf angemessene Erklärungen fĂźr deren personalisierte Empfehlungen. Diese Erklärungen sollen den Benutzern helfen zu verstehen, warum ihnen bestimmte Dinge empfohlen wurden und wie sich ihre frĂźheren Eingaben in das System auf die Generierung solcher Empfehlungen auswirken. AuĂerdem kĂśnnen Erklärungen fĂźr den Fall, dass unerwĂźnschte Inhalte empfohlen werden, wertvolle Informationen darĂźber enthalten, wie das Verhalten des Systems entsprechend geändert werden kann. In dieser Dissertation stellen wir unsere Beiträge zu Erklärbarkeit und Nachvollziehbarkeit von Empfehlungsdiensten vor. ⢠Mit FAIRY stellen wir ein benutzerzentriertes Framework vor, mit dem post-hoc Erklärungen fĂźr die von Black-Box-Plattformen generierten sozialen Feeds entdeckt und bewertet werden kĂśnnen. Diese Erklärungen zeigen Beziehungen zwischen BenutzerproďŹlen und deren Feeds auf und werden aus den lokalen Interaktionsgraphen der Benutzer extrahiert. FAIRY verwendet eine LTR-Methode (Learning-to-Rank), um die Erklärungen anhand ihrer Relevanz und ihres Grads unerwarteter Empfehlungen zu bewerten. ⢠Mit der PRINCE-Methode erleichtern wir das anbieterseitige Generieren von Erklärungen fĂźr PageRank-basierte Empfehlungsdienste. PRINCE-Erklärungen sind fĂźr Benutzer verständlich, da sie Teilmengen frĂźherer Nutzerinteraktionen darstellen, die fĂźr die erhaltenen Empfehlungen verantwortlich sind. PRINCE-Erklärungen sind somit kausaler Natur und werden von einem Algorithmus mit polynomieller Laufzeit erzeugt , um präzise Erklärungen zu ďŹnden. ⢠Wir präsentieren ein Human-in-the-Loop-Framework, ELIXIR, um die Nachvollziehbarkeit der Empfehlungsmodelle und die Qualität der Empfehlungen zu verbessern. Mit ELIXIR kĂśnnen Empfehlungsdienste Benutzerfeedback zu Empfehlungen und Erklärungen sammeln. Das Feedback wird in das Modell einbezogen, indem benutzerspeziďŹscher Einbettungen von Objekten gelernt werden. Wir evaluieren alle Modelle und Methoden in Benutzerstudien und demonstrieren ihren Nutzen hinsichtlich Erklärbarkeit und Nachvollziehbarkeit von Empfehlungsdiensten
Discovering Beaten Paths in Collaborative Ontology-Engineering Projects using Markov Chains
Biomedical taxonomies, thesauri and ontologies in the form of the
International Classification of Diseases (ICD) as a taxonomy or the National
Cancer Institute Thesaurus as an OWL-based ontology, play a critical role in
acquiring, representing and processing information about human health. With
increasing adoption and relevance, biomedical ontologies have also
significantly increased in size. For example, the 11th revision of the ICD,
which is currently under active development by the WHO contains nearly 50,000
classes representing a vast variety of different diseases and causes of death.
This evolution in terms of size was accompanied by an evolution in the way
ontologies are engineered. Because no single individual has the expertise to
develop such large-scale ontologies, ontology-engineering projects have evolved
from small-scale efforts involving just a few domain experts to large-scale
projects that require effective collaboration between dozens or even hundreds
of experts, practitioners and other stakeholders. Understanding how these
stakeholders collaborate will enable us to improve editing environments that
support such collaborations. We uncover how large ontology-engineering
projects, such as the ICD in its 11th revision, unfold by analyzing usage logs
of five different biomedical ontology-engineering projects of varying sizes and
scopes using Markov chains. We discover intriguing interaction patterns (e.g.,
which properties users subsequently change) that suggest that large
collaborative ontology-engineering projects are governed by a few general
principles that determine and drive development. From our analysis, we identify
commonalities and differences between different projects that have implications
for project managers, ontology editors, developers and contributors working on
collaborative ontology-engineering projects and tools in the biomedical domain.Comment: Published in the Journal of Biomedical Informatic
Factors Influencing Userâs Adoption of Conversational Recommender System Based on Product Functional Requirements
Conversational recommender system (CRS) helps customers get products fitted their needs by repeated interaction mechanisms. When customers want to buy products having many and high tech features (e.g., cars, smartphones, notebook, etc.), most users are not familiar with product technical features. The more natural way to elicit customersâ needs is by asking what they really want to use with the product they want (we call as product functional requirements). In this paper, we analyze four factors, e.g., perceived usefulness, perceived ease of use, trust and perceived enjoyment  associated to userâs intention to adopt the interaction model (in CRS) based on product functional requirements. Result of experiment using technology acceptance model (TAM) indicates that, for users who arenât familiar with technical features, perceives usefulness is a main factor influencing usersâ adoption. Meanwhile, perceived enjoyment plays a role on userâs intention to adopt this interaction model, for users who are familiar with technical features of product
Counterfactual Explanations for Neural Recommenders
Understanding why specific items are recommended to users can significantly increase their trust and satisfaction in the system. While neural recommenders have become the state-of-the-art in recent years, the complexity of deep models still makes the generation of tangible explanations for end users a challenging problem. Existing methods are usually based on attention distributions over a variety of features, which are still questionable regarding their suitability as explanations, and rather unwieldy to grasp for an end user. Counterfactual explanations based on a small set of the user's own actions have been shown to be an acceptable solution to the tangibility problem. However, current work on such counterfactuals cannot be readily applied to neural models. In this work, we propose ACCENT, the first general framework for finding counterfactual explanations for neural recommenders. It extends recently-proposed influence functions for identifying training points most relevant to a recommendation, from a single to a pair of items, while deducing a counterfactual set in an iterative process. We use ACCENT to generate counterfactual explanations for two popular neural models, Neural Collaborative Filtering (NCF) and Relational Collaborative Filtering (RCF), and demonstrate its feasibility on a sample of the popular MovieLens 100K dataset
A Co-design Study for Multi-Stakeholder Job Recommender System Explanations
Recent legislation proposals have significantly increased the demand for
eXplainable Artificial Intelligence (XAI) in many businesses, especially in
so-called `high-risk' domains, such as recruitment. Within recruitment, AI has
become commonplace, mainly in the form of job recommender systems (JRSs), which
try to match candidates to vacancies, and vice versa. However, common XAI
techniques often fall short in this domain due to the different levels and
types of expertise of the individuals involved, making explanations difficult
to generalize. To determine the explanation preferences of the different
stakeholder types - candidates, recruiters, and companies - we created and
validated a semi-structured interview guide. Using grounded theory, we
structurally analyzed the results of these interviews and found that different
stakeholder types indeed have strongly differing explanation preferences.
Candidates indicated a preference for brief, textual explanations that allow
them to quickly judge potential matches. On the other hand, hiring managers
preferred visual graph-based explanations that provide a more technical and
comprehensive overview at a glance. Recruiters found more exhaustive textual
explanations preferable, as those provided them with more talking points to
convince both parties of the match. Based on these findings, we describe
guidelines on how to design an explanation interface that fulfills the
requirements of all three stakeholder types. Furthermore, we provide the
validated interview guide, which can assist future research in determining the
explanation preferences of different stakeholder types
Beyond accuracy in machine learning.
Machine Learning (ML) algorithms are widely used in our daily lives. The need to increase the accuracy of ML models has led to building increasingly powerful and complex algorithms known as black-box models which do not provide any explanations about the reasons behind their output. On the other hand, there are white-box ML models which are inherently interpretable while having lower accuracy compared to black-box models. To have a productive and practical algorithmic decision system, precise predictions may not be sufficient. The system may need to have transparency and be able to provide explanations, especially in applications with safety-critical contexts such as medicine, aerospace, robotics, and self-driving vehicles; or in socially-sensitive domains such as credit scoring and predictive policing. This is because having transparency can help explain why a certain decision was made and this, in turn, could be useful in discovering possible biases that lead to discrimination against any individual or group of people. Fairness and bias are other aspects that need to be considered in evaluating ML models. Therefore, depending on the application domain, accuracy, explainability, and fairness from bias may be necessary in building a practical and effective algorithmic decision system. However, in practice, it is challenging to have a model that optimizes all of these three aspects simultaneously. In this work, we study ML criteria that go beyond accuracy in two different problems: 1) in collaborative filtering recommendation, where we study explainability and bias in addition to accuracy; and 2) in robotic grasp failure prediction, where we study explainability in addition to prediction accuracy
- âŚ