9 research outputs found
InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanations
While recently developed NLP explainability methods let us open the black box
in various ways (Madsen et al., 2022), a missing ingredient in this endeavor is
an interactive tool offering a conversational interface. Such a dialogue system
can help users explore datasets and models with explanations in a
contextualized manner, e.g. via clarification or follow-up questions, and
through a natural language interface. We adapt the conversational explanation
framework TalkToModel (Slack et al., 2022) to the NLP domain, add new
NLP-specific operations such as free-text rationalization, and illustrate its
generalizability on three NLP tasks (dialogue act classification, question
answering, hate speech detection). To recognize user queries for explanations,
we evaluate fine-tuned and few-shot prompting models and implement a novel
Adapter-based approach. We then conduct two user studies on (1) the perceived
correctness and helpfulness of the dialogues, and (2) the simulatability, i.e.
how objectively helpful dialogical explanations are for humans in figuring out
the model's predicted label when it's not shown. We found rationalization and
feature attribution were helpful in explaining the model behavior. Moreover,
users could more reliably predict the model outcome based on an explanation
dialogue rather than one-off explanations.Comment: EMNLP 2023 Findings. Camera-ready versio
Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methods
Saliency maps can explain a neural model's predictions by identifying
important input features. They are difficult to interpret for laypeople,
especially for instances with many features. In order to make them more
accessible, we formalize the underexplored task of translating saliency maps
into natural language and compare methods that address two key challenges of
this approach -- what and how to verbalize. In both automatic and human
evaluation setups, using token-level attributions from text classification
tasks, we compare two novel methods (search-based and instruction-based
verbalizations) against conventional feature importance representations
(heatmap visualizations and extractive rationales), measuring simulatability,
faithfulness, helpfulness and ease of understanding. Instructing GPT-3.5 to
generate saliency map verbalizations yields plausible explanations which
include associations, abstractive summarization and commonsense reasoning,
achieving by far the highest human ratings, but they are not faithfully
capturing numeric information and are inconsistent in their interpretation of
the task. In comparison, our search-based, model-free verbalization approach
efficiently completes templated verbalizations, is faithful by design, but
falls short in helpfulness and simulatability. Our results suggest that
saliency map verbalization makes feature attribution explanations more
comprehensible and less cognitively challenging to humans than conventional
representations.Comment: ACL 2023 Workshop on Natural Language Reasoning and Structured
Explanations (NLRSE
Inseq: An Interpretability Toolkit for Sequence Generation Models
Past work in natural language processing interpretability focused mainly on
popular classification tasks while largely overlooking generation settings,
partly due to a lack of dedicated tools. In this work, we introduce Inseq, a
Python library to democratize access to interpretability analyses of sequence
generation models. Inseq enables intuitive and optimized extraction of models'
internal information and feature importance scores for popular decoder-only and
encoder-decoder Transformers architectures. We showcase its potential by
adopting it to highlight gender biases in machine translation models and locate
factual knowledge inside GPT-2. Thanks to its extensible interface supporting
cutting-edge techniques such as contrastive feature attribution, Inseq can
drive future advances in explainable natural language generation, centralizing
good practices and enabling fair and reproducible model evaluations.Comment: Library: https://github.com/inseq-team/inseq, Documentation:
https://inseq.readthedocs.io, v0.
Towards an interoperable ecosystem of AI and LT platforms: a roadmap for the implementation of different levels of interoperability
European Language Grid: A Joint Platform for the European Language Technology Community
Europe is a multilingual society, in which dozens of languages are spoken. The only option to enable and to benefit from multilingualism is through Language Technologies (LT), i.e., Natural Language Processing and Speech Technologies. We describe the European Language Grid (ELG), which is targeted to evolve into the primary platform and marketplace for LT in Europe by providing one umbrella platform for the European LT landscape, including research and industry, enabling all stakeholders to upload, share and distribute their services, products and resources. At the end of our EU project, which will establish a legal entity in 2022, the ELG will provide access to approx. 1300 services for all European languages as well as thousands of data sets