73 research outputs found
Targeted Data Generation: Finding and Fixing Model Weaknesses
Even when aggregate accuracy is high, state-of-the-art NLP models often fail
systematically on specific subgroups of data, resulting in unfair outcomes and
eroding user trust. Additional data collection may not help in addressing these
weaknesses, as such challenging subgroups may be unknown to users, and
underrepresented in the existing and new data. We propose Targeted Data
Generation (TDG), a framework that automatically identifies challenging
subgroups, and generates new data for those subgroups using large language
models (LLMs) with a human in the loop. TDG estimates the expected benefit and
potential harm of data augmentation for each subgroup, and selects the ones
most likely to improve within group performance without hurting overall
performance. In our experiments, TDG significantly improves the accuracy on
challenging subgroups for state-of-the-art sentiment analysis and natural
language inference models, while also improving overall test accuracy.Comment: Accepted to ACL 202
Adaptive Testing of Computer Vision Models
Vision models often fail systematically on groups of data that share common
semantic characteristics (e.g., rare objects or unusual scenes), but
identifying these failure modes is a challenge. We introduce AdaVision, an
interactive process for testing vision models which helps users identify and
fix coherent failure modes. Given a natural language description of a coherent
group, AdaVision retrieves relevant images from LAION-5B with CLIP. The user
then labels a small amount of data for model correctness, which is used in
successive retrieval rounds to hill-climb towards high-error regions, refining
the group definition. Once a group is saturated, AdaVision uses GPT-3 to
suggest new group descriptions for the user to explore. We demonstrate the
usefulness and generality of AdaVision in user studies, where users find major
bugs in state-of-the-art classification, object detection, and image captioning
models. These user-discovered groups have failure rates 2-3x higher than those
surfaced by automatic error clustering methods. Finally, finetuning on examples
found with AdaVision fixes the discovered bugs when evaluated on unseen
examples, without degrading in-distribution accuracy, and while also improving
performance on out-of-distribution datasets.Comment: ICCV camera-read
Supporting Human-AI Collaboration in Auditing LLMs with LLMs
Large language models are becoming increasingly pervasive and ubiquitous in
society via deployment in sociotechnical systems. Yet these language models, be
it for classification or generation, have been shown to be biased and behave
irresponsibly, causing harm to people at scale. It is crucial to audit these
language models rigorously. Existing auditing tools leverage either or both
humans and AI to find failures. In this work, we draw upon literature in
human-AI collaboration and sensemaking, and conduct interviews with research
experts in safe and fair AI, to build upon the auditing tool: AdaTest (Ribeiro
and Lundberg, 2022), which is powered by a generative large language model
(LLM). Through the design process we highlight the importance of sensemaking
and human-AI communication to leverage complementary strengths of humans and
generative models in collaborative auditing. To evaluate the effectiveness of
the augmented tool, AdaTest++, we conduct user studies with participants
auditing two commercial language models: OpenAI's GPT-3 and Azure's sentiment
analysis model. Qualitative analysis shows that AdaTest++ effectively leverages
human strengths such as schematization, hypothesis formation and testing.
Further, with our tool, participants identified a variety of failures modes,
covering 26 different topics over 2 tasks, that have been shown before in
formal audits and also those previously under-reported.Comment: 21 pages, 3 figure
ScatterShot: Interactive In-context Example Curation for Text Transformation
The in-context learning capabilities of LLMs like GPT-3 allow annotators to
customize an LLM to their specific tasks with a small number of examples.
However, users tend to include only the most obvious patterns when crafting
examples, resulting in underspecified in-context functions that fall short on
unseen cases. Further, it is hard to know when "enough" examples have been
included even for known patterns. In this work, we present ScatterShot, an
interactive system for building high-quality demonstration sets for in-context
learning. ScatterShot iteratively slices unlabeled data into task-specific
patterns, samples informative inputs from underexplored or not-yet-saturated
slices in an active learning manner, and helps users label more efficiently
with the help of an LLM and the current example set. In simulation studies on
two text perturbation scenarios, ScatterShot sampling improves the resulting
few-shot functions by 4-5 percentage points over random sampling, with less
variance as more examples are added. In a user study, ScatterShot greatly helps
users in covering different patterns in the input space and labeling in-context
examples more efficiently, resulting in better in-context learning and less
user effort.Comment: IUI 2023: 28th International Conference on Intelligent User
Interface
A study on the Interpretability of Neural Retrieval Models using DeepSHAP
A recent trend in IR has been the usage of neural networks to learn retrieval
models for text based adhoc search. While various approaches and architectures
have yielded significantly better performance than traditional retrieval models
such as BM25, it is still difficult to understand exactly why a document is
relevant to a query. In the ML community several approaches for explaining
decisions made by deep neural networks have been proposed -- including DeepSHAP
which modifies the DeepLift algorithm to estimate the relative importance
(shapley values) of input features for a given decision by comparing the
activations in the network for a given image against the activations caused by
a reference input. In image classification, the reference input tends to be a
plain black image. While DeepSHAP has been well studied for image
classification tasks, it remains to be seen how we can adapt it to explain the
output of Neural Retrieval Models (NRMs). In particular, what is a good "black"
image in the context of IR? In this paper we explored various reference input
document construction techniques. Additionally, we compared the explanations
generated by DeepSHAP to LIME (a model agnostic approach) and found that the
explanations differ considerably. Our study raises concerns regarding the
robustness and accuracy of explanations produced for NRMs. With this paper we
aim to shed light on interesting problems surrounding interpretability in NRMs
and highlight areas of future work.Comment: 4 pages; SIGIR 2019 Short Pape
Editing Models with Task Arithmetic
Changing how pre-trained models behave -- e.g., improving their performance
on a downstream task or mitigating biases learned during pre-training -- is a
common practice when developing machine learning systems. In this work, we
propose a new paradigm for steering the behavior of neural networks, centered
around \textit{task vectors}. A task vector specifies a direction in the weight
space of a pre-trained model, such that movement in that direction improves
performance on the task. We build task vectors by subtracting the weights of a
pre-trained model from the weights of the same model after fine-tuning on a
task. We show that these task vectors can be modified and combined together
through arithmetic operations such as negation and addition, and the behavior
of the resulting model is steered accordingly. Negating a task vector decreases
performance on the target task, with little change in model behavior on control
tasks. Moreover, adding task vectors together can improve performance on
multiple tasks at once. Finally, when tasks are linked by an analogy
relationship of the form ``A is to B as C is to D", combining task vectors from
three of the tasks can improve performance on the fourth, even when no data
from the fourth task is used for training. Overall, our experiments with
several models, modalities and tasks show that task arithmetic is a simple,
efficient and effective way of editing models.Comment: In Proceedings of the 11th International Conference on Learning
Representations (ICLR 2023
Actionable Recourse in Linear Classification
Machine learning models are increasingly used to automate decisions that
affect humans - deciding who should receive a loan, a job interview, or a
social service. In such applications, a person should have the ability to
change the decision of a model. When a person is denied a loan by a credit
score, for example, they should be able to alter its input variables in a way
that guarantees approval. Otherwise, they will be denied the loan as long as
the model is deployed. More importantly, they will lack the ability to
influence a decision that affects their livelihood.
In this paper, we frame these issues in terms of recourse, which we define as
the ability of a person to change the decision of a model by altering
actionable input variables (e.g., income vs. age or marital status). We present
integer programming tools to ensure recourse in linear classification problems
without interfering in model development. We demonstrate how our tools can
inform stakeholders through experiments on credit scoring problems. Our results
show that recourse can be significantly affected by standard practices in model
development, and motivate the need to evaluate recourse in practice.Comment: Extended version. ACM Conference on Fairness, Accountability and
Transparency [FAT2019
- …