416,281 research outputs found
Beyond Labels: Empowering Human Annotators with Natural Language Explanations through a Novel Active-Learning Architecture
Real-world domain experts (e.g., doctors) rarely annotate only a decision
label in their day-to-day workflow without providing explanations. Yet,
existing low-resource learning techniques, such as Active Learning (AL), that
aim to support human annotators mostly focus on the label while neglecting the
natural language explanation of a data point. This work proposes a novel AL
architecture to support experts' real-world need for label and explanation
annotations in low-resource scenarios. Our AL architecture leverages an
explanation-generation model to produce explanations guided by human
explanations, a prediction model that utilizes generated explanations toward
prediction faithfully, and a novel data diversity-based AL sampling strategy
that benefits from the explanation annotations. Automated and human evaluations
demonstrate the effectiveness of incorporating explanations into AL sampling
and the improved human annotation efficiency and trustworthiness with our AL
architecture. Additional ablation studies illustrate the potential of our AL
architecture for transfer learning, generalizability, and integration with
large language models (LLMs). While LLMs exhibit exceptional
explanation-generation capabilities for relatively simple tasks, their
effectiveness in complex real-world tasks warrants further in-depth study.Comment: Accepted to EMNLP 2023 Finding
Leveraging Multiple Teachers for Test-Time Adaptation of Language-Guided Classifiers
Recent approaches have explored language-guided classifiers capable of
classifying examples from novel tasks when provided with task-specific natural
language explanations, instructions or prompts (Sanh et al., 2022; R. Menon et
al., 2022). While these classifiers can generalize in zero-shot settings, their
task performance often varies substantially between different language
explanations in unpredictable ways (Lu et al., 2022; Gonen et al., 2022). Also,
current approaches fail to leverage unlabeled examples that may be available in
many scenarios. Here, we introduce TALC, a framework that uses data programming
to adapt a language-guided classifier for a new task during inference when
provided with explanations from multiple teachers and unlabeled test examples.
Our results show that TALC consistently outperforms a competitive baseline from
prior work by an impressive 9.3% (relative improvement). Further, we
demonstrate the robustness of TALC to variations in the quality and quantity of
provided explanations, highlighting its potential in scenarios where learning
from multiple teachers or a crowd is involved. Our code is available at:
https://github.com/WeiKangda/TALC.git
Why Attention is Not Explanation: Surgical Intervention and Causal Reasoning about Neural Models
As the demand for explainable deep learning grows in the evaluation of language technologies, the value of a principled grounding for those explanations grows as well. Here we study the state-of-the-art in explanation for neural models for natural-language processing (NLP) tasks from the viewpoint of philosophy of science. We focus on recent evaluation work that finds brittleness in explanations obtained through attention mechanisms.We harness philosophical accounts of explanation to suggest broader conclusions from these studies. From this analysis, we assert the impossibility of causal explanations from attention layers over text data. We then introduce NLP researchers to contemporary philosophy of science theories that allow robust yet non-causal reasoning in explanation, giving computer scientists a vocabulary for future researc
INVESTIGATING GENDER INFLUENCE ON LANGUAGE LEARNING BELIEFS
This paper reports part of a study that examines undergraduate English as second language (ESL) students’ English learning beliefs, language anxiety and learning outcome in a university of Pakistan. As a pilot of the study, this paper uses Horwitz’s Belief about Language Learning Inventory (BALLI) to collect data from 404 undergraduate ESL students, and explores the effects of gender on Pakistani undergraduate ESL students’ English language learning beliefs. The results indicate that males and females held similar beliefs in the factor of “motivations and expectations”, but significantly differed in the factor of “nature of language learning”. There were gender differences in the other three factors as well, but those were statistically insignificant. Possible explanations are provided for the differences. Based upon the findings, pedagogical implications are provided for improvement of teaching and learning of English in Pakistan
Interpretable Categorization of Heterogeneous Time Series Data
Understanding heterogeneous multivariate time series data is important in
many applications ranging from smart homes to aviation. Learning models of
heterogeneous multivariate time series that are also human-interpretable is
challenging and not adequately addressed by the existing literature. We propose
grammar-based decision trees (GBDTs) and an algorithm for learning them. GBDTs
extend decision trees with a grammar framework. Logical expressions derived
from a context-free grammar are used for branching in place of simple
thresholds on attributes. The added expressivity enables support for a wide
range of data types while retaining the interpretability of decision trees. In
particular, when a grammar based on temporal logic is used, we show that GBDTs
can be used for the interpretable classi cation of high-dimensional and
heterogeneous time series data. Furthermore, we show how GBDTs can also be used
for categorization, which is a combination of clustering and generating
interpretable explanations for each cluster. We apply GBDTs to analyze the
classic Australian Sign Language dataset as well as data on near mid-air
collisions (NMACs). The NMAC data comes from aircraft simulations used in the
development of the next-generation Airborne Collision Avoidance System (ACAS
X).Comment: 9 pages, 5 figures, 2 tables, SIAM International Conference on Data
Mining (SDM) 201
- …