7 research outputs found
Context Embedding Networks
Low dimensional embeddings that capture the main variations of interest in
collections of data are important for many applications. One way to construct
these embeddings is to acquire estimates of similarity from the crowd. However,
similarity is a multi-dimensional concept that varies from individual to
individual. Existing models for learning embeddings from the crowd typically
make simplifying assumptions such as all individuals estimate similarity using
the same criteria, the list of criteria is known in advance, or that the crowd
workers are not influenced by the data that they see. To overcome these
limitations we introduce Context Embedding Networks (CENs). In addition to
learning interpretable embeddings from images, CENs also model worker biases
for different attributes along with the visual context i.e. the visual
attributes highlighted by a set of images. Experiments on two noisy crowd
annotated datasets show that modeling both worker bias and visual context
results in more interpretable embeddings compared to existing approaches.Comment: CVPR 2018 spotligh
Teaching Categories to Human Learners with Visual Explanations
We study the problem of computer-assisted teaching with explanations.
Conventional approaches for machine teaching typically only provide feedback at
the instance level e.g., the category or label of the instance. However, it is
intuitive that clear explanations from a knowledgeable teacher can
significantly improve a student's ability to learn a new concept. To address
these existing limitations, we propose a teaching framework that provides
interpretable explanations as feedback and models how the learner incorporates
this additional information. In the case of images, we show that we can
automatically generate explanations that highlight the parts of the image that
are responsible for the class label. Experiments on human learners illustrate
that, on average, participants achieve better test set performance on
challenging categorization tasks when taught with our interpretable approach
compared to existing methods
Teaching categories to human learners with visual explanations
We study the problem of computer-assisted teaching with explanations. Conventional approaches for machine teaching typically only provide feedback at the instance level e.g., the category or label of the instance. However, it is intuitive that clear explanations from a knowledgeable teacher can significantly improve a student's ability to learn a new concept. To address these existing limitations, we propose a teaching framework that provides interpretable explanations as feedback and models how the learner incorporates this additional information. In the case of images, we show that we can automatically generate explanations that highlight the parts of the image that are responsible for the class label. Experiments on human learners illustrate that, on average, participants achieve better test set performance on challenging categorization tasks when taught with our interpretable approach compared to existing methods
Visual Knowledge Tracing
Each year, thousands of people learn new visual categorization tasks --
radiologists learn to recognize tumors, birdwatchers learn to distinguish
similar species, and crowd workers learn how to annotate valuable data for
applications like autonomous driving. As humans learn, their brain updates the
visual features it extracts and attend to, which ultimately informs their final
classification decisions. In this work, we propose a novel task of tracing the
evolving classification behavior of human learners as they engage in
challenging visual classification tasks. We propose models that jointly extract
the visual features used by learners as well as predicting the classification
functions they utilize. We collect three challenging new datasets from real
human learners in order to evaluate the performance of different visual
knowledge tracing methods. Our results show that our recurrent models are able
to predict the classification behavior of human learners on three challenging
medical image and species identification tasks.Comment: 14 pages, 4 figures, 14 supplemental pages, 11 supplemental figures,
accepted to European Conference on Computer Vision (ECCV) 202
Few-Shot Attribute Learning
Semantic concepts are frequently defined by combinations of underlying
attributes. As mappings from attributes to classes are often simple,
attribute-based representations facilitate novel concept learning with zero or
few examples. A significant limitation of existing attribute-based learning
paradigms, such as zero-shot learning, is that the attributes are assumed to be
known and fixed. In this work we study the rapid learning of attributes that
were not previously labeled. Compared to standard few-shot learning of semantic
classes, in which novel classes may be defined by attributes that were relevant
at training time, learning new attributes imposes a stiffer challenge. We found
that supervised learning with training attributes does not generalize well to
new test attributes, whereas self-supervised pre-training brings significant
improvement. We further experimented with random splits of the attribute space
and found that predictability of test attributes provides an informative
estimate of a model's generalization ability.Comment: Technical report, 25 page
Teaching categories to human learners with visual explanations
We study the problem of computer-assisted teaching with explanations. Conventional approaches for machine teaching typically only provide feedback at the instance level e.g., the category or label of the instance. However, it is intuitive that clear explanations from a knowledgeable teacher can significantly improve a student's ability to learn a new concept. To address these existing limitations, we propose a teaching framework that provides interpretable explanations as feedback and models how the learner incorporates this additional information. In the case of images, we show that we can automatically generate explanations that highlight the parts of the image that are responsible for the class label. Experiments on human learners illustrate that, on average, participants achieve better test set performance on challenging categorization tasks when taught with our interpretable approach compared to existing methods