11,136 research outputs found
Becoming the Expert - Interactive Multi-Class Machine Teaching
Compared to machines, humans are extremely good at classifying images into
categories, especially when they possess prior knowledge of the categories at
hand. If this prior information is not available, supervision in the form of
teaching images is required. To learn categories more quickly, people should
see important and representative images first, followed by less important
images later - or not at all. However, image-importance is individual-specific,
i.e. a teaching image is important to a student if it changes their overall
ability to discriminate between classes. Further, students keep learning, so
while image-importance depends on their current knowledge, it also varies with
time.
In this work we propose an Interactive Machine Teaching algorithm that
enables a computer to teach challenging visual concepts to a human. Our
adaptive algorithm chooses, online, which labeled images from a teaching set
should be shown to the student as they learn. We show that a teaching strategy
that probabilistically models the student's ability and progress, based on
their correct and incorrect answers, produces better 'experts'. We present
results using real human participants across several varied and challenging
real-world datasets.Comment: CVPR 201
Fidelity-Weighted Learning
Training deep neural networks requires many training samples, but in practice
training labels are expensive to obtain and may be of varying quality, as some
may be from trusted expert labelers while others might be from heuristics or
other sources of weak supervision such as crowd-sourcing. This creates a
fundamental quality versus-quantity trade-off in the learning process. Do we
learn from the small amount of high-quality data or the potentially large
amount of weakly-labeled data? We argue that if the learner could somehow know
and take the label-quality into account when learning the data representation,
we could get the best of both worlds. To this end, we propose
"fidelity-weighted learning" (FWL), a semi-supervised student-teacher approach
for training deep neural networks using weakly-labeled data. FWL modulates the
parameter updates to a student network (trained on the task we care about) on a
per-sample basis according to the posterior confidence of its label-quality
estimated by a teacher (who has access to the high-quality labels). Both
student and teacher are learned from the data. We evaluate FWL on two tasks in
information retrieval and natural language processing where we outperform
state-of-the-art alternative semi-supervised methods, indicating that our
approach makes better use of strong and weak labels, and leads to better
task-dependent data representations.Comment: Published as a conference paper at ICLR 201
Crowdsourcing in Computer Vision
Computer vision systems require large amounts of manually annotated data to
properly learn challenging visual concepts. Crowdsourcing platforms offer an
inexpensive method to capture human knowledge and understanding, for a vast
number of visual perception tasks. In this survey, we describe the types of
annotations computer vision researchers have collected using crowdsourcing, and
how they have ensured that this data is of high quality while annotation effort
is minimized. We begin by discussing data collection on both classic (e.g.,
object recognition) and recent (e.g., visual story-telling) vision tasks. We
then summarize key design decisions for creating effective data collection
interfaces and workflows, and present strategies for intelligently selecting
the most important data instances to annotate. Finally, we conclude with some
thoughts on the future of crowdsourcing in computer vision.Comment: A 69-page meta review of the field, Foundations and Trends in
Computer Graphics and Vision, 201
My association with William Steel Creighton
The recent (1986) publication of âMy Association with William Morton Wheelerâ evidently stirred my latent autobiographical urge. It is quite reasonable that I should next apply it to William Steel Creighton, for he certainly ranks next to W. M. Wheeler among American myrmecologists. It will be quite different, however, because my actual association with Creighton was very brief twice a dinner guest in New York City and two visits in La Feria, Texas. Correspondence, however, is quite different. I received 45 letters from W. M. Wheeler between 1919 and 1936; the last was dated four months before his death. They dealt chiefly with our proposed treatise on ant larvae; only two ran over to the second page. From Creighton I received 81 letters between 1929 (while he was still a graduate student at Harvard) and 1973 (dated 12 days before his death). His early letters filled one page; the length increased gradually to three pages. The subject matter was chiefly practical taxonomy, but the wide variety of topics treated makes them just as interesting as when they were written
The Teacher-Student Writing Conference Reimaged: Entangled Becoming-Writingconferencing
This analysis is experimental: we attempt to read data with the work of Karen Barad and in doing so âseeâ teacher-student writing conferences (a common pedagogy of US elementary school writing) as intra-activity. Data were gathered during teacher-student writing conferences in a grade five US classroom over a six week period. One conference between a researcher and a male Latino student, a Student of Labels, is diffracted. Reading and writing and thinking with Barad disrupts our habitual ways of privileging language as representational. Rather, we consider the material-discursive practices of schooling that produce what comes to matter, leading us to reimage the teacher-student writing conference as entangled becoming-writingconferencing, speaking to the multiplicity of participants, merging of bodies, continual movement, open-ended possibilities, and anticipated transformation of intra-action
Evorus: A Crowd-powered Conversational Assistant Built to Automate Itself Over Time
Crowd-powered conversational assistants have been shown to be more robust
than automated systems, but do so at the cost of higher response latency and
monetary costs. A promising direction is to combine the two approaches for high
quality, low latency, and low cost solutions. In this paper, we introduce
Evorus, a crowd-powered conversational assistant built to automate itself over
time by (i) allowing new chatbots to be easily integrated to automate more
scenarios, (ii) reusing prior crowd answers, and (iii) learning to
automatically approve response candidates. Our 5-month-long deployment with 80
participants and 281 conversations shows that Evorus can automate itself
without compromising conversation quality. Crowd-AI architectures have long
been proposed as a way to reduce cost and latency for crowd-powered systems;
Evorus demonstrates how automation can be introduced successfully in a deployed
system. Its architecture allows future researchers to make further innovation
on the underlying automated components in the context of a deployed open domain
dialog system.Comment: 10 pages. To appear in the Proceedings of the Conference on Human
Factors in Computing Systems 2018 (CHI'18
One-shot Machine Teaching: Cost Very Few Examples to Converge Faster
Artificial intelligence is to teach machines to take actions like humans. To
achieve intelligent teaching, the machine learning community becomes to think
about a promising topic named machine teaching where the teacher is to design
the optimal (usually minimal) teaching set given a target model and a specific
learner. However, previous works usually require numerous teaching examples
along with large iterations to guide learners to converge, which is costly. In
this paper, we consider a more intelligent teaching paradigm named one-shot
machine teaching which costs fewer examples to converge faster. Different from
typical teaching, this advanced paradigm establishes a tractable mapping from
the teaching set to the model parameter. Theoretically, we prove that this
mapping is surjective, which serves to an existence guarantee of the optimal
teaching set. Then, relying on the surjective mapping from the teaching set to
the parameter, we develop a design strategy of the optimal teaching set under
appropriate settings, of which two popular efficiency metrics, teaching
dimension and iterative teaching dimension are one. Extensive experiments
verify the efficiency of our strategy and further demonstrate the intelligence
of this new teaching paradigm
Visual Knowledge Tracing
Each year, thousands of people learn new visual categorization tasks --
radiologists learn to recognize tumors, birdwatchers learn to distinguish
similar species, and crowd workers learn how to annotate valuable data for
applications like autonomous driving. As humans learn, their brain updates the
visual features it extracts and attend to, which ultimately informs their final
classification decisions. In this work, we propose a novel task of tracing the
evolving classification behavior of human learners as they engage in
challenging visual classification tasks. We propose models that jointly extract
the visual features used by learners as well as predicting the classification
functions they utilize. We collect three challenging new datasets from real
human learners in order to evaluate the performance of different visual
knowledge tracing methods. Our results show that our recurrent models are able
to predict the classification behavior of human learners on three challenging
medical image and species identification tasks.Comment: 14 pages, 4 figures, 14 supplemental pages, 11 supplemental figures,
accepted to European Conference on Computer Vision (ECCV) 202
- âŠ