4 research outputs found
Bayesian Active Learning With Abstention Feedbacks
We study pool-based active learning with abstention feedbacks where a labeler
can abstain from labeling a queried example with some unknown abstention rate.
This is an important problem with many useful applications. We take a Bayesian
approach to the problem and develop two new greedy algorithms that learn both
the classification problem and the unknown abstention rate at the same time.
These are achieved by simply incorporating the estimated average abstention
rate into the greedy criteria. We prove that both algorithms have
near-optimality guarantees: they respectively achieve a
constant factor approximation of the optimal expected or worst-case value of a
useful utility function. Our experiments show the algorithms perform well in
various practical scenarios.Comment: Poster presented at 2019 ICML Workshop on Human in the Loop Learning
2019 (non-archival). arXiv admin note: substantial text overlap with
arXiv:1705.0848
Active learning from oracle with knowledge blind spot
Active learning traditionally assumes that an oracle is capable of providing labeling information for each query instance. This paper formulates a new research problem which allows an oracle admit that he/she is incapable of labeling some query instances or simply answer "I don't know the label". We define a unified objective function to ensure that each query instance submitted to the oracle is the one mostly needed for labeling and the oracle should also has the knowledge to label. Experiments based on different types of knowledge blind spot (KBS) models demonstrate the effectiveness of the proposed design. Copyright © 2012, Association for the Advancement of Artificial Intelligence. All rights reserved
Active Learning from Oracle with Knowledge Blind Spot
Active learning traditionally assumes that an oracle is capable of providing labeling information for each query instance. This paper formulates a new research problem which allows an oracle admit that he/she is incapable of labeling some query instances or simply answer "I don't know the label." We define a unified objectivefunction to ensure that each query instance submitted to the oracleis the one mostly needed for labeling and the oracle should also hasthe knowledge to label. Experiments based on different types of knowledge blind spot (KBS) models demonstrate the effectiveness of theproposed design