4 research outputs found
Robustness of Bayesian Pool-based Active Learning Against Prior Misspecification
We study the robustness of active learning (AL) algorithms against prior
misspecification: whether an algorithm achieves similar performance using a
perturbed prior as compared to using the true prior. In both the average and
worst cases of the maximum coverage setting, we prove that all
-approximate algorithms are robust (i.e., near -approximate) if
the utility is Lipschitz continuous in the prior. We further show that
robustness may not be achieved if the utility is non-Lipschitz. This suggests
we should use a Lipschitz utility for AL if robustness is required. For the
minimum cost setting, we can also obtain a robustness result for approximate AL
algorithms. Our results imply that many commonly used AL algorithms are robust
against perturbed priors. We then propose the use of a mixture prior to
alleviate the problem of prior misspecification. We analyze the robustness of
the uniform mixture prior and show experimentally that it performs reasonably
well in practice.Comment: This paper is published at AAAI Conference on Artificial Intelligence
(AAAI 2016
Bayesian Active Learning With Abstention Feedbacks
We study pool-based active learning with abstention feedbacks where a labeler
can abstain from labeling a queried example with some unknown abstention rate.
This is an important problem with many useful applications. We take a Bayesian
approach to the problem and develop two new greedy algorithms that learn both
the classification problem and the unknown abstention rate at the same time.
These are achieved by simply incorporating the estimated average abstention
rate into the greedy criteria. We prove that both algorithms have
near-optimality guarantees: they respectively achieve a
constant factor approximation of the optimal expected or worst-case value of a
useful utility function. Our experiments show the algorithms perform well in
various practical scenarios.Comment: Poster presented at 2019 ICML Workshop on Human in the Loop Learning
2019 (non-archival). arXiv admin note: substantial text overlap with
arXiv:1705.0848
Robustness of Bayesian pool-based active learning against prior misspecification
We study the robustness of active learning (AL) algorithms against prior misspecification: whether an algorithm achieves similar performance using a perturbed prior as compared to using the true prior. In both the average and worst cases of the maximum coverage setting, we prove that all α-approximate algorithms are robust (i.e., near α-approximate) if the utility is Lipschitz continuous in the prior. We further show that robustness may not be achieved if the utility is non-Lipschitz. This suggests we should use a Lipschitz utility for AL if robustness is required. For the minimum cost setting, we can also obtain a robustness result for approximate AL algorithms. Our results imply that many commonly used AL algorithms are robust against perturbed priors. We then propose the use of a mixture prior to alleviate the problem of prior misspecification. We analyze the robustness of the uniform mixture prior and show experimentally that it performs reasonably well in practice