We consider two active binary-classification problems with atypical
objectives. In the first, active search, our goal is to actively uncover as
many members of a given class as possible. In the second, active surveying, our
goal is to actively query points to ultimately predict the proportion of a
given class. Numerous real-world problems can be framed in these terms, and in
either case typical model-based concerns such as generalization error are only
of secondary importance.
We approach these problems via Bayesian decision theory; after choosing
natural utility functions, we derive the optimal policies. We provide three
contributions. In addition to introducing the active surveying problem, we
extend previous work on active search in two ways. First, we prove a novel
theoretical result, that less-myopic approximations to the optimal policy can
outperform more-myopic approximations by any arbitrary degree. We then derive
bounds that for certain models allow us to reduce (in practice dramatically)
the exponential search space required by a naive implementation of the optimal
policy, enabling further lookahead while still ensuring that optimal decisions
are always made.Comment: Appears in Proceedings of the 29th International Conference on
Machine Learning (ICML 2012