4 research outputs found
Sequential Multi-hypothesis Testing in Multi-armed Bandit Problems:An Approach for Asymptotic Optimality
We consider a multi-hypothesis testing problem involving a K-armed bandit.
Each arm's signal follows a distribution from a vector exponential family. The
actual parameters of the arms are unknown to the decision maker. The decision
maker incurs a delay cost for delay until a decision and a switching cost
whenever he switches from one arm to another. His goal is to minimise the
overall cost until a decision is reached on the true hypothesis. Of interest
are policies that satisfy a given constraint on the probability of false
detection. This is a sequential decision making problem where the decision
maker gets only a limited view of the true state of nature at each stage, but
can control his view by choosing the arm to observe at each stage. An
information-theoretic lower bound on the total cost (expected time for a
reliable decision plus total switching cost) is first identified, and a
variation on a sequential policy based on the generalised likelihood ratio
statistic is then studied. Due to the vector exponential family assumption, the
signal processing at each stage is simple; the associated conjugate prior
distribution on the unknown model parameters enables easy updates of the
posterior distribution. The proposed policy, with a suitable threshold for
stopping, is shown to satisfy the given constraint on the probability of false
detection. Under a continuous selection assumption, the policy is also shown to
be asymptotically optimal in terms of the total cost among all policies that
satisfy the constraint on the probability of false detection