433 research outputs found
On Optimality of Myopic Sensing Policy with Imperfect Sensing in Multi-channel Opportunistic Access
We consider the channel access problem under imperfect sensing of channel
state in a multi-channel opportunistic communication system, where the state of
each channel evolves as an independent and identically distributed Markov
process. The considered problem can be cast into a restless multi-armed bandit
(RMAB) problem that is of fundamental importance in decision theory. It is
well-known that solving the RMAB problem is PSPACE-hard, with the optimal
policy usually intractable due to the exponential computation complexity. A
natural alternative is to consider the easily implementable myopic policy that
maximizes the immediate reward but ignores the impact of the current strategy
on the future reward. In this paper, we perform an analytical study on the
optimality of the myopic policy under imperfect sensing for the considered RMAB
problem. Specifically, for a family of generic and practically important
utility functions, we establish the closed-form conditions under which the
myopic policy is guaranteed to be optimal even under imperfect sensing. Despite
our focus on the opportunistic channel access, the obtained results are generic
in nature and are widely applicable in a wide range of engineering domains.Comment: 21 pages regular pape
Active Sensing as Bayes-Optimal Sequential Decision Making
Sensory inference under conditions of uncertainty is a major problem in both
machine learning and computational neuroscience. An important but poorly
understood aspect of sensory processing is the role of active sensing. Here, we
present a Bayes-optimal inference and control framework for active sensing,
C-DAC (Context-Dependent Active Controller). Unlike previously proposed
algorithms that optimize abstract statistical objectives such as information
maximization (Infomax) [Butko & Movellan, 2010] or one-step look-ahead accuracy
[Najemnik & Geisler, 2005], our active sensing model directly minimizes a
combination of behavioral costs, such as temporal delay, response error, and
effort. We simulate these algorithms on a simple visual search task to
illustrate scenarios in which context-sensitivity is particularly beneficial
and optimization with respect to generic statistical objectives particularly
inadequate. Motivated by the geometric properties of the C-DAC policy, we
present both parametric and non-parametric approximations, which retain
context-sensitivity while significantly reducing computational complexity.
These approximations enable us to investigate the more complex problem
involving peripheral vision, and we notice that the difference between C-DAC
and statistical policies becomes even more evident in this scenario.Comment: Scheduled to appear in UAI 201
- …