288 research outputs found
Active Inference for Autonomous Decision-Making with Contextual Multi-Armed Bandits
In autonomous robotic decision-making under uncertainty, the tradeoff between
exploitation and exploration of available options must be considered. If
secondary information associated with options can be utilized, such
decision-making problems can often be formulated as a contextual multi-armed
bandits (CMABs). In this study, we apply active inference, which has been
actively studied in the field of neuroscience in recent years, as an
alternative action selection strategy for CMABs. Unlike conventional action
selection strategies, it is possible to rigorously evaluate the uncertainty of
each option when calculating the expected free energy (EFE) associated with the
decision agent's probabilistic model, as derived from the free-energy
principle. We specifically address the case where a categorical observation
likelihood function is used, such that EFE values are analytically intractable.
We introduce new approximation methods for computing the EFE based on
variational and Laplace approximations. Extensive simulation study results
demonstrate that, compared to other strategies, active inference generally
requires far fewer iterations to identify optimal options and generally
achieves superior cumulative regret, for relatively low extra computational
cost
- …