262,802 research outputs found
Inferring Hierarchical Structure in Multi-Room Maze Environments
Cognitive maps play a crucial role in facilitating flexible behaviour by
representing spatial and conceptual relationships within an environment. The
ability to learn and infer the underlying structure of the environment is
crucial for effective exploration and navigation. This paper introduces a
hierarchical active inference model addressing the challenge of inferring
structure in the world from pixel-based observations. We propose a three-layer
hierarchical model consisting of a cognitive map, an allocentric, and an
egocentric world model, combining curiosity-driven exploration with
goal-oriented behaviour at the different levels of reasoning from context to
place to motion. This allows for efficient exploration and goal-directed search
in room-structured mini-grid environments.Comment: ICML 2023 Worksho
Model-based Reinforcement Learning with Parametrized Physical Models and Optimism-Driven Exploration
In this paper, we present a robotic model-based reinforcement learning method
that combines ideas from model identification and model predictive control. We
use a feature-based representation of the dynamics that allows the dynamics
model to be fitted with a simple least squares procedure, and the features are
identified from a high-level specification of the robot's morphology,
consisting of the number and connectivity structure of its links. Model
predictive control is then used to choose the actions under an optimistic model
of the dynamics, which produces an efficient and goal-directed exploration
strategy. We present real time experimental results on standard benchmark
problems involving the pendulum, cartpole, and double pendulum systems.
Experiments indicate that our method is able to learn a range of benchmark
tasks substantially faster than the previous best methods. To evaluate our
approach on a realistic robotic control task, we also demonstrate real time
control of a simulated 7 degree of freedom arm.Comment: 8 page
Learning Augmented, Multi-Robot Long-Horizon Navigation in Partially Mapped Environments
We present a novel approach for efficient and reliable goal-directed
long-horizon navigation for a multi-robot team in a structured, unknown
environment by predicting statistics of unknown space. Building on recent work
in learning-augmented model based planning under uncertainty, we introduce a
high-level state and action abstraction that lets us approximate the
challenging Dec-POMDP into a tractable stochastic MDP. Our Multi-Robot Learning
over Subgoals Planner (MR-LSP) guides agents towards coordinated exploration of
regions more likely to reach the unseen goal. We demonstrate improvement in
cost against other multi-robot strategies; in simulated office-like
environments, we show that our approach saves 13.29% (2 robot) and 4.6% (3
robot) average cost versus standard non-learned optimistic planning and a
learning-informed baseline.Comment: 7 pages, 7 figures, ICRA202
Learning action-oriented models through active inference
Converging theories suggest that organisms learn and exploit probabilistic models of their environment. However, it remains unclear how such models can be learned in practice. The open-ended complexity of natural environments means that it is generally infeasible for organisms to model their environment comprehensively. Alternatively, action-oriented models attempt to encode a parsimonious representation of adaptive agent-environment interactions. One approach to learning action-oriented models is to learn online in the presence of goal-directed behaviours. This constrains an agent to behaviourally relevant trajectories, reducing the diversity of the data a model need account for. Unfortunately, this approach can cause models to prematurely converge to sub-optimal solutions, through a process we refer to as a bad-bootstrap. Here, we exploit the normative framework of active inference to show that efficient action-oriented models can be learned by balancing goal-oriented and epistemic (information-seeking) behaviours in a principled manner. We illustrate our approach using a simple agent-based model of bacterial chemotaxis. We first demonstrate that learning via goal-directed behaviour indeed constrains models to behaviorally relevant aspects of the environment, but that this approach is prone to sub-optimal convergence. We then demonstrate that epistemic behaviours facilitate the construction of accurate and comprehensive models, but that these models are not tailored to any specific behavioural niche and are therefore less efficient in their use of data. Finally, we show that active inference agents learn models that are parsimonious, tailored to action, and which avoid bad bootstraps and sub-optimal convergence. Critically, our results indicate that models learned through active inference can support adaptive behaviour in spite of, and indeed because of, their departure from veridical representations of the environment. Our approach provides a principled method for learning adaptive models from limited interactions with an environment, highlighting a route to sample efficient learning algorithms
- …