Learning action-oriented models through active inference

Converging theories suggest that organisms learn and exploit probabilistic models of their environment. However, it remains unclear how such models can be learned in practice. The open-ended complexity of natural environments means that it is generally infeasible for organisms to model their environment comprehensively. Alternatively, action-oriented models attempt to encode a parsimonious representation of adaptive agent-environment interactions. One approach to learning action-oriented models is to learn online in the presence of goal-directed behaviours. This constrains an agent to behaviourally relevant trajectories, reducing the diversity of the data a model need account for. Unfortunately, this approach can cause models to prematurely converge to sub-optimal solutions, through a process we refer to as a bad-bootstrap. Here, we exploit the normative framework of active inference to show that efficient action-oriented models can be learned by balancing goal-oriented and epistemic (information-seeking) behaviours in a principled manner. We illustrate our approach using a simple agent-based model of bacterial chemotaxis. We first demonstrate that learning via goal-directed behaviour indeed constrains models to behaviorally relevant aspects of the environment, but that this approach is prone to sub-optimal convergence. We then demonstrate that epistemic behaviours facilitate the construction of accurate and comprehensive models, but that these models are not tailored to any specific behavioural niche and are therefore less efficient in their use of data. Finally, we show that active inference agents learn models that are parsimonious, tailored to action, and which avoid bad bootstraps and sub-optimal convergence. Critically, our results indicate that models learned through active inference can support adaptive behaviour in spite of, and indeed because of, their departure from veridical representations of the environment. Our approach provides a principled method for learning adaptive models from limited interactions with an environment, highlighting a route to sample efficient learning algorithms

From Bayesian principles to Bayesian processes

This thesis considers the free energy principle (FEP) and its corollary, active inference, which form an explanatory framework that prescribes a Bayesian interpretation of self-organizing systems. The FEP originated in the domain of neuroscience, where it underwrote a unified theory that described perception, action and learning as emerging from minimizing a single objective function - variational free energy. However, since its conception, the FEP has transcended into physics and pure mathematics. Here, it presents itself as a set of mathematical arguments culminating in an inferential interpretation of a specific class of systems. The result has fundamentally changed the epistemological status of the FEP, moving it from the world of empirical hypotheses to the unfalsifiable territory of mathematical equivalences and tautological constructions. While the FEP may present a historical development that further unravels the symmetries that govern the laws of (our own) physics, its growth has left a range of epistemological confusion. In the current thesis, we evaluate how to maneuver from the principles of the FEP to the processes it purportedly explains. We identify four key areas in which the FEP can inform empirical science: 1) The FEP can aid us in designing intelligent agents by providing novel functionals that respect inherent uncertainty in the environment. We demonstrate equivalences between active inference and reinforcement learning, offer a novel implementation of active inference that utilizes amortized inference, and show that the proposed algorithm enables efficient exploration while offering improved sample efficiency compared to modern reinforcement learning algorithms. 2) We describe how the FEP can help us understand the nature of representation in living systems. Specifically, we show how the normative aspects of the FEP promote learning representations oriented towards action rather than veridical reconstructions of the environment. 3) We show how the FEP provides a framework for modeling perception, action, and learning in systems that can be empirically measured. An eye-tracking study demonstrates that an active inference model best explains human information-seeking, offering insights into the underlying mechanisms of perception and action. 4) In the final section, we ask whether active inference can inform the development of novel process theories in computational neuroscience. A biologically-plausible learning algorithm is developed and verified on various computer vision and reinforcement learning tasks. The resulting model explains a range of empirical phenomena and offers a new perspective on the role of bottom-up information in perception. This thesis affirms the role of the FEP and active inference as a generative framework for developing testable scientific theories

The signature of cusped hyperbolic 4-manifolds

In this note we show that every integer is the signature of a non-compact, oriented, hyperbolic 4-manifold of finite volume, and give some partial results on the geography of such manifolds. The main ingredients are a theorem of Long and Reid, and the explicit construction of a hyperbolic 24-cell manifold with some special topological properties.Comment: 13 pages, 4 figures, 7 table