Learning action-oriented models through active inference

Converging theories suggest that organisms learn and exploit probabilistic models of their environment. However, it remains unclear how such models can be learned in practice. The open-ended complexity of natural environments means that it is generally infeasible for organisms to model their environment comprehensively. Alternatively, action-oriented models attempt to encode a parsimonious representation of adaptive agent-environment interactions. One approach to learning action-oriented models is to learn online in the presence of goal-directed behaviours. This constrains an agent to behaviourally relevant trajectories, reducing the diversity of the data a model need account for. Unfortunately, this approach can cause models to prematurely converge to sub-optimal solutions, through a process we refer to as a bad-bootstrap. Here, we exploit the normative framework of active inference to show that efficient action-oriented models can be learned by balancing goal-oriented and epistemic (information-seeking) behaviours in a principled manner. We illustrate our approach using a simple agent-based model of bacterial chemotaxis. We first demonstrate that learning via goal-directed behaviour indeed constrains models to behaviorally relevant aspects of the environment, but that this approach is prone to sub-optimal convergence. We then demonstrate that epistemic behaviours facilitate the construction of accurate and comprehensive models, but that these models are not tailored to any specific behavioural niche and are therefore less efficient in their use of data. Finally, we show that active inference agents learn models that are parsimonious, tailored to action, and which avoid bad bootstraps and sub-optimal convergence. Critically, our results indicate that models learned through active inference can support adaptive behaviour in spite of, and indeed because of, their departure from veridical representations of the environment. Our approach provides a principled method for learning adaptive models from limited interactions with an environment, highlighting a route to sample efficient learning algorithms

Use of the geometric mean as a statistic for the scale of the coupled Gaussian distributions

The geometric mean is shown to be an appropriate statistic for the scale of a heavy-tailed coupled Gaussian distribution or equivalently the Student's t distribution. The coupled Gaussian is a member of a family of distributions parameterized by the nonlinear statistical coupling which is the reciprocal of the degree of freedom and is proportional to fluctuations in the inverse scale of the Gaussian. Existing estimators of the scale of the coupled Gaussian have relied on estimates of the full distribution, and they suffer from problems related to outliers in heavy-tailed distributions. In this paper, the scale of a coupled Gaussian is proven to be equal to the product of the generalized mean and the square root of the coupling. From our numerical computations of the scales of coupled Gaussians using the generalized mean of random samples, it is indicated that only samples from a Cauchy distribution (with coupling parameter one) form an unbiased estimate with diminishing variance for large samples. Nevertheless, we also prove that the scale is a function of the geometric mean, the coupling term and a harmonic number. Numerical experiments show that this estimator is unbiased with diminishing variance for large samples for a broad range of coupling values.Comment: 17 pages, 5 figure

Can anticipatory feelings explain anomalous choices of information sources?

The well-being of agents is often directly affected by their beliefs, in the form of anticipatory feelings such as anxiety and hopefulness. Economists have tried to model this effect by introducing beliefs as arguments in decision makers' vNM utility function. One might expect that such a model would be capable of explaining anomalous attitudes to information that we observe in reality. We show that the model has several shortcomings in this regard, as long as Bayesian updating is retained. (c) 2005 Elsevier Inc. All rights reserved

Nonparametric Methods in Astronomy: Think, Regress, Observe -- Pick Any Three

Telescopes are much more expensive than astronomers, so it is essential to minimize required sample sizes by using the most data-efficient statistical methods possible. However, the most commonly used model-independent techniques for finding the relationship between two variables in astronomy are flawed. In the worst case they can lead without warning to subtly yet catastrophically wrong results, and even in the best case they require more data than necessary. Unfortunately, there is no single best technique for nonparametric regression. Instead, we provide a guide for how astronomers can choose the best method for their specific problem and provide a python library with both wrappers for the most useful existing algorithms and implementations of two new algorithms developed here.Comment: 19 pages, PAS

A Conceptual Model of Investor Behavior

Based on a survey of behavioral finance literature, this paper presents a descriptive model of individual investor behavior in which investment decisions are seen as an iterative process of interactions between the investor and the investment environment. This investment process is influenced by a number of interdependent variables and driven by dual mental systems, the interplay of which contributes to boundedly rational behavior where investors use various heuristics and may exhibit behavioral biases. In the modeling tradition of cognitive science and intelligent systems, the investor is seen as a learning, adapting, and evolving entity that perceives the environment, processes information, acts upon it, and updates his or her internal states. This conceptual model can be used to build stylized representations of (classes of) individual investors, and further studied using the paradigm of agent-based artificial financial markets. By allowing us to implement individual investor behavior, to choose various market mechanisms, and to analyze the obtained asset prices, agent-based models can bridge the gap between the micro level of individual investor behavior and the macro level of aggregate market phenomena. It has been recognized, yet not fully explored, that these models could be used as a tool to generate or test various behavioral hypothesis.behavioral finance;financial decision making;agent-based artificial financial markets;cognitive modeling;investor behavior