A Tutorial on Fisher Information

In many statistical applications that concern mathematical psychologists, the concept of Fisher information plays an important role. In this tutorial we clarify the concept of Fisher information as it manifests itself across three different statistical paradigms. First, in the frequentist paradigm, Fisher information is used to construct hypothesis tests and confidence intervals using maximum likelihood estimators; second, in the Bayesian paradigm, Fisher information is used to define a default prior; lastly, in the minimum description length paradigm, Fisher information is used to measure model complexity

Psychophysical identity and free energy

An approach to implementing variational Bayesian inference in biological systems is considered, under which the thermodynamic free energy of a system directly encodes its variational free energy. In the case of the brain, this assumption places constraints on the neuronal encoding of generative and recognition densities, in particular requiring a stochastic population code. The resulting relationship between thermodynamic and variational free energies is prefigured in mind-brain identity theses in philosophy and in the Gestalt hypothesis of psychophysical isomorphism.Comment: 22 pages; published as a research article on 8/5/2020 in Journal of the Royal Society Interfac

The Price equation program: simple invariances unify population dynamics, thermodynamics, probability, information and inference

The fundamental equations of various disciplines often seem to share the same basic structure. Natural selection increases information in the same way that Bayesian updating increases information. Thermodynamics and the forms of common probability distributions express maximum increase in entropy, which appears mathematically as loss of information. Physical mechanics follows paths of change that maximize Fisher information. The information expressions typically have analogous interpretations as the Newtonian balance between force and acceleration, representing a partition between direct causes of change and opposing changes in the frame of reference. This web of vague analogies hints at a deeper common mathematical structure. I suggest that the Price equation expresses that underlying universal structure. The abstract Price equation describes dynamics as the change between two sets. One component of dynamics expresses the change in the frequency of things, holding constant the values associated with things. The other component of dynamics expresses the change in the values of things, holding constant the frequency of things. The separation of frequency from value generalizes Shannon's separation of the frequency of symbols from the meaning of symbols in information theory. The Price equation's generalized separation of frequency and value reveals a few simple invariances that define universal geometric aspects of change. For example, the conservation of total frequency, although a trivial invariance by itself, creates a powerful constraint on the geometry of change. That constraint plus a few others seem to explain the common structural forms of the equations in different disciplines. From that abstract perspective, interpretations such as selection, information, entropy, force, acceleration, and physical work arise from the same underlying geometry expressed by the Price equation.Comment: Version 3: added figure illustrating geometry; added table of symbols and two tables summarizing mathematical relations; this version accepted for publication in Entrop

Equations defining probability tree models

Coloured probability tree models are statistical models coding conditional independence between events depicted in a tree graph. They are more general than the very important class of context-specific Bayesian networks. In this paper, we study the algebraic properties of their ideal of model invariants. The generators of this ideal can be easily read from the tree graph and have a straightforward interpretation in terms of the underlying model: they are differences of odds ratios coming from conditional probabilities. One of the key findings in this analysis is that the tree is a convenient tool for understanding the exact algebraic way in which the sum-to-1 conditions on the parameter space translate into the sum-to-one conditions on the joint probabilities of the statistical model. This enables us to identify necessary and sufficient graphical conditions for a staged tree model to be a toric variety intersected with a probability simplex.Comment: 22 pages, 4 figure

Bayesian non-linear large scale structure inference of the Sloan Digital Sky Survey data release 7

In this work we present the first non-linear, non-Gaussian full Bayesian large scale structure analysis of the cosmic density field conducted so far. The density inference is based on the Sloan Digital Sky Survey data release 7, which covers the northern galactic cap. We employ a novel Bayesian sampling algorithm, which enables us to explore the extremely high dimensional non-Gaussian, non-linear log-normal Poissonian posterior of the three dimensional density field conditional on the data. These techniques are efficiently implemented in the HADES computer algorithm and permit the precise recovery of poorly sampled objects and non-linear density fields. The non-linear density inference is performed on a 750 Mpc cube with roughly 3 Mpc grid-resolution, while accounting for systematic effects, introduced by survey geometry and selection function of the SDSS, and the correct treatment of a Poissonian shot noise contribution. Our high resolution results represent remarkably well the cosmic web structure of the cosmic density field. Filaments, voids and clusters are clearly visible. Further, we also conduct a dynamical web classification, and estimated the web type posterior distribution conditional on the SDSS data.Comment: 18 pages, 11 figure