Minimum Conditional Description Length Estimation for Markov Random Fields

In this paper we discuss a method, which we call Minimum Conditional Description Length (MCDL), for estimating the parameters of a subset of sites within a Markov random field. We assume that the edges are known for the entire graph $G=(V,E)$. Then, for a subset $U\subset V$, we estimate the parameters for nodes and edges in $U$ as well as for edges incident to a node in $U$, by finding the exponential parameter for that subset that yields the best compression conditioned on the values on the boundary $\partial U$. Our estimate is derived from a temporally stationary sequence of observations on the set $U$. We discuss how this method can also be applied to estimate a spatially invariant parameter from a single configuration, and in so doing, derive the Maximum Pseudo-Likelihood (MPL) estimate.Comment: Information Theory and Applications (ITA) workshop, February 201

Fuzzy transformations and extremality of Gibbs measures for the Potts model on a Cayley tree

We continue our study of the full set of translation-invariant splitting Gibbs measures (TISGMs, translation-invariant tree-indexed Markov chains) for the $q$-state Potts model on a Cayley tree. In our previous work \cite{KRK} we gave a full description of the TISGMs, and showed in particular that at sufficiently low temperatures their number is $2^{q}-1$. In this paper we find some regions for the temperature parameter ensuring that a given TISGM is (non-)extreme in the set of all Gibbs measures. In particular we show the existence of a temperature interval for which there are at least $2^{q-1} + q$ extremal TISGMs. For the Cayley tree of order two we give explicit formulae and some numerical values.Comment: 44 pages. To appear in Random Structures and Algorithm

Differential entropy and time

We give a detailed analysis of the Gibbs-type entropy notion and its dynamical behavior in case of time-dependent continuous probability distributions of varied origins: related to classical and quantum systems. The purpose-dependent usage of conditional Kullback-Leibler and Gibbs (Shannon) entropies is explained in case of non-equilibrium Smoluchowski processes. A very different temporal behavior of Gibbs and Kullback entropies is confronted. A specific conceptual niche is addressed, where quantum von Neumann, classical Kullback-Leibler and Gibbs entropies can be consistently introduced as information measures for the same physical system. If the dynamics of probability densities is driven by the Schr\"{o}dinger picture wave-packet evolution, Gibbs-type and related Fisher information functionals appear to quantify nontrivial power transfer processes in the mean. This observation is found to extend to classical dissipative processes and supports the view that the Shannon entropy dynamics provides an insight into physically relevant non-equilibrium phenomena, which are inaccessible in terms of the Kullback-Leibler entropy and typically ignored in the literature.Comment: Final, unabridged version; http://www.mdpi.org/entropy/ Dedicated to Professor Rafael Sorkin on his 60th birthda

Statistical Mechanics of Surjective Cellular Automata

Reversible cellular automata are seen as microscopic physical models, and their states of macroscopic equilibrium are described using invariant probability measures. We establish a connection between the invariance of Gibbs measures and the conservation of additive quantities in surjective cellular automata. Namely, we show that the simplex of shift-invariant Gibbs measures associated to a Hamiltonian is invariant under a surjective cellular automaton if and only if the cellular automaton conserves the Hamiltonian. A special case is the (well-known) invariance of the uniform Bernoulli measure under surjective cellular automata, which corresponds to the conservation of the trivial Hamiltonian. As an application, we obtain results indicating the lack of (non-trivial) Gibbs or Markov invariant measures for "sufficiently chaotic" cellular automata. We discuss the relevance of the randomization property of algebraic cellular automata to the problem of approach to macroscopic equilibrium, and pose several open questions. As an aside, a shift-invariant pre-image of a Gibbs measure under a pre-injective factor map between shifts of finite type turns out to be always a Gibbs measure. We provide a sufficient condition under which the image of a Gibbs measure under a pre-injective factor map is not a Gibbs measure. We point out a potential application of pre-injective factor maps as a tool in the study of phase transitions in statistical mechanical models.Comment: 50 pages, 7 figure

On the small-scale structure of turbulence and its impact on the pressure field

Understanding the small-scale structure of incompressible turbulence and its implications for the non-local pressure field is one of the fundamental challenges in fluid mechanics. Intense velocity gradient structures tend to cluster on a range of scales which affects the pressure through a Poisson equation. Here we present a quantitative investigation of the spatial distribution of these structures conditional on their intensity for Taylor-based Reynolds numbers in the range [160, 380]. We find that the correlation length, the second invariant of the velocity gradient, is proportional to the Kolmogorov scale. It also is a good indicator for the spatial localization of intense enstrophy and strain-dominated regions, as well as the separation between them. We describe and quantify the differences in the two-point statistics of these regions and the impact they have on the non-locality of the pressure field as a function of the intensity of the regions. Specifically, across the examined range of Reynolds numbers, the pressure in strong rotation-dominated regions is governed by a dissipation-scale neighbourhood. In strong strain-dominated regions, on the other hand, it is determined primarily by a larger neighbourhood reaching inertial scales.Comment: Accepted for publication by the Journal of Fluid Mechanic