Information in statistical physics

We review with a tutorial scope the information theory foundations of quantum statistical physics. Only a small proportion of the variables that characterize a system at the microscopic scale can be controlled, for both practical and theoretical reasons, and a probabilistic description involving the observers is required. The criterion of maximum von Neumann entropy is then used for making reasonable inferences. It means that no spurious information is introduced besides the known data. Its outcomes can be given a direct justification based on the principle of indifference of Laplace. We introduce the concept of relevant entropy associated with some set of relevant variables; it characterizes the information that is missing at the microscopic level when only these variables are known. For equilibrium problems, the relevant variables are the conserved ones, and the Second Law is recovered as a second step of the inference process. For non-equilibrium problems, the increase of the relevant entropy expresses an irretrievable loss of information from the relevant variables towards the irrelevant ones. Two examples illustrate the flexibility of the choice of relevant variables and the multiplicity of the associated entropies: the thermodynamic entropy (satisfying the Clausius-Duhem inequality) and the Boltzmann entropy (satisfying the H-theorem). The identification of entropy with missing information is also supported by the paradox of Maxwell's demon. Spin-echo experiments show that irreversibility itself is not an absolute concept: use of hidden information may overcome the arrow of time.Comment: latex InfoStatPhys-unix.tex, 3 files, 2 figures, 32 pages http://www-spht.cea.fr/articles/T04/18

Information theoretic thresholding techniques based on particle swarm optimization.

In this dissertation, we discuss multi-level image thresholding techniques based on information theoretic entropies. In order to apply the correlation information of neighboring pixels of an image to obtain better segmentation results, we propose several multi-level thresholding models by using Gray-Level & Local-Average histogram (GLLA) and Gray-Level & Local-Variance histogram (GLLV). Firstly, a RGB color image thresholding model based on GLLA histogram and Tsallis-Havrda-Charv\u27at entropy is discussed. We validate the multi-level thresholding criterion function by using mathematical induction. For each component image, we assign the mean value from each thresholded class to obtain three segmented component images independently. Then we obtain the segmented color image by combining the three segmented component images. Secondly, we use the GLLV histogram to propose three novel entropic multi-level thresholding models based on Shannon entropy, R\u27enyi entropy and Tsallis-Havrda-Charv\u27at entropy respectively. Then we apply these models on the three components of a RGB color image to complete the RGB color image segmentation. An entropic thresholding model is mostly about searching for the optimal threshold values by maximizing or minimizing a criterion function. We apply particle swarm optimization (PSO) algorithm to search the optimal threshold values for all the models. We conduct the experiments extensively on The Berkeley Segmentation Dataset and Benchmark (BSDS300) and calculate the average four performance indices (Probability Rand Index, PRI, Global Consistency Error, GCE, Variation of Information, VOI and Boundary Displacement Error, BDE) to show the effectiveness and reasonability of the proposed models

Information measures in distributed multitarget tracking

In this paper, we consider the role that different information measures play in the problem of decentralised multi-target tracking. In many sensor networks, it is not possible to maintain the full joint probability distribution and so suboptimal algorithms must be used. We use a distributed form of the Probability Hypothesis Density (PHD) filter based on a generalisation of covariance intersection known as exponential mixture densities (EMDs). However, EMD-based fusion must be actively controlled to optimise the relative weights placed on different information sources. We explore the performance consequences of using different information measures to optimise the update. By considering approaches that minimise absolute information (entropy and Rényi entropy) or equalise divergence (Kullback-Leibler Divergence and Rényi Divergence), we show that the divergence measures are both simpler and easier to work with. Furthermore, in our simulation scenario, the performance is very similar with all the information measures considered, suggesting that the simpler measures can be used. © 2011 IEEE

Relaxed plasma equilibria and entropy-related plasma self-organization principles

The concept of plasma relaxation as a constrained energy minimization is reviewed. Recent work by the authors on generalizing this approach to partially relaxed threedimensional plasma systems in a way consistent with chaos theory is discussed, with a view to clarifying the thermodynamic aspects of the variational approach used. Other entropy-related approaches to finding long-time steady states of turbulent or chaotic plasma systems are also briefly reviewed

Digital Alchemy for Materials Design: Colloids and Beyond

Starting with the early alchemists, a holy grail of science has been to make desired materials by modifying the attributes of basic building blocks. Building blocks that show promise for assembling new complex materials can be synthesized at the nanoscale with attributes that would astonish the ancient alchemists in their versatility. However, this versatility means that making direct connection between building block attributes and bulk behavior is both necessary for rationally engineering materials, and difficult because building block attributes can be altered in many ways. Here we show how to exploit the malleability of the valence of colloidal nanoparticle "elements" to directly and quantitatively link building block attributes to bulk behavior through a statistical thermodynamic framework we term "digital alchemy". We use this framework to optimize building blocks for a given target structure, and to determine which building block attributes are most important to control for self assembly, through a set of novel thermodynamic response functions, moduli and susceptibilities. We thereby establish direct links between the attributes of colloidal building blocks and the bulk structures they form. Moreover, our results give concrete solutions to the more general conceptual challenge of optimizing emergent behaviors in nature, and can be applied to other types of matter. As examples, we apply digital alchemy to systems of truncated tetrahedra, rhombic dodecahedra, and isotropically interacting spheres that self assemble diamond, FCC, and icosahedral quasicrystal structures, respectively.Comment: 17 REVTeX pages, title fixed to match journal versio

AN INFORMATION THEORETIC APPROACH TO INTERACTING MULTIPLE MODEL ESTIMATION FOR AUTONOMOUS UNDERWATER VEHICLES

Accurate and robust autonomous underwater navigation (AUV) requires the fundamental task of position estimation in a variety of conditions. Additionally, the U.S. Navy would prefer to have systems that are not dependent on external beacon systems such as global positioning system (GPS), since they are subject to jamming and spoofing and can reduce operational effectiveness. Current methodologies such as Terrain-Aided Navigation (TAN) use exteroceptive imaging sensors for building a local reference position estimate and will not be useful when those sensors are out of range. What is needed are multiple navigation filters where each can be more effective depending on the mission conditions. This thesis investigates how to combine multiple navigation filters to provide a more robust AUV position estimate. The solution presented is to blend two different filtering methodologies utilizing an interacting multiple model (IMM) estimation approach based on an information theoretic framework. The first filter is a model-based Extended Kalman Filter (EKF) that is effective under dead reckoning (DR) conditions. The second is a Particle Filter approach for Active Terrain Aided Navigation (ATAN) that is appropriate when in sensor range. Using data collected at Lake Crescent, Washington, each of the navigation filters are developed with results and then we demonstrate how an IMM information theoretic approach can be used to blend approaches to improve position and orientation estimation.Lieutenant, United States NavyApproved for public release. Distribution is unlimited

The Zeroth Law of Thermodynamics and Volume-Preserving Conservative Dynamics with Equilibrium Stochastic Damping

We propose a mathematical formulation of the zeroth law of thermodynamics and develop a stochastic dynamical theory, with a consistent irreversible thermodynamics, for systems possessing sustained conservative stationary current in phase space while in equilibrium with a heat bath. The theory generalizes underdamped mechanical equilibrium: $dx=gdt+\{-D\nabla\phi dt+\sqrt{2D}dB(t)\}$, with $\nabla\cdot g=0$ and $\{\cdots\}$ respectively representing phase-volume preserving dynamics and stochastic damping. The zeroth law implies stationary distribution $u^{ss}(x)=e^{-\phi(x)}$. We find an orthogonality $\nabla\phi\cdot g=0$ as a hallmark of the system. Stochastic thermodynamics based on time reversal $\big(t,\phi,g\big)\rightarrow\big(-t,\phi,-g\big)$ is formulated: entropy production $e_p^{\#}(t)=-dF(t)/dt$; generalized "heat" $h_d^{\#}(t)=-dU(t)/dt$, $U(t)=\int_{\mathbb{R}^n} \phi(x)u(x,t)dx$ being "internal energy", and "free energy" $F(t)=U(t)+\int_{\mathbb{R}^n} u(x,t)\ln u(x,t)dx$ never increases. Entropy follows $\frac{dS}{dt}=e_p^{\#}-h_d^{\#}$. Our formulation is shown to be consistent with an earlier theory of P. Ao. Its contradistinctions to other theories, potential-flux decomposition, stochastic Hamiltonian system with even and odd variables, Klein-Kramers equation, Freidlin-Wentzell's theory, and GENERIC, are discussed.Comment: 25 page