A measure of statistical complexity based on predictive information

We introduce an information theoretic measure of statistical structure, called 'binding information', for sets of random variables, and compare it with several previously proposed measures including excess entropy, Bialek et al.'s predictive information, and the multi-information. We derive some of the properties of the binding information, particularly in relation to the multi-information, and show that, for finite sets of binary random variables, the processes which maximises binding information are the 'parity' processes. Finally we discuss some of the implications this has for the use of the binding information as a measure of complexity.Comment: 4 pages, 3 figure

Analysing Large Scale Structure: I. Weighted Scaling Indices and Constrained Randomisation

The method of constrained randomisation is applied to three-dimensional simulated galaxy distributions. With this technique we generate for a given data set surrogate data sets which have the same linear properties as the original data whereas higher order or nonlinear correlations are not preserved. The analysis of the original and surrogate data sets with measures, which are sensitive to nonlinearities, yields information about the existence of nonlinear correlations in the data. We demonstrate how to generate surrogate data sets from a given point distribution, which have the same linear properties (power spectrum) as well as the same density amplitude distribution. We propose weighted scaling indices as a nonlinear statistical measure to quantify local morphological elements in large scale structure. Using surrogates is is shown that the data sets with the same 2-point correlation functions have slightly different void probability functions and especially a different set of weighted scaling indices. Thus a refined analysis of the large scale structure becomes possible by calculating local scaling properties whereby the method of constrained randomisation yields a vital tool for testing the performance of statistical measures in terms of sensitivity to different topological features and discriminative power.Comment: 10 pages, 5 figures, accepted for publication in MNRA

Describing the complexity of systems: multi-variable "set complexity" and the information basis of systems biology

Context dependence is central to the description of complexity. Keying on the pairwise definition of "set complexity" we use an information theory approach to formulate general measures of systems complexity. We examine the properties of multi-variable dependency starting with the concept of interaction information. We then present a new measure for unbiased detection of multi-variable dependency, "differential interaction information." This quantity for two variables reduces to the pairwise "set complexity" previously proposed as a context-dependent measure of information in biological systems. We generalize it here to an arbitrary number of variables. Critical limiting properties of the "differential interaction information" are key to the generalization. This measure extends previous ideas about biological information and provides a more sophisticated basis for study of complexity. The properties of "differential interaction information" also suggest new approaches to data analysis. Given a data set of system measurements differential interaction information can provide a measure of collective dependence, which can be represented in hypergraphs describing complex system interaction patterns. We investigate this kind of analysis using simulated data sets. The conjoining of a generalized set complexity measure, multi-variable dependency analysis, and hypergraphs is our central result. While our focus is on complex biological systems, our results are applicable to any complex system.Comment: 44 pages, 12 figures; made revisions after peer revie

Generalized compactness in linear spaces and its applications

The class of subsets of locally convex spaces called $\mu$-compact sets is considered. This class contains all compact sets as well as several noncompact sets widely used in applications. It is shown that many results well known for compact sets can be generalized to $\mu$-compact sets. Several examples are considered. The main result of the paper is a generalization to $\mu$-compact convex sets of the Vesterstrom-O'Brien theorem showing equivalence of the particular properties of a compact convex set (s.t. openness of the mixture map, openness of the barycenter map and of its restriction to maximal measures, continuity of a convex hull of any continuous function, continuity of a convex hull of any concave continuous function). It is shown that the Vesterstrom-O'Brien theorem does not hold for pointwise $\mu$-compact convex sets defined by the slight relaxing of the $\mu$-compactness condition. Applications of the obtained results to quantum information theory are considered.Comment: 27 pages, the minor corrections have been mad

SOME TRIGONOMETRIC SIMILARITY MEASURES OF COMPLEX FUZZY SETS WITH APPLICATION

Similarity measures of fuzzy sets are applied to compare the closeness among fuzzy sets. These measures have numerous applications in pattern recognition, image processing, texture synthesis, medical diagnosis, etc. However, in many cases of pattern recognition, digital image processing, signal processing, and so forth, the similarity measures of the fuzzy sets are not appropriate due to the presence of dual information of an object, such as amplitude term and phase term. In these cases, similarity measures of complex fuzzy sets are the most suitable for measuring proximity between objects with two-dimensional information. In the present paper, we propose some trigonometric similarity measures of the complex fuzzy sets involving similarity measures based on the sine, tangent, cosine, and cotangent functions. Furthermore, in many situations in real life, the weight of an attribute plays an important role in making the right decisions using similarity measures. So in this paper, we also consider the weighted trigonometric similarity measures of the complex fuzzy sets, namely, the weighted similarity measures based on the sine, tangent, cosine, and cotangent functions. Some properties of the similarity measures and the weighted similarity measures are discussed. We also apply our proposed methods to the pattern recognition problem and compare them with existing methods to show the validity and effectiveness of our proposed methods