Mutual Dimension

We define the lower and upper mutual dimensions $mdim(x:y)$ and $Mdim(x:y)$ between any two points $x$ and $y$ in Euclidean space. Intuitively these are the lower and upper densities of the algorithmic information shared by $x$ and $y$. We show that these quantities satisfy the main desiderata for a satisfactory measure of mutual algorithmic information. Our main theorem, the data processing inequality for mutual dimension, says that, if $f:\mathbb{R}^m \rightarrow \mathbb{R}^n$ is computable and Lipschitz, then the inequalities $mdim(f(x):y) \leq mdim(x:y)$ and $Mdim(f(x):y) \leq Mdim(x:y)$ hold for all $x \in \mathbb{R}^m$ and $y \in \mathbb{R}^t$. We use this inequality and related inequalities that we prove in like fashion to establish conditions under which various classes of computable functions on Euclidean space preserve or otherwise transform mutual dimensions between points.Comment: This article is 29 pages and has been submitted to ACM Transactions on Computation Theory. A preliminary version of part of this material was reported at the 2013 Symposium on Theoretical Aspects of Computer Science in Kiel, German

The similarity metric

A new class of distances appropriate for measuring similarity relations between sequences, say one type of similarity per distance, is studied. We propose a new ``normalized information distance'', based on the noncomputable notion of Kolmogorov complexity, and show that it is in this class and it minorizes every computable distance in the class (that is, it is universal in that it discovers all computable similarities). We demonstrate that it is a metric and call it the {\em similarity metric}. This theory forms the foundation for a new practical tool. To evidence generality and robustness we give two distinctive applications in widely divergent areas using standard compression programs like gzip and GenCompress. First, we compare whole mitochondrial genomes and infer their evolutionary history. This results in a first completely automatic computed whole mitochondrial phylogeny tree. Secondly, we fully automatically compute the language tree of 52 different languages.Comment: 13 pages, LaTex, 5 figures, Part of this work appeared in Proc. 14th ACM-SIAM Symp. Discrete Algorithms, 2003. This is the final, corrected, version to appear in IEEE Trans Inform. T

Mutual dimension, data processing inequalities, and randomness

This dissertation makes progress in the area of constructive dimension, an effectivization of classical Hausdorff dimension. Using constructive dimension, one may assign a non-zero number to the dimension of individual sequences and individual points in Euclidean space. The primary objective of this dissertation is to develop a framework for mutual dimension, i.e., the density of algorithmic mutual information between two infinite objects, that has similar properties as those of classical Shannon mutual information. Chapter 1 presents a brief history of the development of constructive dimension along with its relationships to algorithmic information theory, algorithmic randomness, and classical Hausdorff dimension. Some applications of this field are discussed and an overview of each subsequent chapter is provided. Chapter 2 defines and analyzes the mutual algorithmic information between two points x and y at a given precision r. In fact, we describe two plausible definitions for this quantity, I_r(x:y) and J_r(x:y), and show that they are closely related. In order to do this, we prove and make use of a generalization of Levin\u27s coding theorem. Chapter 3 defines the lower and upper mutual dimensions between two points in Euclidean space and presents results on its basic properties. A large portion of this chapter is dedicated to studying data processing inequalities for points in Euclidean space. Generally speaking, a data processing inequality says that the amount of information between two objects cannot be significantly increased when one of the objects is processed by a particular type of transformation. We show that it is possible to derive several kinds of data processing inequalities for points in Euclidean space depending on the continuity properties of the computable transformation that is used. Chapter 4 focuses on extending mutual dimension to sequences over an arbitrary alphabet. First, we prove that the mutual dimension between two sequences is equal to the mutual dimension between the sequences\u27 real representations. Using this result, we show that the lower and upper mutual dimensions between sequences have nice properties. We also provide an analysis of data processing inequalities for sequences where transformations are represented by Turing functionals whose use and yield are bounded by computable functions. Chapter 5 relates mutual dimension to the study of algorithmic randomness. Specifically, we show that a particular class of coupled random sequences, i.e., sequences generated by independent tosses of coins whose biases may or may not be correlated, can be characterized by classical Shannon mutual information. We also prove that any two sequences that are independently random with respect to computable probability measures have zero mutual dimension and that the converse of this statement is not true. We conclude this chapter with some initial investigations on Billingsley mutual dimension, i.e., mutual dimension with respect to probability measures, and prove the existence of a mutual divergence formula