Finite-State Dimension and Real Arithmetic

We use entropy rates and Schur concavity to prove that, for every integer k >= 2, every nonzero rational number q, and every real number alpha, the base-k expansions of alpha, q+alpha, and q*alpha all have the same finite-state dimension and the same finite-state strong dimension. This extends, and gives a new proof of, Wall's 1949 theorem stating that the sum or product of a nonzero rational number and a Borel normal number is always Borel normal.Comment: 15 page

A Divergence Formula for Randomness and Dimension (Short Version)

If $S$ is an infinite sequence over a finite alphabet $\Sigma$ and $\beta$ is a probability measure on $\Sigma$, then the {\it dimension} of $S$ with respect to $\beta$, written $\dim^\beta(S)$, is a constructive version of Billingsley dimension that coincides with the (constructive Hausdorff) dimension $\dim(S)$ when $\beta$ is the uniform probability measure. This paper shows that $\dim^\beta(S)$ and its dual \Dim^\beta(S), the {\it strong dimension} of $S$ with respect to $\beta$, can be used in conjunction with randomness to measure the similarity of two probability measures $\alpha$ and $\beta$ on $\Sigma$. Specifically, we prove that the {\it divergence formula} \dim^\beta(R) = \Dim^\beta(R) =\CH(\alpha) / (\CH(\alpha) + \D(\alpha || \beta)) holds whenever $\alpha$ and $\beta$ are computable, positive probability measures on $\Sigma$ and $R \in \Sigma^\infty$ is random with respect to $\alpha$. In this formula, \CH(\alpha) is the Shannon entropy of $\alpha$, and \D(\alpha||\beta) is the Kullback-Leibler divergence between $\alpha$ and $\beta$

Dimensions of Copeland-Erdos Sequences

The base-$k$ {\em Copeland-Erd\"os sequence} given by an infinite set $A$ of positive integers is the infinite sequence \CE_k(A) formed by concatenating the base-$k$ representations of the elements of $A$ in numerical order. This paper concerns the following four quantities. The {\em finite-state dimension} \dimfs (\CE_k(A)), a finite-state version of classical Hausdorff dimension introduced in 2001. The {\em finite-state strong dimension} \Dimfs(\CE_k(A)), a finite-state version of classical packing dimension introduced in 2004. This is a dual of \dimfs(\CE_k(A)) satisfying \Dimfs(\CE_k(A)) \geq \dimfs(\CE_k(A)). The {\em zeta-dimension} \Dimzeta(A), a kind of discrete fractal dimension discovered many times over the past few decades. The {\em lower zeta-dimension} \dimzeta(A), a dual of \Dimzeta(A) satisfying \dimzeta(A)\leq \Dimzeta(A). We prove the following. \dimfs(\CE_k(A))\geq \dimzeta(A). This extends the 1946 proof by Copeland and Erd\"os that the sequence \CE_k(\mathrm{PRIMES}) is Borel normal. \Dimfs(\CE_k(A))\geq \Dimzeta(A). These bounds are tight in the strong sense that these four quantities can have (simultaneously) any four values in $[0,1]$ satisfying the four above-mentioned inequalities.Comment: 19 page

Finite-State Dimension and Lossy Decompressors

This paper examines information-theoretic questions regarding the difficulty of compressing data versus the difficulty of decompressing data and the role that information loss plays in this interaction. Finite-state compression and decompression are shown to be of equivalent difficulty, even when the decompressors are allowed to be lossy. Inspired by Kolmogorov complexity, this paper defines the optimal *decompression *ratio achievable on an infinite sequence by finite-state decompressors (that is, finite-state transducers outputting the sequence in question). It is shown that the optimal compression ratio achievable on a sequence S by any *information lossless* finite state compressor, known as the finite-state dimension of S, is equal to the optimal decompression ratio achievable on S by any finite-state decompressor. This result implies a new decompression characterization of finite-state dimension in terms of lossy finite-state transducers.Comment: We found that Theorem 3.11, which was basically the motive for this paper, was already proven by Sheinwald, Ziv, and Lempel in 1991 and 1995 paper

Constructive Dimension and Turing Degrees

This paper examines the constructive Hausdorff and packing dimensions of Turing degrees. The main result is that every infinite sequence S with constructive Hausdorff dimension dim_H(S) and constructive packing dimension dim_P(S) is Turing equivalent to a sequence R with dim_H(R) <= (dim_H(S) / dim_P(S)) - epsilon, for arbitrary epsilon > 0. Furthermore, if dim_P(S) > 0, then dim_P(R) >= 1 - epsilon. The reduction thus serves as a *randomness extractor* that increases the algorithmic randomness of S, as measured by constructive dimension. A number of applications of this result shed new light on the constructive dimensions of Turing degrees. A lower bound of dim_H(S) / dim_P(S) is shown to hold for the Turing degree of any sequence S. A new proof is given of a previously-known zero-one law for the constructive packing dimension of Turing degrees. It is also shown that, for any regular sequence S (that is, dim_H(S) = dim_P(S)) such that dim_H(S) > 0, the Turing degree of S has constructive Hausdorff and packing dimension equal to 1. Finally, it is shown that no single Turing reduction can be a universal constructive Hausdorff dimension extractor, and that bounded Turing reductions cannot extract constructive Hausdorff dimension. We also exhibit sequences on which weak truth-table and bounded Turing reductions differ in their ability to extract dimension.Comment: The version of this paper appearing in Theory of Computing Systems, 45(4):740-755, 2009, had an error in the proof of Theorem 2.4, due to insufficient care with the choice of delta. This version modifies that proof to fix the error

A Divergence Formula for Randomness and Dimension

If $S$ is an infinite sequence over a finite alphabet $\Sigma$ and $\beta$ is a probability measure on $\Sigma$, then the {\it dimension} of $S$ with respect to $\beta$, written $\dim^\beta(S)$, is a constructive version of Billingsley dimension that coincides with the (constructive Hausdorff) dimension $\dim(S)$ when $\beta$ is the uniform probability measure. This paper shows that $\dim^\beta(S)$ and its dual \Dim^\beta(S), the {\it strong dimension} of $S$ with respect to $\beta$, can be used in conjunction with randomness to measure the similarity of two probability measures $\alpha$ and $\beta$ on $\Sigma$. Specifically, we prove that the {\it divergence formula} \dim^\beta(R) = \Dim^\beta(R) =\frac{\CH(\alpha)}{\CH(\alpha) + \D(\alpha || \beta)} holds whenever $\alpha$ and $\beta$ are computable, positive probability measures on $\Sigma$ and $R \in \Sigma^\infty$ is random with respect to $\alpha$. In this formula, \CH(\alpha) is the Shannon entropy of $\alpha$, and \D(\alpha||\beta) is the Kullback-Leibler divergence between $\alpha$ and $\beta$. We also show that the above formula holds for all sequences $R$ that are $\alpha$-normal (in the sense of Borel) when $\dim^\beta(R)$ and \Dim^\beta(R) are replaced by the more effective finite-state dimensions \dimfs^\beta(R) and \Dimfs^\beta(R). In the course of proving this, we also prove finite-state compression characterizations of \dimfs^\beta(S) and \Dimfs^\beta(S).Comment: 18 page

Dimensions of Copeland-Erdos Sequences

The base-k Copeland-Erdös sequence given by an infinite set A of positive integers is the infinite sequence CEk(A) formed by concatenating the base-k representations of the elements of A in numerical order. This paper concerns the following four quantities. • The finite-state dimension dimFS(CEk(A)), a finite-state version of classical Hausdorff dimension introduced in 2001. • The finite-state strong dimension DimFS(CEk(A)), a finite-state version of classical packing dimension introduced in 2004. This is a dual of dimFS(CEk(A)) satisfying DimFS(CEk(A)) ≥ dimFS(CEk(A)). • The zeta-dimension Dimζ(A), a kind of discrete fractal dimension discovered many times over the past few decades. • The lower zeta-dimension dimζ(A), a dual of Dimζ(A) satisfying dimζ(A) ≤ Dimζ(A). We prove the following. 1. dimFS(CEk(A)) ≥ dimζ(A). This extends the 1946 proof by Copeland and Erdös that the sequence CEk(PRIMES) is Borel normal. 2. DimFS(CEk(A)) ≥ Dimζ(A). 3. These bounds are tight in the strong sense that these four quantities can have (simultane-ously) any four values in [0, 1] satisfying the four above-mentioned inequalities

Dimension, Pseudorandomness and Extraction of Pseudorandomness

In this paper we propose a quantification of distributions on a set of strings, in terms of how close to pseudorandom a distribution is. The quantification is an adaptation of the theory of dimension of sets of infinite sequences introduced by Lutz. Adapting Hitchcock\u27s work, we also show that the logarithmic loss incurred by a predictor on a distribution is quantitatively equivalent to the notion of dimension we define. Roughly, this captures the equivalence between pseudorandomness defined via indistinguishability and via unpredictability. Later we show some natural properties of our notion of dimension. We also do a comparative study among our proposed notion of dimension and two well known notions of computational analogue of entropy, namely HILL-type pseudo min-entropy and next-bit pseudo Shannon entropy. Further, we apply our quantification to the following problem. If we know that the dimension of a distribution on the set of n-length strings is s in (0,1], can we extract out O(sn) pseudorandom bits out of the distribution? We show that to construct such extractor, one need at least Omega(log n) bits of pure randomness. However, it is still open to do the same using O(log n) random bits. We show that deterministic extraction is possible in a special case - analogous to the bit-fixing sources introduced by Chor et al., which we term nonpseudorandom bit-fixing source. We adapt the techniques of Gabizon, Raz and Shaltiel to construct a deterministic pseudorandom extractor for this source. By the end, we make a little progress towards P vs. BPP problem by showing that existence of optimal stretching function that stretches O(log n) input bits to produce n output bits such that output distribution has dimension s in (0,1], implies P=BPP

Dimension Extractors and Optimal Decompression

A *dimension extractor* is an algorithm designed to increase the effective dimension -- i.e., the amount of computational randomness -- of an infinite binary sequence, in order to turn a "partially random" sequence into a "more random" sequence. Extractors are exhibited for various effective dimensions, including constructive, computable, space-bounded, time-bounded, and finite-state dimension. Using similar techniques, the Kucera-Gacs theorem is examined from the perspective of decompression, by showing that every infinite sequence S is Turing reducible to a Martin-Loef random sequence R such that the asymptotic number of bits of R needed to compute n bits of S, divided by n, is precisely the constructive dimension of S, which is shown to be the optimal ratio of query bits to computed bits achievable with Turing reductions. The extractors and decompressors that are developed lead directly to new characterizations of some effective dimensions in terms of optimal decompression by Turing reductions.Comment: This report was combined with a different conference paper "Every Sequence is Decompressible from a Random One" (cs.IT/0511074, at http://dx.doi.org/10.1007/11780342_17), and both titles were changed, with the conference paper incorporated as section 5 of this new combined paper. The combined paper was accepted to the journal Theory of Computing Systems, as part of a special issue of invited papers from the second conference on Computability in Europe, 200