Minimum Description Length Induction, Bayesianism, and Kolmogorov Complexity

The relationship between the Bayesian approach and the minimum description length approach is established. We sharpen and clarify the general modeling principles MDL and MML, abstracted as the ideal MDL principle and defined from Bayes's rule by means of Kolmogorov complexity. The basic condition under which the ideal principle should be applied is encapsulated as the Fundamental Inequality, which in broad terms states that the principle is valid when the data are random, relative to every contemplated hypothesis and also these hypotheses are random relative to the (universal) prior. Basically, the ideal principle states that the prior probability associated with the hypothesis should be given by the algorithmic universal probability, and the sum of the log universal probability of the model plus the log of the probability of the data given the model should be minimized. If we restrict the model class to the finite sets then application of the ideal principle turns into Kolmogorov's minimal sufficient statistic. In general we show that data compression is almost always the best strategy, both in hypothesis identification and prediction.Comment: 35 pages, Latex. Submitted IEEE Trans. Inform. Theor

Asymmetry of the Kolmogorov complexity of online predicting odd and even bits

Symmetry of information states that $C(x) + C(y|x) = C(x,y) + O(\log C(x))$. We show that a similar relation for online Kolmogorov complexity does not hold. Let the even (online Kolmogorov) complexity of an n-bitstring $x_1x_2... x_n$ be the length of a shortest program that computes $x_2$ on input $x_1$, computes $x_4$ on input $x_1x_2x_3$, etc; and similar for odd complexity. We show that for all n there exist an n-bit x such that both odd and even complexity are almost as large as the Kolmogorov complexity of the whole string. Moreover, flipping odd and even bits to obtain a sequence $x_2x_1x_4x_3\ldots$, decreases the sum of odd and even complexity to $C(x)$.Comment: 20 pages, 7 figure

Kolmogorov complexity spectrum for use in analysis of UV-B radiation time series

We have used the Kolmogorov complexity and sample entropy measures to estimate the complexity of the UV-B radiation time series in the Vojvodina region (Serbia) for the period 1990-2007. We defined the Kolmogorov complexity spectrum and have introduced the Kolmogorov complexity spectrum highest value (KLM). We have established the UV-B radiation time series on the basis of their daily sum (dose) for seven representative places in this region using (i) measured data, (ii) data calculated via a derived empirical formula and (iii) data obtained by a parametric UV radiation model. We have calculated the Kolmogorov complexity (KL) based on the Lempel-Ziv Algorithm (LZA), KLM and Sample Entropy (SE) values for each time series. We have divided the period 1990-2007 into two sub-intervals: (a) 1990-1998 and (b)1999-2007 and calculated the KL, KLM and SE values for the various time series in these sub-intervals. It is found that during the period 1999-2007, there is a decrease in the KL, KLM and SE, comparing to the period 1990-1998. This complexity loss may be attributed to (i) the increased human intervention in the post civil war period causing increase of the air pollution and (ii) the increased cloudiness due to climate changes.Comment: 10 pages, 1 figure, 1 table. arXiv admin note: substantial text overlap with arXiv:1301.2039; This paper has been accepted in Modern Physics Letters B on Aug 14, 201

On Generalized Computable Universal Priors and their Convergence

Solomonoff unified Occam's razor and Epicurus' principle of multiple explanations to one elegant, formal, universal theory of inductive inference, which initiated the field of algorithmic information theory. His central result is that the posterior of the universal semimeasure M converges rapidly to the true sequence generating posterior mu, if the latter is computable. Hence, M is eligible as a universal predictor in case of unknown mu. The first part of the paper investigates the existence and convergence of computable universal (semi)measures for a hierarchy of computability classes: recursive, estimable, enumerable, and approximable. For instance, M is known to be enumerable, but not estimable, and to dominate all enumerable semimeasures. We present proofs for discrete and continuous semimeasures. The second part investigates more closely the types of convergence, possibly implied by universality: in difference and in ratio, with probability 1, in mean sum, and for Martin-Loef random sequences. We introduce a generalized concept of randomness for individual sequences and use it to exhibit difficulties regarding these issues. In particular, we show that convergence fails (holds) on generalized-random sequences in gappy (dense) Bernoulli classes.Comment: 22 page