Minimum Description Length Induction, Bayesianism, and Kolmogorov Complexity

The relationship between the Bayesian approach and the minimum description length approach is established. We sharpen and clarify the general modeling principles MDL and MML, abstracted as the ideal MDL principle and defined from Bayes's rule by means of Kolmogorov complexity. The basic condition under which the ideal principle should be applied is encapsulated as the Fundamental Inequality, which in broad terms states that the principle is valid when the data are random, relative to every contemplated hypothesis and also these hypotheses are random relative to the (universal) prior. Basically, the ideal principle states that the prior probability associated with the hypothesis should be given by the algorithmic universal probability, and the sum of the log universal probability of the model plus the log of the probability of the data given the model should be minimized. If we restrict the model class to the finite sets then application of the ideal principle turns into Kolmogorov's minimal sufficient statistic. In general we show that data compression is almost always the best strategy, both in hypothesis identification and prediction.Comment: 35 pages, Latex. Submitted IEEE Trans. Inform. Theor

Reactive Turing Machines

We propose reactive Turing machines (RTMs), extending classical Turing machines with a process-theoretical notion of interaction, and use it to define a notion of executable transition system. We show that every computable transition system with a bounded branching degree is simulated modulo divergence-preserving branching bisimilarity by an RTM, and that every effective transition system is simulated modulo the variant of branching bisimilarity that does not require divergence preservation. We conclude from these results that the parallel composition of (communicating) RTMs can be simulated by a single RTM. We prove that there exist universal RTMs modulo branching bisimilarity, but these essentially employ divergence to be able to simulate an RTM of arbitrary branching degree. We also prove that modulo divergence-preserving branching bisimilarity there are RTMs that are universal up to their own branching degree. Finally, we establish a correspondence between executability and finite definability in a simple process calculus

Changing a semantics: opportunism or courage?

The generalized models for higher-order logics introduced by Leon Henkin, and their multiple offspring over the years, have become a standard tool in many areas of logic. Even so, discussion has persisted about their technical status, and perhaps even their conceptual legitimacy. This paper gives a systematic view of generalized model techniques, discusses what they mean in mathematical and philosophical terms, and presents a few technical themes and results about their role in algebraic representation, calibrating provability, lowering complexity, understanding fixed-point logics, and achieving set-theoretic absoluteness. We also show how thinking about Henkin's approach to semantics of logical systems in this generality can yield new results, dispelling the impression of adhocness. This paper is dedicated to Leon Henkin, a deep logician who has changed the way we all work, while also being an always open, modest, and encouraging colleague and friend.Comment: 27 pages. To appear in: The life and work of Leon Henkin: Essays on his contributions (Studies in Universal Logic) eds: Manzano, M., Sain, I. and Alonso, E., 201

On the Structure of Solutions of Computable Real Functions

The relationship between the structure of a domain and the complexity of computing over that domain is a fundamental question of computer science. This paper studies how the structure of the real numbers constrains the behavior of computable real functions. In particular, we uncover a close correlation between the structure of the zero set of a computable real function, and the complexity of the zeros. We show that computable real functions with hard solutions perforce have many solutions. Furthermore, as the complexity of solutions increases, the number of solutions increases. We prove that computable real functions with nonrecursive, nonarithmetical, or random zeros have solution sets that are, respectively, infinite,“˜ uncountable, or of positive measure. In addition, we show that the computational complexity of the zero set of a computable real function is limited by its topological complexity. These results suggest an emerging paradigm-the inability of machines to name complex strings can serve as the basis of powerful proof techniques in computational complexity theory

Computability in constructive type theory

We give a formalised and machine-checked account of computability theory in the Calculus of Inductive Constructions (CIC), the constructive type theory underlying the Coq proof assistant. We first develop synthetic computability theory, pioneered by Richman, Bridges, and Bauer, where one treats all functions as computable, eliminating the need for a model of computation. We assume a novel parametric axiom for synthetic computability and give proofs of results like Rice’s theorem, the Myhill isomorphism theorem, and the existence of Post’s simple and hypersimple predicates relying on no other axioms such as Markov’s principle or choice axioms. As a second step, we introduce models of computation. We give a concise overview of definitions of various standard models and contribute machine-checked simulation proofs, posing a non-trivial engineering effort. We identify a notion of synthetic undecidability relative to a fixed halting problem, allowing axiom-free machine-checked proofs of undecidability. We contribute such undecidability proofs for the historical foundational problems of computability theory which require the identification of invariants left out in the literature and now form the basis of the Coq Library of Undecidability Proofs. We then identify the weak call-by-value λ-calculus L as sweet spot for programming in a model of computation. We introduce a certifying extraction framework and analyse an axiom stating that every function of type ℕ → ℕ is L-computable.Wir behandeln eine formalisierte und maschinengeprüfte Betrachtung von Berechenbarkeitstheorie im Calculus of Inductive Constructions (CIC), der konstruktiven Typtheorie die dem Beweisassistenten Coq zugrunde liegt. Wir entwickeln erst synthetische Berechenbarkeitstheorie, vorbereitet durch die Arbeit von Richman, Bridges und Bauer, wobei alle Funktionen als berechenbar behandelt werden, ohne Notwendigkeit eines Berechnungsmodells. Wir nehmen ein neues, parametrisches Axiom für synthetische Berechenbarkeit an und beweisen Resultate wie das Theorem von Rice, das Isomorphismus Theorem von Myhill und die Existenz von Post’s simplen und hypersimplen Prädikaten ohne Annahme von anderen Axiomen wie Markov’s Prinzip oder Auswahlaxiomen. Als zweiten Schritt führen wir Berechnungsmodelle ein. Wir geben einen kompakten Überblick über die Definition von verschiedenen Berechnungsmodellen und erklären maschinengeprüfte Simulationsbeweise zwischen diesen Modellen, welche einen hohen Konstruktionsaufwand beinhalten. Wir identifizieren einen Begriff von synthetischer Unentscheidbarkeit relativ zu einem fixierten Halteproblem welcher axiomenfreie maschinengeprüfte Unentscheidbarkeitsbeweise erlaubt. Wir erklären solche Beweise für die historisch grundlegenden Probleme der Berechenbarkeitstheorie, die das Identifizieren von Invarianten die normalerweise in der Literatur ausgelassen werden benötigen und nun die Basis der Coq Library of Undecidability Proofs bilden. Wir identifizieren dann den call-by-value λ-Kalkül L als sweet spot für die Programmierung in einem Berechnungsmodell. Wir führen ein zertifizierendes Extraktionsframework ein und analysieren ein Axiom welches postuliert dass jede Funktion vom Typ N→N L-berechenbar ist

Subshifts as Models for MSO Logic

We study the Monadic Second Order (MSO) Hierarchy over colourings of the discrete plane, and draw links between classes of formula and classes of subshifts. We give a characterization of existential MSO in terms of projections of tilings, and of universal sentences in terms of combinations of "pattern counting" subshifts. Conversely, we characterise logic fragments corresponding to various classes of subshifts (subshifts of finite type, sofic subshifts, all subshifts). Finally, we show by a separation result how the situation here is different from the case of tiling pictures studied earlier by Giammarresi et al.Comment: arXiv admin note: substantial text overlap with arXiv:0904.245

Computable analysis on the space of marked groups

We investigate decision problems for groups described by word problem algorithms. This is equivalent to studying groups described by labelled Cayley graphs. We show that this corresponds to the study of computable analysis on the space of marked groups, and point out several results of computable analysis that can be directly applied to obtain group theoretical results. Those results, used in conjunction with the version of Higman's Embedding Theorem that preserves solvability of the word problem, provide powerful tools to build finitely presented groups with solvable word problem but with various undecidable properties. We also investigate the first levels of an effective Borel hierarchy on the space of marked groups, and show that on many group properties usually considered, this effective hierarchy corresponds sharply to the Borel hierarchy. Finally, we prove that the space of marked groups is a Polish space that is not effectively Polish. Because of this, many of the most important results of computable analysis cannot be applied to the space of marked groups. This includes the Kreisel-Lacombe-Schoenfield-Ceitin Theorem and a theorem of Moschovakis. The space of marked groups constitutes the first natural example of a Polish space that is not effectively Polish.Comment: 46 pages, Theorem 4.6 was false as stated, it appears now, having been corrected, as Theorem 5.