Coding-theorem Like Behaviour and Emergence of the Universal Distribution from Resource-bounded Algorithmic Probability

Previously referred to as `miraculous' in the scientific literature because of its powerful properties and its wide application as optimal solution to the problem of induction/inference, (approximations to) Algorithmic Probability (AP) and the associated Universal Distribution are (or should be) of the greatest importance in science. Here we investigate the emergence, the rates of emergence and convergence, and the Coding-theorem like behaviour of AP in Turing-subuniversal models of computation. We investigate empirical distributions of computing models in the Chomsky hierarchy. We introduce measures of algorithmic probability and algorithmic complexity based upon resource-bounded computation, in contrast to previously thoroughly investigated distributions produced from the output distribution of Turing machines. This approach allows for numerical approximations to algorithmic (Kolmogorov-Chaitin) complexity-based estimations at each of the levels of a computational hierarchy. We demonstrate that all these estimations are correlated in rank and that they converge both in rank and values as a function of computational power, despite fundamental differences between computational models. In the context of natural processes that operate below the Turing universal level because of finite resources and physical degradation, the investigation of natural biases stemming from algorithmic rules may shed light on the distribution of outcomes. We show that up to 60\% of the simplicity/complexity bias in distributions produced even by the weakest of the computational models can be accounted for by Algorithmic Probability in its approximation to the Universal Distribution.Comment: 27 pages main text, 39 pages including supplement. Online complexity calculator: http://complexitycalculator.com

Efficient Tabular LR Parsing

We give a new treatment of tabular LR parsing, which is an alternative to Tomita's generalized LR algorithm. The advantage is twofold. Firstly, our treatment is conceptually more attractive because it uses simpler concepts, such as grammar transformations and standard tabulation techniques also know as chart parsing. Secondly, the static and dynamic complexity of parsing, both in space and time, is significantly reduced.Comment: 8 pages, uses aclap.st

Towards Compositional CLP-based Test Data Generation for Imperative Languages.

Test data generation (TDG) is the process of automatically generating test-cases for interesting test coverage criteria. The coverage criteria measure how well the program is exercised by a test suite. Examples of coverage criteria are: statement coverage which requires that each line of the code is executed; path coverage whic

A survey of compiler development aids

A theoretical background was established for the compilation process by dividing it into five phases and explaining the concepts and algorithms that underpin each. The five selected phases were lexical analysis, syntax analysis, semantic analysis, optimization, and code generation. Graph theoretical optimization techniques were presented, and approaches to code generation were described for both one-pass and multipass compilation environments. Following the initial tutorial sections, more than 20 tools that were developed to aid in the process of writing compilers were surveyed. Eight of the more recent compiler development aids were selected for special attention - SIMCMP/STAGE2, LANG-PAK, COGENT, XPL, AED, CWIC, LIS, and JOCIT. The impact of compiler development aids were assessed some of their shortcomings and some of the areas of research currently in progress were inspected

Modular and type-safe definition of Attribute Grammars with AspectAG

AspectAG is a Haskell-embedded domain-specific language (EDSL) that encodes first-class attribute grammars (AGs). AspectAG ensures the wellformedness of AGs at compile time by using extensible records and predicates encoded using old-fashioned type-level programming features, such as multiparameter type classes and functional dependencies. AspectAG suffers the usual drawbacks of EDSLs: when type errors occur they usually do not deliver error messages that refer to domain terms, but to the host language. Often, implementation details of the EDSL are leaked in those messages. The use of type-level programming techniques makes the situation worse since type-level abstraction mechanisms are quite poor. Additionally, old-fashioned type-level programs are untyped at type-level, which is inconsistent with the general approach of strongly-typed functional programming. By using modern Haskell extensions and techniques we propose a reworked version of AspectAG that tackles those weaknesses. New AG definitions are safer, both at the level of types and at the level of kinds. Furthemore, a set of identified domain-specific errors are reported with DSL-oriented messages. To achieve this, we define and use a framework for manipulating type errors that can be used in any EDSL. We show the pragmatics of AspectAG by defining languages and extending them both with new syntax and semantics. We use MateFun, a purelyfunctional language used to teach mathematics as a case study.AspectAG es un lenguaje de dominio específico embebido (EDSL) que codifica gramáticas de atributos (AGs) como ciudadanos de primera clase. AspectAG garantiza la buena formación de las AGs en tiempo de compilación por medio del uso de registros extensibles y predicados, codificados gracias al uso de características antiguas de programación a nivel de tipos, como clases multiparámetro y dependencias funcionales. AspectAG sufre las desventajas usuales de los EDSLs: cuando ocurren errores de tipado, los mensajes de error reportados no se expresan en términos del dominio, sino del lenguaje anfitrión. También es usual que detalles de implementación del EDSL se vean filtrados en estos mensajes. El uso de técnicas de programación a nivel de tipos agrava la situación porque los mecanismos de abstracción a nivel de tipos son pobres. Ademas, las técnicas de programación a nivel de tipos usadas en AspectAG son esencialmente no tipadas, lo que es inconsistente con nuestro enfoque de tipado fuerte. Usando extensiones modernas al sistema de tipos de Haskell, proponemos una nueva versión de la biblioteca AspectAG, abordando los problemas antes mencionados. Las nuevas definiciones de AGs son mas seguras tanto a nivel de tipado como a nivel de kinds (tipado a nivel de tipos). Ademas, un conjunto identificado de errores específicos del dominio son reportados con mensajes referentes al mismo. Para lograr esto, definimos y utilizamos un framework para manipular errores de tipado, que puede ser aplicado a cualquier EDSL. Mostramos la pragmática de AspectAG definiendo lenguajes y extendiéndoles con nueva sintaxis y con nueva semántica. Utilizamos el lenguaje MateFun, un lenguaje funcional puro utilizado para enseñar matemáticas como caso de estudio

A Tutorial on the Expectation-Maximization Algorithm Including Maximum-Likelihood Estimation and EM Training of Probabilistic Context-Free Grammars

The paper gives a brief review of the expectation-maximization algorithm (Dempster 1977) in the comprehensible framework of discrete mathematics. In Section 2, two prominent estimation methods, the relative-frequency estimation and the maximum-likelihood estimation are presented. Section 3 is dedicated to the expectation-maximization algorithm and a simpler variant, the generalized expectation-maximization algorithm. In Section 4, two loaded dice are rolled. A more interesting example is presented in Section 5: The estimation of probabilistic context-free grammars.Comment: Presented at the 15th European Summer School in Logic, Language and Information (ESSLLI 2003). Example 5 extended (and partially corrected