On the Compression of Unknown Sources

Ph.D. Thesis. University of Hawaiʻi at Mānoa 2018

Shannon Information and Kolmogorov Complexity

We compare the elementary theories of Shannon information and Kolmogorov complexity, the extent to which they have a common purpose, and where they are fundamentally different. We discuss and relate the basic notions of both theories: Shannon entropy versus Kolmogorov complexity, the relation of both to universal coding, Shannon mutual information versus Kolmogorov (`algorithmic') mutual information, probabilistic sufficient statistic versus algorithmic sufficient statistic (related to lossy compression in the Shannon theory versus meaningful information in the Kolmogorov theory), and rate distortion theory versus Kolmogorov's structure function. Part of the material has appeared in print before, scattered through various publications, but this is the first comprehensive systematic comparison. The last mentioned relations are new.Comment: Survey, LaTeX 54 pages, 3 figures, Submitted to IEEE Trans Information Theor

Large deviations and applications : the finite dimensional case

Includes bibliographical references (p. 96-98).Research supported by the Air Force Office of Scientific Research. AFOSR-89-0276B Research supported by the Army Research Office. DAAL03-86-K-171A. Dembo, O. Zeitouni

Tree models :algorithms and information theoretic properties

La tesis estudia propiedades fundamentales y algoritmos relacionados con modelos árbol. Estos modelos requieren una cantidad relativamente pequeña de parámetros para representar fuentes de memoria finita (Markov) sobre alfabetos finitos, cuando el largo de la cantidad de símbolos pasados necesaria para determinar la distribución de probabilidad condicional del siguiente símbolo no es fija, sino que depende del contexto en el cual ocurre el símbolo. La tesis define estructuras combinatorias como árboles de contexto generalizados y sus clausuras FSM (del inglés finite state machine), y aplica estas estructuras para describir la primera implementación en tiempo lineal de codificación y decodificación de la versión semi-predictiva del algoritmo Context, un esquema doblemente universal que alcanza una tasa de convergencia óptima a la entropía en la clases de modelos árbol. La tesis analiza luego clases de tipo para modelos árbol, extendiendo el método de tipos previamente estudiado para modelos FSM. Se deriva una fórmula exacta para la cardinalidad de una clase de tipo para una secuencia de largo n dada, así como una estimación asintótica del valor esperado del logaritmo del tamaño de una clase de tipo, y una estimación asintótica del número de clases de tipo diferentes para secuencias de un largo dado. Estos resultados asintóticos se derivan con la ayuda del nuevo concepto de extensión canónica mínima de un árbol de contexto, un objeto combinatorio fundamental que se encuentra entre el árbol original y su clausura FSM. Como aplicaciones de las nuevas propiedades descubiertas para modelos árbol, se presentan algoritmos de codificación enumerativa doblemente universales y esquemas de simulación universal para secuencias individuales. Finalmente, la tesis presenta algunos problemas abiertos y direcciones para investigaciones futuras en esta área

Hidden Markov Models

Hidden Markov Models (HMMs), although known for decades, have made a big career nowadays and are still in state of development. This book presents theoretical issues and a variety of HMMs applications in speech recognition and synthesis, medicine, neurosciences, computational biology, bioinformatics, seismology, environment protection and engineering. I hope that the reader will find this book useful and helpful for their own research

Mathematical Logic and Its Applications 2020

The issue "Mathematical Logic and Its Applications 2020" contains articles related to the following three directions: Descriptive Set Theory (3 articles). Solutions for long-standing problems, including those of A. Tarski and H. Friedman, are presented. Exact combinatorial optimization algorithms, in which the complexity relative to the source data is characterized by a low, or even first degree, polynomial (1 article). III. Applications of mathematical logic and the theory of algorithms (2 articles). The first article deals with the Jacobian and M. Kontsevich’s conjectures, and algorithmic undecidability; for these purposes, non-standard analysis is used. The second article provides a quantitative description of the balance and adaptive resource of a human. Submissions are invited for the next issue "Mathematical Logic and Its Applications 2021

Weighted Tree Automata -- May it be a little more?

This is a book on weighted tree automata. We present the basic definitions and some of the important results in a coherent form with full proofs. The concept of weighted tree automata is part of Automata Theory and it touches the area of Universal Algebra. It originated from two sources: weighted string automata and finite-state tree automata