Asymmetry of the Kolmogorov complexity of online predicting odd and even bits

Symmetry of information states that $C(x) + C(y|x) = C(x,y) + O(\log C(x))$. We show that a similar relation for online Kolmogorov complexity does not hold. Let the even (online Kolmogorov) complexity of an n-bitstring $x_1x_2... x_n$ be the length of a shortest program that computes $x_2$ on input $x_1$, computes $x_4$ on input $x_1x_2x_3$, etc; and similar for odd complexity. We show that for all n there exist an n-bit x such that both odd and even complexity are almost as large as the Kolmogorov complexity of the whole string. Moreover, flipping odd and even bits to obtain a sequence $x_2x_1x_4x_3\ldots$, decreases the sum of odd and even complexity to $C(x)$.Comment: 20 pages, 7 figure

Influence tests I: ideal composite hypothesis tests, and causal semimeasures

Ratios of universal enumerable semimeasures corresponding to hypotheses are investigated as a solution for statistical composite hypotheses testing if an unbounded amount of computation time can be assumed. Influence testing for discrete time series is defined using generalized structural equations. Several ideal tests are introduced, and it is argued that when Halting information is transmitted, in some cases, instantaneous cause and consequence can be inferred where this is not possible classically. The approach is contrasted with Bayesian definitions of influence, where it is left open whether all Bayesian causal associations of universal semimeasures are equal within a constant. Finally the approach is also contrasted with existing engineering procedures for influence and theoretical definitions of causation.Comment: 29 pages, 3 figures, draf

ADAPTIVE SEARCH AND THE PRELIMINARY DESIGN OF GAS TURBINE BLADE COOLING SYSTEMS

This research concerns the integration of Adaptive Search (AS) technique such as the Genetic Algorithms (GA) with knowledge based software to develop a research prototype of an Adaptive Search Manager (ASM). The developed approach allows to utilise both quantitative and qualitative information in engineering design decision making. A Fuzzy Expert System manipulates AS software within the design environment concerning the preliminary design of gas turbine blade cooling systems. Steady state cooling hole geometry models have been developed for the project in collaboration with Rolls Royce plc. The research prototype of ASM uses a hybrid of Adaptive Restricted Tournament Selection (ARTS) and Knowledge Based Hill Climbing (KBHC) to identify multiple "good" design solutions as potential design options. ARTS is a GA technique that is particularly suitable for real world problems having multiple sub-optima. KBHC uses information gathered during the ARTS search as well as information from the designer to perform a deterministic hill climbing. Finally, a local stochastic hill climbing fine tunes the "good" designs. Design solution sensitivity, design variable sensitivities and constraint sensitivities are calculated following Taguchi's methodology, which extracts sensitivity information with a very small number of model evaluations. Each potential design option is then qualitatively evaluated separately for manufacturability, choice of materials and some designer's special preferences using the knowledge of domain experts. In order to guarantee that the qualitative evaluation module can evaluate any design solution from the entire design space with a reasonably small number of rules, a novel knowledge representation technique is developed. The knowledge is first separated in three categories: inter-variable knowledge, intra-variable knowledge and heuristics. Inter-variable knowledge and intra-variable knowledge are then integrated using a concept of compromise. Information about the "good" design solutions is presented to the designer through a designer's interface for decision support.Rolls Royce plc., Bristol (UK

Quantum entanglement

All our former experience with application of quantum theory seems to say: {\it what is predicted by quantum formalism must occur in laboratory}. But the essence of quantum formalism - entanglement, recognized by Einstein, Podolsky, Rosen and Schr\"odinger - waited over 70 years to enter to laboratories as a new resource as real as energy. This holistic property of compound quantum systems, which involves nonclassical correlations between subsystems, is a potential for many quantum processes, including ``canonical'' ones: quantum cryptography, quantum teleportation and dense coding. However, it appeared that this new resource is very complex and difficult to detect. Being usually fragile to environment, it is robust against conceptual and mathematical tools, the task of which is to decipher its rich structure. This article reviews basic aspects of entanglement including its characterization, detection, distillation and quantifying. In particular, the authors discuss various manifestations of entanglement via Bell inequalities, entropic inequalities, entanglement witnesses, quantum cryptography and point out some interrelations. They also discuss a basic role of entanglement in quantum communication within distant labs paradigm and stress some peculiarities such as irreversibility of entanglement manipulations including its extremal form - bound entanglement phenomenon. A basic role of entanglement witnesses in detection of entanglement is emphasized.Comment: 110 pages, 3 figures, ReVTex4, Improved (slightly extended) presentation, updated references, minor changes, submitted to Rev. Mod. Phys

Studying Evolutionary Change: Transdisciplinary Advances in Understanding and Measuring Evolution

Evolutionary processes can be found in almost any historical, i.e. evolving, system that erroneously copies from the past. Well studied examples do not only originate in evolutionary biology but also in historical linguistics. Yet an approach that would bind together studies of such evolving systems is still elusive. This thesis is an attempt to narrowing down this gap to some extend. An evolving system can be described using characters that identify their changing features. While the problem of a proper choice of characters is beyond the scope of this thesis and remains in the hands of experts we concern ourselves with some theoretical as well data driven approaches. Having a well chosen set of characters describing a system of different entities such as homologous genes, i.e. genes of same origin in different species, we can build a phylogenetic tree. Consider the special case of gene clusters containing paralogous genes, i.e. genes of same origin within a species usually located closely, such as the well known HOX cluster. These are formed by step- wise duplication of its members, often involving unequal crossing over forming hybrid genes. Gene conversion and possibly other mechanisms of concerted evolution further obfuscate phylogenetic relationships. Hence, it is very difficult or even impossible to disentangle the detailed history of gene duplications in gene clusters. Expanding gene clusters that use unequal crossing over as proposed by Walter Gehring leads to distinctive patterns of genetic distances. We show that this special class of distances helps in extracting phylogenetic information from the data still. Disregarding genome rearrangements, we find that the shortest Hamiltonian path then coincides with the ordering of paralogous genes in a cluster. This observation can be used to detect ancient genomic rearrangements of gene clus- ters and to distinguish gene clusters whose evolution was dominated by unequal crossing over within genes from those that expanded through other mechanisms. While the evolution of DNA or protein sequences is well studied and can be formally described, we find that this does not hold for other systems such as language evolution. This is due to a lack of detectable mechanisms that drive the evolutionary processes in other fields. Hence, it is hard to quantify distances between entities, e.g. languages, and therefore the characters describing them. Starting out with distortions of distances, we first see that poor choices of the distance measure can lead to incorrect phylogenies. Given that phylogenetic inference requires additive metrics we can infer the correct phylogeny from a distance matrix D if there is a monotonic, subadditive function ζ such that ζ^−1(D) is additive. We compute the metric-preserving transformation ζ as the solution of an optimization problem. This result shows that the problem of phylogeny reconstruction is well defined even if a detailed mechanistic model of the evolutionary process is missing. Yet, this does not hinder studies of language evolution using automated tools. As the amount of available and large digital corpora increased so did the possibilities to study them automatically. The obvious parallels between historical linguistics and phylogenetics lead to many studies adapting bioinformatics tools to fit linguistics means. Here, we use jAlign to calculate bigram alignments, i.e. an alignment algorithm that operates with regard to adjacency of letters. Its performance is tested in different cognate recognition tasks. Using pairwise alignments one major obstacle is the systematic errors they make such as underestimation of gaps and their misplacement. Applying multiple sequence alignments instead of a pairwise algorithm implicitly includes more evolutionary information and thus can overcome the problem of correct gap placement. They can be seen as a generalization of the string-to-string edit problem to more than two strings. With the steady increase in computational power, exact, dynamic programming solutions have become feasible in practice also for 3- and 4-way alignments. For the pairwise (2-way) case, there is a clear distinction between local and global alignments. As more sequences are consid- ered, this distinction, which can in fact be made independently for both ends of each sequence, gives rise to a rich set of partially local alignment problems. So far these have remained largely unexplored. Thus, a general formal frame- work that gives raise to a classification of partially local alignment problems is introduced. It leads to a generic scheme that guides the principled design of exact dynamic programming solutions for particular partially local alignment problems

Information-Theoretic Causal Discovery

It is well-known that correlation does not equal causation, but how can we infer causal relations from data? Causal discovery tries to answer precisely this question by rigorously analyzing under which assumptions it is feasible to infer causal networks from passively collected, so-called observational data. Particularly, causal discovery aims to infer a directed graph among a set of observed random variables under assumptions which are as realistic as possible. A key assumption in causal discovery is faithfulness. That is, we assume that separations in the true graph imply independencies in the distribution and vice versa. If faithfulness holds and we have access to a perfect independence oracle, traditional causal discovery approaches can infer the Markov equivalence class of the true causal graph---i.e., infer the correct undirected network and even some of the edge directions. In a real-world setting, faithfulness may be violated, however, and neither do we have access to such an independence oracle. Beyond that, we are interested in inferring the complete DAG structure and not just the Markov equivalence class. To circumvent or at least alleviate these limitations, we take an information-theoretic approach. In the first part of this thesis, we consider violations of faithfulness that can be induced by exclusive or relations or cancelling paths, and develop a weaker faithfulness assumption, called 2-adjacency faithfulness, to detect some of these mechanisms. Further, we analyze under which conditions it is possible to infer the correct DAG structure even if such violations occur. In the second part, we focus on independence testing via conditional mutual information (CMI). CMI is an information-theoretic measure of dependence based on Shannon entropy. We first suggest estimating CMI for discrete variables via normalized maximum likelihood instead of the plug-in maximum likelihood estimator that tends to overestimate dependencies. On top of that, we show that CMI can be consistently estimated for discrete-continuous mixture random variables by simply discretizing the continuous parts of each variable. Last, we consider the problem of distinguishing the two Markov equivalent graphs X to Y and Y to X, which is a necessary step towards discovering all edge directions. To solve this problem, it is inevitable to make assumptions about the generating mechanism. We build upon the idea which states that the cause is algorithmically independent of its mechanism. We propose two methods to approximate this postulate via the Minimum Description Length (MDL) principle: one for univariate numeric data and one for multivariate mixed-type data. Finally, we combine insights from our MDL-based approach and regression-based methods with strong guarantees and show we can identify cause and effect via L0-regularized regression