Distinguishing Cause and Effect via Second Order Exponential Models

We propose a method to infer causal structures containing both discrete and continuous variables. The idea is to select causal hypotheses for which the conditional density of every variable, given its causes, becomes smooth. We define a family of smooth densities and conditional densities by second order exponential models, i.e., by maximizing conditional entropy subject to first and second statistical moments. If some of the variables take only values in proper subsets of R^n, these conditionals can induce different families of joint distributions even for Markov-equivalent graphs. We consider the case of one binary and one real-valued variable where the method can distinguish between cause and effect. Using this example, we describe that sometimes a causal hypothesis must be rejected because P(effect|cause) and P(cause) share algorithmic information (which is untypical if they are chosen independently). This way, our method is in the same spirit as faithfulness-based causal inference because it also rejects non-generic mutual adjustments among DAG-parameters.Comment: 36 pages, 8 figure

Random Subgroups of Rationals

This paper introduces and studies a notion of algorithmic randomness for subgroups of rationals. Given a randomly generated additive subgroup (G,+) of rationals, two main questions are addressed: first, what are the model-theoretic and recursion-theoretic properties of (G,+); second, what learnability properties can one extract from G and its subclass of finitely generated subgroups? For the first question, it is shown that the theory of (G,+) coincides with that of the additive group of integers and is therefore decidable; furthermore, while the word problem for G with respect to any generating sequence for G is not even semi-decidable, one can build a generating sequence beta such that the word problem for G with respect to beta is co-recursively enumerable (assuming that the set of generators of G is limit-recursive). In regard to the second question, it is proven that there is a generating sequence beta for G such that every non-trivial finitely generated subgroup of G is recursively enumerable and the class of all such subgroups of G is behaviourally correctly learnable, that is, every non-trivial finitely generated subgroup can be semantically identified in the limit (again assuming that the set of generators of G is limit-recursive). On the other hand, the class of non-trivial finitely generated subgroups of G cannot be syntactically identified in the limit with respect to any generating sequence for G. The present work thus contributes to a recent line of research studying algorithmically random infinite structures and uncovers an interesting connection between the arithmetical complexity of the set of generators of a randomly generated subgroup of rationals and the learnability of its finitely generated subgroups

Computability and Evolutionary Complexity: Markets As Complex Adaptive Systems (CAS)

The purpose of this Feature is to critically examine and to contribute to the burgeoning multi disciplinary literature on markets as complex adaptive systems (CAS). Three economists, Robert Axtell, Steven Durlauf and Arthur Robson who have distinguished themselves as pioneers in different aspects of how the thesis of evolutionary complexity pertains to market environments have contributed to this special issue. Axtell is concerned about the procedural aspects of attaining market equilibria in a decentralized setting and argues that principles on the complexity of feasible computation should rule in or out widely held models such as the Walrasian one. Robson puts forward the hypothesis called the Red Queen principle, well known from evolutionary biology, as a possible explanation for the evolution of complexity itself. Durlauf examines some of the claims that have been made in the name of complex systems theory to see whether these present testable hypothesis for economic models. My overview aims to use the wider literature on complex systems to provide a conceptual framework within which to discuss the issues raised for Economics in the above contributions and elsewhere. In particular, some assessment will be made on the extent to which modern complex systems theory and its application to markets as CAS constitutes a paradigm shift from more mainstream economic analysis

Norms as Emergent Properties of Adaptive Learning: The Case of Economic Routines

Strategic interaction among autonomous decision-makers is usually modelled in economics in game-theoretic terms or within the framework of General Equilibrium. Game-theoretic and General Equilibrium models deal almost exclusively with the existence of equilibria and do not analyse the processes which might lead to them. Even when existence proofs can be given, two questions are still open. The first concerns the possibility of multiple equilibria, which game theory has shown to be the case even in very simple models and which makes the outcome of interaction unpredictable. The second relates to the computability and complexity of the decision procedures which agents should adopt and questions the possibility of reaching an equilibrium by means of an algorithmically implementable strategy. Some theorems have recently proved that in many economically relevant problems equilibria are not computable. A different approach to the problem of strategic interaction is a "constructivist" one. Such a perspective, instead of being based upon an axiomatic view of human behaviour grounded on the principle of optimisation, focuses on algorithmically implementable "satisfycing" decision procedures. Once the axiomatic approach has been abandoned, decision procedures cannot be deduced from rationality assumptions, but must be the evolving outcome of a process of learning and adaptation to the particular environment in which the decision must be made. This paper considers one of the most recently proposed adaptive learning models: Genetic Programming and applies it to one the mostly studied and still controversial economic interaction environment, that of oligopolistic markets. Genetic Programming evolves decision procedures, represented by elements in the space of functions, balancing the exploitation of knowledge previously obtained with the search of more productive procedures. The results obtained are consistent with the evidence from the observation of the behaviour of real economic agents

Reasoning about Independence in Probabilistic Models of Relational Data

We extend the theory of d-separation to cases in which data instances are not independent and identically distributed. We show that applying the rules of d-separation directly to the structure of probabilistic models of relational data inaccurately infers conditional independence. We introduce relational d-separation, a theory for deriving conditional independence facts from relational models. We provide a new representation, the abstract ground graph, that enables a sound, complete, and computationally efficient method for answering d-separation queries about relational models, and we present empirical results that demonstrate effectiveness.Comment: 61 pages, substantial revisions to formalisms, theory, and related wor

A Novel Machine Learning Classifier Based on a Qualia Modeling Agent (QMA)

This dissertation addresses a problem found in supervised machine learning (ML) classification, that the target variable, i.e., the variable a classifier predicts, has to be identified before training begins and cannot change during training and testing. This research develops a computational agent, which overcomes this problem. The Qualia Modeling Agent (QMA) is modeled after two cognitive theories: Stanovich\u27s tripartite framework, which proposes learning results from interactions between conscious and unconscious processes; and, the Integrated Information Theory (IIT) of Consciousness, which proposes that the fundamental structural elements of consciousness are qualia. By modeling the informational relationships of qualia, the QMA allows for retaining and reasoning-over data sets in a non-ontological, non-hierarchical qualia space (QS). This novel computational approach supports concept drift, by allowing the target variable to change ad infinitum without re-training while achieving classification accuracy comparable to or greater than benchmark classifiers. Additionally, the research produced a functioning model of Stanovich\u27s framework, and a computationally tractable working solution for a representation of qualia, which when exposed to new examples, is able to match the causal structure and generate new inferences