6,227 research outputs found
Informational and Causal Architecture of Continuous-time Renewal and Hidden Semi-Markov Processes
We introduce the minimal maximally predictive models ({\epsilon}-machines) of
processes generated by certain hidden semi-Markov models. Their causal states
are either hybrid discrete-continuous or continuous random variables and
causal-state transitions are described by partial differential equations.
Closed-form expressions are given for statistical complexities, excess
entropies, and differential information anatomy rates. We present a complete
analysis of the {\epsilon}-machines of continuous-time renewal processes and,
then, extend this to processes generated by unifilar hidden semi-Markov models
and semi-Markov models. Our information-theoretic analysis leads to new
expressions for the entropy rate and the rates of related information measures
for these very general continuous-time process classes.Comment: 16 pages, 7 figures;
http://csc.ucdavis.edu/~cmg/compmech/pubs/ctrp.ht
Structural Intervention Distance (SID) for Evaluating Causal Graphs
Causal inference relies on the structure of a graph, often a directed acyclic
graph (DAG). Different graphs may result in different causal inference
statements and different intervention distributions. To quantify such
differences, we propose a (pre-) distance between DAGs, the structural
intervention distance (SID). The SID is based on a graphical criterion only and
quantifies the closeness between two DAGs in terms of their corresponding
causal inference statements. It is therefore well-suited for evaluating graphs
that are used for computing interventions. Instead of DAGs it is also possible
to compare CPDAGs, completed partially directed acyclic graphs that represent
Markov equivalence classes. Since it differs significantly from the popular
Structural Hamming Distance (SHD), the SID constitutes a valuable additional
measure. We discuss properties of this distance and provide an efficient
implementation with software code available on the first author's homepage (an
R package is under construction)
Estimating the effect of joint interventions from observational data in sparse high-dimensional settings
We consider the estimation of joint causal effects from observational data.
In particular, we propose new methods to estimate the effect of multiple
simultaneous interventions (e.g., multiple gene knockouts), under the
assumption that the observational data come from an unknown linear structural
equation model with independent errors. We derive asymptotic variances of our
estimators when the underlying causal structure is partly known, as well as
high-dimensional consistency when the causal structure is fully unknown and the
joint distribution is multivariate Gaussian. We also propose a generalization
of our methodology to the class of nonparanormal distributions. We evaluate the
estimators in simulation studies and also illustrate them on data from the
DREAM4 challenge.Comment: 30 pages, 3 figures, 45 pages supplemen
Informational and Causal Architecture of Discrete-Time Renewal Processes
Renewal processes are broadly used to model stochastic behavior consisting of
isolated events separated by periods of quiescence, whose durations are
specified by a given probability law. Here, we identify the minimal sufficient
statistic for their prediction (the set of causal states), calculate the
historical memory capacity required to store those states (statistical
complexity), delineate what information is predictable (excess entropy), and
decompose the entropy of a single measurement into that shared with the past,
future, or both. The causal state equivalence relation defines a new subclass
of renewal processes with a finite number of causal states despite having an
unbounded interevent count distribution. We use these formulae to analyze the
output of the parametrized Simple Nonunifilar Source, generated by a simple
two-state hidden Markov model, but with an infinite-state epsilon-machine
presentation. All in all, the results lay the groundwork for analyzing
processes with infinite statistical complexity and infinite excess entropy.Comment: 18 pages, 9 figures, 1 table;
http://csc.ucdavis.edu/~cmg/compmech/pubs/dtrp.ht
Prediction and Generation of Binary Markov Processes: Can a Finite-State Fox Catch a Markov Mouse?
Understanding the generative mechanism of a natural system is a vital
component of the scientific method. Here, we investigate one of the fundamental
steps toward this goal by presenting the minimal generator of an arbitrary
binary Markov process. This is a class of processes whose predictive model is
well known. Surprisingly, the generative model requires three distinct
topologies for different regions of parameter space. We show that a previously
proposed generator for a particular set of binary Markov processes is, in fact,
not minimal. Our results shed the first quantitative light on the relative
(minimal) costs of prediction and generation. We find, for instance, that the
difference between prediction and generation is maximized when the process is
approximately independently, identically distributed.Comment: 12 pages, 12 figures;
http://csc.ucdavis.edu/~cmg/compmech/pubs/gmc.ht
Distributional Equivalence and Structure Learning for Bow-free Acyclic Path Diagrams
We consider the problem of structure learning for bow-free acyclic path
diagrams (BAPs). BAPs can be viewed as a generalization of linear Gaussian DAG
models that allow for certain hidden variables. We present a first method for
this problem using a greedy score-based search algorithm. We also prove some
necessary and some sufficient conditions for distributional equivalence of BAPs
which are used in an algorithmic ap- proach to compute (nearly) equivalent
model structures. This allows us to infer lower bounds of causal effects. We
also present applications to real and simulated datasets using our publicly
available R-package
Robust causal structure learning with some hidden variables
We introduce a new method to estimate the Markov equivalence class of a
directed acyclic graph (DAG) in the presence of hidden variables, in settings
where the underlying DAG among the observed variables is sparse, and there are
a few hidden variables that have a direct effect on many of the observed ones.
Building on the so-called low rank plus sparse framework, we suggest a
two-stage approach which first removes the effect of the hidden variables, and
then estimates the Markov equivalence class of the underlying DAG under the
assumption that there are no remaining hidden variables. This approach is
consistent in certain high-dimensional regimes and performs favourably when
compared to the state of the art, both in terms of graphical structure recovery
and total causal effect estimation
The Origins of Computational Mechanics: A Brief Intellectual History and Several Clarifications
The principle goal of computational mechanics is to define pattern and
structure so that the organization of complex systems can be detected and
quantified. Computational mechanics developed from efforts in the 1970s and
early 1980s to identify strange attractors as the mechanism driving weak fluid
turbulence via the method of reconstructing attractor geometry from measurement
time series and in the mid-1980s to estimate equations of motion directly from
complex time series. In providing a mathematical and operational definition of
structure it addressed weaknesses of these early approaches to discovering
patterns in natural systems.
Since then, computational mechanics has led to a range of results from
theoretical physics and nonlinear mathematics to diverse applications---from
closed-form analysis of Markov and non-Markov stochastic processes that are
ergodic or nonergodic and their measures of information and intrinsic
computation to complex materials and deterministic chaos and intelligence in
Maxwellian demons to quantum compression of classical processes and the
evolution of computation and language.
This brief review clarifies several misunderstandings and addresses concerns
recently raised regarding early works in the field (1980s). We show that
misguided evaluations of the contributions of computational mechanics are
groundless and stem from a lack of familiarity with its basic goals and from a
failure to consider its historical context. For all practical purposes, its
modern methods and results largely supersede the early works. This not only
renders recent criticism moot and shows the solid ground on which computational
mechanics stands but, most importantly, shows the significant progress achieved
over three decades and points to the many intriguing and outstanding challenges
in understanding the computational nature of complex dynamic systems.Comment: 11 pages, 123 citations;
http://csc.ucdavis.edu/~cmg/compmech/pubs/cmr.ht
- …