Reductions of Hidden Information Sources

In all but special circumstances, measurements of time-dependent processes reflect internal structures and correlations only indirectly. Building predictive models of such hidden information sources requires discovering, in some way, the internal states and mechanisms. Unfortunately, there are often many possible models that are observationally equivalent. Here we show that the situation is not as arbitrary as one would think. We show that generators of hidden stochastic processes can be reduced to a minimal form and compare this reduced representation to that provided by computational mechanics--the epsilon-machine. On the way to developing deeper, measure-theoretic foundations for the latter, we introduce a new two-step reduction process. The first step (internal-event reduction) produces the smallest observationally equivalent sigma-algebra and the second (internal-state reduction) removes sigma-algebra components that are redundant for optimal prediction. For several classes of stochastic dynamical systems these reductions produce representations that are equivalent to epsilon-machines.Comment: 12 pages, 4 figures; 30 citations; Updates at http://www.santafe.edu/~cm

Has the Nitrate-Nitrogen in Streams Draining Agricultural Watersheds in Kentucky Changed in the Last 18 Years?

In 1971 and 1972, we sampled streams across Kentucky for nitrate-nitrogen (NO3-N) during the high-water flow months of January through June. The results were variable and showed a dominant effect of geology and lesser effects of both time and land use on the results. Since that time, use of fertilizer nitrogen (N) has nearly doubled in Kentucky and, further, there is a high degree of concern among the public that NO3-N from fertilizer use may be contaminating streams. The US Environmental Protection Agency has set a maximum of 10 ppm NO3-N in water as being safe for human consumption

Spectral Simplicity of Apparent Complexity, Part II: Exact Complexities and Complexity Spectra

The meromorphic functional calculus developed in Part I overcomes the nondiagonalizability of linear operators that arises often in the temporal evolution of complex systems and is generic to the metadynamics of predicting their behavior. Using the resulting spectral decomposition, we derive closed-form expressions for correlation functions, finite-length Shannon entropy-rate approximates, asymptotic entropy rate, excess entropy, transient information, transient and asymptotic state uncertainty, and synchronization information of stochastic processes generated by finite-state hidden Markov models. This introduces analytical tractability to investigating information processing in discrete-event stochastic processes, symbolic dynamics, and chaotic dynamical systems. Comparisons reveal mathematical similarities between complexity measures originally thought to capture distinct informational and computational properties. We also introduce a new kind of spectral analysis via coronal spectrograms and the frequency-dependent spectra of past-future mutual information. We analyze a number of examples to illustrate the methods, emphasizing processes with multivariate dependencies beyond pairwise correlation. An appendix presents spectral decomposition calculations for one example in full detail.Comment: 27 pages, 12 figures, 2 tables; most recent version at http://csc.ucdavis.edu/~cmg/compmech/pubs/sdscpt2.ht

Structural Information in Two-Dimensional Patterns: Entropy Convergence and Excess Entropy

We develop information-theoretic measures of spatial structure and pattern in more than one dimension. As is well known, the entropy density of a two-dimensional configuration can be efficiently and accurately estimated via a converging sequence of conditional entropies. We show that the manner in which these conditional entropies converge to their asymptotic value serves as a measure of global correlation and structure for spatial systems in any dimension. We compare and contrast entropy-convergence with mutual-information and structure-factor techniques for quantifying and detecting spatial structure.Comment: 11 pages, 5 figures, http://www.santafe.edu/projects/CompMech/papers/2dnnn.htm

Many Roads to Synchrony: Natural Time Scales and Their Algorithms

We consider two important time scales---the Markov and cryptic orders---that monitor how an observer synchronizes to a finitary stochastic process. We show how to compute these orders exactly and that they are most efficiently calculated from the epsilon-machine, a process's minimal unifilar model. Surprisingly, though the Markov order is a basic concept from stochastic process theory, it is not a probabilistic property of a process. Rather, it is a topological property and, moreover, it is not computable from any finite-state model other than the epsilon-machine. Via an exhaustive survey, we close by demonstrating that infinite Markov and infinite cryptic orders are a dominant feature in the space of finite-memory processes. We draw out the roles played in statistical mechanical spin systems by these two complementary length scales.Comment: 17 pages, 16 figures: http://cse.ucdavis.edu/~cmg/compmech/pubs/kro.htm. Santa Fe Institute Working Paper 10-11-02

Local Causal States and Discrete Coherent Structures

Coherent structures form spontaneously in nonlinear spatiotemporal systems and are found at all spatial scales in natural phenomena from laboratory hydrodynamic flows and chemical reactions to ocean, atmosphere, and planetary climate dynamics. Phenomenologically, they appear as key components that organize the macroscopic behaviors in such systems. Despite a century of effort, they have eluded rigorous analysis and empirical prediction, with progress being made only recently. As a step in this, we present a formal theory of coherent structures in fully-discrete dynamical field theories. It builds on the notion of structure introduced by computational mechanics, generalizing it to a local spatiotemporal setting. The analysis' main tool employs the \localstates, which are used to uncover a system's hidden spatiotemporal symmetries and which identify coherent structures as spatially-localized deviations from those symmetries. The approach is behavior-driven in the sense that it does not rely on directly analyzing spatiotemporal equations of motion, rather it considers only the spatiotemporal fields a system generates. As such, it offers an unsupervised approach to discover and describe coherent structures. We illustrate the approach by analyzing coherent structures generated by elementary cellular automata, comparing the results with an earlier, dynamic-invariant-set approach that decomposes fields into domains, particles, and particle interactions.Comment: 27 pages, 10 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/dcs.ht

Enumerating Finitary Processes

We show how to efficiently enumerate a class of finite-memory stochastic processes using the causal representation of -machines. We characterize -machines in the language of automata theory and adapt a recent algorithm for generating accessible deterministic finite automata, pruning this over-large class down to that of -machines. As an application, we exactly enumerate topological -machines up to eight states and six-letter alphabets

Expression of a Constitutively Active Nitrate Reductase Variant in Tobacco Reduces Tobacco-Specific Nitrosamine Accumulation in Cured Leaves and Cigarette Smoke

Burley tobaccos (Nicotiana tabacum) display a nitrogen-use-deficiency phenotype that is associated with the accumulation of high levels of nitrate within the leaf, a trait correlated with production of a class of compounds referred to as tobacco-specific nitrosamines (TSNAs). Two TSNA species, 4-(methylnitrosamino)-1-(3-pyridyl)-1-butanone (NNK) and N-nitrosonornicotine (NNN), have been shown to be strong carcinogens in numerous animal studies. We investigated the potential of molecular genetic strategies to lower nitrate levels in burley tobaccos by overexpressing genes encoding key enzymes of the nitrogen-assimilation pathway. Of the various constructs tested, only the expression of a constitutively active nitrate reductase (NR) dramatically decreased free nitrate levels in the leaves. Field-grown tobacco plants expressing this NR variant exhibited greatly reduced levels of TSNAs in both cured leaves and mainstream smoke of cigarettes made from these materials. Decreasing leaf nitrate levels via expression of a constitutively active NR enzyme represents an exceptionally promising means for reducing the production of NNN and NNK, two of the most well-documented animal carcinogens found in tobacco products

Minimum memory for generating rare events

We classify the rare events of structured, memoryful stochastic processes and use this to analyze sequential and parallel generators for these events. Given a stochastic process, we introduce a method to construct a process whose typical realizations are a given process' rare events. This leads to an expression for the minimum memory required to generate rare events. We then show that the recently discovered classical-quantum ambiguity of simplicity also occurs when comparing the structure of process fluctuations