The Origins of Computational Mechanics: A Brief Intellectual History and Several Clarifications

The principle goal of computational mechanics is to define pattern and structure so that the organization of complex systems can be detected and quantified. Computational mechanics developed from efforts in the 1970s and early 1980s to identify strange attractors as the mechanism driving weak fluid turbulence via the method of reconstructing attractor geometry from measurement time series and in the mid-1980s to estimate equations of motion directly from complex time series. In providing a mathematical and operational definition of structure it addressed weaknesses of these early approaches to discovering patterns in natural systems. Since then, computational mechanics has led to a range of results from theoretical physics and nonlinear mathematics to diverse applications---from closed-form analysis of Markov and non-Markov stochastic processes that are ergodic or nonergodic and their measures of information and intrinsic computation to complex materials and deterministic chaos and intelligence in Maxwellian demons to quantum compression of classical processes and the evolution of computation and language. This brief review clarifies several misunderstandings and addresses concerns recently raised regarding early works in the field (1980s). We show that misguided evaluations of the contributions of computational mechanics are groundless and stem from a lack of familiarity with its basic goals and from a failure to consider its historical context. For all practical purposes, its modern methods and results largely supersede the early works. This not only renders recent criticism moot and shows the solid ground on which computational mechanics stands but, most importantly, shows the significant progress achieved over three decades and points to the many intriguing and outstanding challenges in understanding the computational nature of complex dynamic systems.Comment: 11 pages, 123 citations; http://csc.ucdavis.edu/~cmg/compmech/pubs/cmr.ht

Structure or Noise?

We show how rate-distortion theory provides a mechanism for automated theory building by naturally distinguishing between regularity and randomness. We start from the simple principle that model variables should, as much as possible, render the future and past conditionally independent. From this, we construct an objective function for model making whose extrema embody the trade-off between a model's structural complexity and its predictive power. The solutions correspond to a hierarchy of models that, at each level of complexity, achieve optimal predictive power at minimal cost. In the limit of maximal prediction the resulting optimal model identifies a process's intrinsic organization by extracting the underlying causal states. In this limit, the model's complexity is given by the statistical complexity, which is known to be minimal for achieving maximum prediction. Examples show how theory building can profit from analyzing a process's causal compressibility, which is reflected in the optimal models' rate-distortion curve--the process's characteristic for optimally balancing structure and noise at different levels of representation.Comment: 6 pages, 2 figures; http://cse.ucdavis.edu/~cmg/compmech/pubs/son.htm

Efficient inference of parsimonious phenomenological models of cellular dynamics using S-systems and alternating regression

The nonlinearity of dynamics in systems biology makes it hard to infer them from experimental data. Simple linear models are computationally efficient, but cannot incorporate these important nonlinearities. An adaptive method based on the S-system formalism, which is a sensible representation of nonlinear mass-action kinetics typically found in cellular dynamics, maintains the efficiency of linear regression. We combine this approach with adaptive model selection to obtain efficient and parsimonious representations of cellular dynamics. The approach is tested by inferring the dynamics of yeast glycolysis from simulated data. With little computing time, it produces dynamical models with high predictive power and with structural complexity adapted to the difficulty of the inference problem.Comment: 14 pages, 2 figure

Parametric, nonparametric and parametric modelling of a chaotic circuit time series

The determination of a differential equation underlying a measured time series is a frequently arising task in nonlinear time series analysis. In the validation of a proposed model one often faces the dilemma that it is hard to decide whether possible discrepancies between the time series and model output are caused by an inappropriate model or by bad estimates of parameters in a correct type of model, or both. We propose a combination of parametric modelling based on Bock's multiple shooting algorithm and nonparametric modelling based on optimal transformations as a strategy to test proposed models and if rejected suggest and test new ones. We exemplify this strategy on an experimental time series from a chaotic circuit where we obtain an extremely accurate reconstruction of the observed attractor.Comment: 19 pages, 8 Fig

Identification of parameters in amplitude equations describing coupled wakes

We study the flow behind an array of equally spaced parallel cylinders. A system of Stuart-Landau equations with complex parameters is used to model the oscillating wakes. Our purpose is to identify the 6 scalar parameters which most accurately reproduce the experimental data of Chauve and Le Gal [{Physica D {\bf 58}}, pp 407--413, (1992)]. To do so, we perform a computational search for the minimum of a distance \calj. We define \calj as the sum-square difference of the data and amplitudes reconstructed using coupled equations. The search algorithm is made more efficient through the use of a partially analytical expression for the gradient $\nabla \cal J$. Indeed $\nabla \cal J$ can be obtained by the integration of a dynamical system propagating backwards in time (a backpropagation equation for the Lagrange multipliers). Using the parameters computed via the backpropagation method, the coupled Stuart-Landau equations accurately predicted the experimental data from Chauve and Le Gal over a correlation time of the system. Our method turns out to be quite robust as evidenced by using noisy synthetic data obtained from integrations of the coupled Stuart-Landau equations. However, a difficulty remains with experimental data: in that case the several sets of identified parameters are shown to yield equivalent predictions. This is due to a strong discretization or ``round-off" error arising from the digitalization of the video images in the experiment. This ambiguity in parameter identification has been reproduced with synthetic data subjected to the same kind of discretization.Comment: 25 pages uuencoded compressed PostScript file (58K) with 13 figures (155K in separated file) Submitted to Physica

An alternative solution to the model structure selection problem

An alternative solution to the model structure selection problem is introduced by conducting a forward search through the many possible candidate model terms initially and then performing an exhaustive all subset model selection on the resulting model. An example is included to demonstrate that this approach leads to dynamically valid nonlinear model