6,733 research outputs found
Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillation
The control of nonlinear dynamical systems remains a major challenge for
autonomous agents. Current trends in reinforcement learning (RL) focus on
complex representations of dynamics and policies, which have yielded impressive
results in solving a variety of hard control tasks. However, this new
sophistication and extremely over-parameterized models have come with the cost
of an overall reduction in our ability to interpret the resulting policies. In
this paper, we take inspiration from the control community and apply the
principles of hybrid switching systems in order to break down complex dynamics
into simpler components. We exploit the rich representational power of
probabilistic graphical models and derive an expectation-maximization (EM)
algorithm for learning a sequence model to capture the temporal structure of
the data and automatically decompose nonlinear dynamics into stochastic
switching linear dynamical systems. Moreover, we show how this framework of
switching models enables extracting hierarchies of Markovian and
auto-regressive locally linear controllers from nonlinear experts in an
imitation learning scenario.Comment: 2nd Annual Conference on Learning for Dynamics and Contro
The Origins of Computational Mechanics: A Brief Intellectual History and Several Clarifications
The principle goal of computational mechanics is to define pattern and
structure so that the organization of complex systems can be detected and
quantified. Computational mechanics developed from efforts in the 1970s and
early 1980s to identify strange attractors as the mechanism driving weak fluid
turbulence via the method of reconstructing attractor geometry from measurement
time series and in the mid-1980s to estimate equations of motion directly from
complex time series. In providing a mathematical and operational definition of
structure it addressed weaknesses of these early approaches to discovering
patterns in natural systems.
Since then, computational mechanics has led to a range of results from
theoretical physics and nonlinear mathematics to diverse applications---from
closed-form analysis of Markov and non-Markov stochastic processes that are
ergodic or nonergodic and their measures of information and intrinsic
computation to complex materials and deterministic chaos and intelligence in
Maxwellian demons to quantum compression of classical processes and the
evolution of computation and language.
This brief review clarifies several misunderstandings and addresses concerns
recently raised regarding early works in the field (1980s). We show that
misguided evaluations of the contributions of computational mechanics are
groundless and stem from a lack of familiarity with its basic goals and from a
failure to consider its historical context. For all practical purposes, its
modern methods and results largely supersede the early works. This not only
renders recent criticism moot and shows the solid ground on which computational
mechanics stands but, most importantly, shows the significant progress achieved
over three decades and points to the many intriguing and outstanding challenges
in understanding the computational nature of complex dynamic systems.Comment: 11 pages, 123 citations;
http://csc.ucdavis.edu/~cmg/compmech/pubs/cmr.ht
Statistical Mechanics and Information-Theoretic Perspectives on Complexity in the Earth System
Peer reviewedPublisher PD
Parameter estimation for Boolean models of biological networks
Boolean networks have long been used as models of molecular networks and play
an increasingly important role in systems biology. This paper describes a
software package, Polynome, offered as a web service, that helps users
construct Boolean network models based on experimental data and biological
input. The key feature is a discrete analog of parameter estimation for
continuous models. With only experimental data as input, the software can be
used as a tool for reverse-engineering of Boolean network models from
experimental time course data.Comment: Web interface of the software is available at
http://polymath.vbi.vt.edu/polynome
Information Anatomy of Stochastic Equilibria
A stochastic nonlinear dynamical system generates information, as measured by
its entropy rate. Some---the ephemeral information---is dissipated and
some---the bound information---is actively stored and so affects future
behavior. We derive analytic expressions for the ephemeral and bound
informations in the limit of small-time discretization for two classical
systems that exhibit dynamical equilibria: first-order Langevin equations (i)
where the drift is the gradient of a potential function and the diffusion
matrix is invertible and (ii) with a linear drift term (Ornstein-Uhlenbeck) but
a noninvertible diffusion matrix. In both cases, the bound information is
sensitive only to the drift, while the ephemeral information is sensitive only
to the diffusion matrix and not to the drift. Notably, this information anatomy
changes discontinuously as any of the diffusion coefficients vanishes,
indicating that it is very sensitive to the noise structure. We then calculate
the information anatomy of the stochastic cusp catastrophe and of particles
diffusing in a heat bath in the overdamped limit, both examples of stochastic
gradient descent on a potential landscape. Finally, we use our methods to
calculate and compare approximations for the so-called time-local predictive
information for adaptive agents.Comment: 35 pages, 3 figures, 1 table;
http://csc.ucdavis.edu/~cmg/compmech/pubs/iase.ht
Beyond the Spectral Theorem: Spectrally Decomposing Arbitrary Functions of Nondiagonalizable Operators
Nonlinearities in finite dimensions can be linearized by projecting them into
infinite dimensions. Unfortunately, often the linear operator techniques that
one would then use simply fail since the operators cannot be diagonalized. This
curse is well known. It also occurs for finite-dimensional linear operators. We
circumvent it by developing a meromorphic functional calculus that can
decompose arbitrary functions of nondiagonalizable linear operators in terms of
their eigenvalues and projection operators. It extends the spectral theorem of
normal operators to a much wider class, including circumstances in which poles
and zeros of the function coincide with the operator spectrum. By allowing the
direct manipulation of individual eigenspaces of nonnormal and
nondiagonalizable operators, the new theory avoids spurious divergences. As
such, it yields novel insights and closed-form expressions across several areas
of physics in which nondiagonalizable dynamics are relevant, including
memoryful stochastic processes, open non unitary quantum systems, and
far-from-equilibrium thermodynamics.
The technical contributions include the first full treatment of arbitrary
powers of an operator. In particular, we show that the Drazin inverse,
previously only defined axiomatically, can be derived as the negative-one power
of singular operators within the meromorphic functional calculus and we give a
general method to construct it. We provide new formulae for constructing
projection operators and delineate the relations between projection operators,
eigenvectors, and generalized eigenvectors.
By way of illustrating its application, we explore several, rather distinct
examples.Comment: 29 pages, 4 figures, expanded historical citations;
http://csc.ucdavis.edu/~cmg/compmech/pubs/bst.ht
A Computational Algebra Approach to the Reverse Engineering of Gene Regulatory Networks
This paper proposes a new method to reverse engineer gene regulatory networks
from experimental data. The modeling framework used is time-discrete
deterministic dynamical systems, with a finite set of states for each of the
variables. The simplest examples of such models are Boolean networks, in which
variables have only two possible states. The use of a larger number of possible
states allows a finer discretization of experimental data and more than one
possible mode of action for the variables, depending on threshold values.
Furthermore, with a suitable choice of state set, one can employ powerful tools
from computational algebra, that underlie the reverse-engineering algorithm,
avoiding costly enumeration strategies. To perform well, the algorithm requires
wildtype together with perturbation time courses. This makes it suitable for
small to meso-scale networks rather than networks on a genome-wide scale. The
complexity of the algorithm is quadratic in the number of variables and cubic
in the number of time points. The algorithm is validated on a recently
published Boolean network model of segment polarity development in Drosophila
melanogaster.Comment: 28 pages, 5 EPS figures, uses elsart.cl
- ā¦