73,046 research outputs found
Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillation
The control of nonlinear dynamical systems remains a major challenge for
autonomous agents. Current trends in reinforcement learning (RL) focus on
complex representations of dynamics and policies, which have yielded impressive
results in solving a variety of hard control tasks. However, this new
sophistication and extremely over-parameterized models have come with the cost
of an overall reduction in our ability to interpret the resulting policies. In
this paper, we take inspiration from the control community and apply the
principles of hybrid switching systems in order to break down complex dynamics
into simpler components. We exploit the rich representational power of
probabilistic graphical models and derive an expectation-maximization (EM)
algorithm for learning a sequence model to capture the temporal structure of
the data and automatically decompose nonlinear dynamics into stochastic
switching linear dynamical systems. Moreover, we show how this framework of
switching models enables extracting hierarchies of Markovian and
auto-regressive locally linear controllers from nonlinear experts in an
imitation learning scenario.Comment: 2nd Annual Conference on Learning for Dynamics and Contro
Model Creation and Equivalence Proofs of Cellular Automata and Artificial Neural Networks
Computational methods and mathematical models have invaded arguably every
scientific discipline forming its own field of research called computational
science. Mathematical models are the theoretical foundation of computational
science. Since Newton's time, differential equations in mathematical models
have been widely and successfully used to describe the macroscopic or global
behaviour of systems. With spatially inhomogeneous, time-varying, local
element-specific, and often non-linear interactions, the dynamics of complex
systems is in contrast more efficiently described by local rules and thus in an
algorithmic and local or microscopic manner. The theory of mathematical
modelling taking into account these characteristics of complex systems has to
be established still. We recently presented a so-called allagmatic method
including a system metamodel to provide a framework for describing, modelling,
simulating, and interpreting complex systems. Implementations of cellular
automata and artificial neural networks were described and created with that
method. Guidance from philosophy were helpful in these first studies focusing
on programming and feasibility. A rigorous mathematical formalism, however, is
still missing. This would not only more precisely describe and define the
system metamodel, it would also further generalise it and with that extend its
reach to formal treatment in applied mathematics and theoretical aspects of
computational science as well as extend its applicability to other mathematical
and computational models such as agent-based models. Here, a mathematical
definition of the system metamodel is provided. Based on the presented
formalism, model creation and equivalence of cellular automata and artificial
neural networks are proved. It thus provides a formal approach for studying the
creation of mathematical models as well as their structural and operational
comparison.Comment: 13 pages, 1 tabl
- …