13,634 research outputs found
Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillation
The control of nonlinear dynamical systems remains a major challenge for
autonomous agents. Current trends in reinforcement learning (RL) focus on
complex representations of dynamics and policies, which have yielded impressive
results in solving a variety of hard control tasks. However, this new
sophistication and extremely over-parameterized models have come with the cost
of an overall reduction in our ability to interpret the resulting policies. In
this paper, we take inspiration from the control community and apply the
principles of hybrid switching systems in order to break down complex dynamics
into simpler components. We exploit the rich representational power of
probabilistic graphical models and derive an expectation-maximization (EM)
algorithm for learning a sequence model to capture the temporal structure of
the data and automatically decompose nonlinear dynamics into stochastic
switching linear dynamical systems. Moreover, we show how this framework of
switching models enables extracting hierarchies of Markovian and
auto-regressive locally linear controllers from nonlinear experts in an
imitation learning scenario.Comment: 2nd Annual Conference on Learning for Dynamics and Contro
Locally-Stable Macromodels of Integrated Digital Devices for Multimedia Applications
This paper addresses the development of accurate and efficient behavioral models of digital integrated circuits for the assessment of high-speed systems. Device models are based on suitable parametric expressions estimated from port transient responses and are effective at system level, where the quality of functional signals and the impact of supply noise need to be simulated. A potential limitation of some state-of-the-art modeling techniques resides in hidden instabilities manifesting themselves in the use of models, without being evident in the building phase of the same models. This contribution compares three recently-proposed model structures, and selects the local-linear state-space modeling technique as an optimal candidate for the signal integrity assessment of data links. In fact, this technique combines a simple verification of the local stability of models with a limited model size and an easy implementation in commercial simulation tools. An application of the proposed methodology to a real problem involving commercial devices and a data-link of a wireless device demonstrates the validity of this approac
Echo State Networks: analysis, training and predictive control
The goal of this paper is to investigate the theoretical properties, the
training algorithm, and the predictive control applications of Echo State
Networks (ESNs), a particular kind of Recurrent Neural Networks. First, a
condition guaranteeing incremetal global asymptotic stability is devised. Then,
a modified training algorithm allowing for dimensionality reduction of ESNs is
presented. Eventually, a model predictive controller is designed to solve the
tracking problem, relying on ESNs as the model of the system. Numerical results
concerning the predictive control of a nonlinear process for pH neutralization
confirm the effectiveness of the proposed algorithms for the identification,
dimensionality reduction, and the control design for ESNs.Comment: 6 pages,5 figures, submitted to European Control Conference (ECC
Online Natural Gradient as a Kalman Filter
We cast Amari's natural gradient in statistical learning as a specific case
of Kalman filtering. Namely, applying an extended Kalman filter to estimate a
fixed unknown parameter of a probabilistic model from a series of observations,
is rigorously equivalent to estimating this parameter via an online stochastic
natural gradient descent on the log-likelihood of the observations.
In the i.i.d. case, this relation is a consequence of the "information
filter" phrasing of the extended Kalman filter. In the recurrent (state space,
non-i.i.d.) case, we prove that the joint Kalman filter over states and
parameters is a natural gradient on top of real-time recurrent learning (RTRL),
a classical algorithm to train recurrent models.
This exact algebraic correspondence provides relevant interpretations for
natural gradient hyperparameters such as learning rates or initialization and
regularization of the Fisher information matrix.Comment: 3rd version: expanded intr
- …