663 research outputs found
Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillation
The control of nonlinear dynamical systems remains a major challenge for
autonomous agents. Current trends in reinforcement learning (RL) focus on
complex representations of dynamics and policies, which have yielded impressive
results in solving a variety of hard control tasks. However, this new
sophistication and extremely over-parameterized models have come with the cost
of an overall reduction in our ability to interpret the resulting policies. In
this paper, we take inspiration from the control community and apply the
principles of hybrid switching systems in order to break down complex dynamics
into simpler components. We exploit the rich representational power of
probabilistic graphical models and derive an expectation-maximization (EM)
algorithm for learning a sequence model to capture the temporal structure of
the data and automatically decompose nonlinear dynamics into stochastic
switching linear dynamical systems. Moreover, we show how this framework of
switching models enables extracting hierarchies of Markovian and
auto-regressive locally linear controllers from nonlinear experts in an
imitation learning scenario.Comment: 2nd Annual Conference on Learning for Dynamics and Contro
Reinforcement Learning Adaptive PID Controller for an Under-actuated Robot Arm
Abstract: An adaptive PID controller is used to control of a two degrees of freedom under actuated manipulator. An actor-critic based reinforcement learning is employed for tuning of parameters of the adaptive PID controller. Reinforcement learning is an unsupervised scheme wherein no reference exists to which convergence of algorithm is anticipated. Thus, it is appropriate for real time applications. Controller structure and learning equations as well as update rules are provided. Simulations are performed in SIMULINK and performance of the controller is compared with NARMA-L2 controller. The results verified good performance of the controller in tracking and disturbance rejection tests
Neural Lyapunov Control
We propose new methods for learning control policies and neural network
Lyapunov functions for nonlinear control problems, with provable guarantee of
stability. The framework consists of a learner that attempts to find the
control and Lyapunov functions, and a falsifier that finds counterexamples to
quickly guide the learner towards solutions. The procedure terminates when no
counterexample is found by the falsifier, in which case the controlled
nonlinear system is provably stable. The approach significantly simplifies the
process of Lyapunov control design, provides end-to-end correctness guarantee,
and can obtain much larger regions of attraction than existing methods such as
LQR and SOS/SDP. We show experiments on how the new methods obtain high-quality
solutions for challenging control problems.Comment: NeurIPS 201
- …