13,231 research outputs found
Control of Complex Dynamic Systems by Neural Networks
This paper considers the use of neural networks (NN's) in controlling a nonlinear, stochastic system with unknown process equations. The NN is used to model the resulting unknown control law. The approach here is based on using the output error of the system to train the NN controller without the need to construct a separate model (NN or other type) for the unknown process dynamics. To implement such a direct adaptive control approach, it is required that connection weights in the NN be estimated while the system is being controlled. As a result of the feedback of the unknown process dynamics, however, it is not possible to determine the gradient of the loss function for use in standard (back-propagation-type) weight estimation algorithms. Therefore, this paper considers the use of a new stochastic approximation algorithm for this weight estimation, which is based on a 'simultaneous perturbation' gradient approximation that only requires the system output error. It is shown that this algorithm can greatly enhance the efficiency over more standard stochastic approximation algorithms based on finite-difference gradient approximations
Reinforcement Learning Based on Real-Time Iteration NMPC
Reinforcement Learning (RL) has proven a stunning ability to learn optimal
policies from data without any prior knowledge on the process. The main
drawback of RL is that it is typically very difficult to guarantee stability
and safety. On the other hand, Nonlinear Model Predictive Control (NMPC) is an
advanced model-based control technique which does guarantee safety and
stability, but only yields optimality for the nominal model. Therefore, it has
been recently proposed to use NMPC as a function approximator within RL. While
the ability of this approach to yield good performance has been demonstrated,
the main drawback hindering its applicability is related to the computational
burden of NMPC, which has to be solved to full convergence. In practice,
however, computationally efficient algorithms such as the Real-Time Iteration
(RTI) scheme are deployed in order to return an approximate NMPC solution in
very short time. In this paper we bridge this gap by extending the existing
theoretical framework to also cover RL based on RTI NMPC. We demonstrate the
effectiveness of this new RL approach with a nontrivial example modeling a
challenging nonlinear system subject to stochastic perturbations with the
objective of optimizing an economic cost.Comment: accepted for the IFAC World Congress 202
Stochastic Nonlinear Model Predictive Control with Efficient Sample Approximation of Chance Constraints
This paper presents a stochastic model predictive control approach for
nonlinear systems subject to time-invariant probabilistic uncertainties in
model parameters and initial conditions. The stochastic optimal control problem
entails a cost function in terms of expected values and higher moments of the
states, and chance constraints that ensure probabilistic constraint
satisfaction. The generalized polynomial chaos framework is used to propagate
the time-invariant stochastic uncertainties through the nonlinear system
dynamics, and to efficiently sample from the probability densities of the
states to approximate the satisfaction probability of the chance constraints.
To increase computational efficiency by avoiding excessive sampling, a
statistical analysis is proposed to systematically determine a-priori the least
conservative constraint tightening required at a given sample size to guarantee
a desired feasibility probability of the sample-approximated chance constraint
optimization problem. In addition, a method is presented for sample-based
approximation of the analytic gradients of the chance constraints, which
increases the optimization efficiency significantly. The proposed stochastic
nonlinear model predictive control approach is applicable to a broad class of
nonlinear systems with the sufficient condition that each term is analytic with
respect to the states, and separable with respect to the inputs, states and
parameters. The closed-loop performance of the proposed approach is evaluated
using the Williams-Otto reactor with seven states, and ten uncertain parameters
and initial conditions. The results demonstrate the efficiency of the approach
for real-time stochastic model predictive control and its capability to
systematically account for probabilistic uncertainties in contrast to a
nonlinear model predictive control approaches.Comment: Submitted to Journal of Process Contro
A machine learning framework for data driven acceleration of computations of differential equations
We propose a machine learning framework to accelerate numerical computations
of time-dependent ODEs and PDEs. Our method is based on recasting
(generalizations of) existing numerical methods as artificial neural networks,
with a set of trainable parameters. These parameters are determined in an
offline training process by (approximately) minimizing suitable (possibly
non-convex) loss functions by (stochastic) gradient descent methods. The
proposed algorithm is designed to be always consistent with the underlying
differential equation. Numerical experiments involving both linear and
non-linear ODE and PDE model problems demonstrate a significant gain in
computational efficiency over standard numerical methods
- …