Approximate Predictive Control Barrier Functions using Neural Networks: A Computationally Cheap and Permissive Safety Filter

A predictive control barrier function (PCBF) based safety filter allows for verifying arbitrary control inputs with respect to future constraint satisfaction. The approach relies on the solution of two optimization problems computing the minimal constraint relaxations given the current state, and then computing the minimal deviation from a proposed input such that the relaxed constraints are satisfied. This paper presents an approximation procedure that uses a neural network to approximate the optimal value function of the first optimization problem from samples, such that the computation becomes independent of the prediction horizon. It is shown that this approximation guarantees that states converge to a neighborhood of the implicitly defined safe set of the original problem, where system constraints can be satisfied for all times forward. The convergence result relies on a novel class $\mathcal{K}$ lower bound on the PCBF decrease and depends on the approximation error of the neural network. Lastly, we demonstrate our approach in simulation for an autonomous driving example and show that the proposed approximation leads to a significant decrease in computation time compared to the original approach.Comment: Submitted to ECC2

Optimal control of PDEs using physics-informed neural networks

Physics-informed neural networks (PINNs) have recently become a popular method for solving forward and inverse problems governed by partial differential equations (PDEs). By incorporating the residual of the PDE into the loss function of a neural network-based surrogate model for the unknown state, PINNs can seamlessly blend measurement data with physical constraints. Here, we extend this framework to PDE-constrained optimal control problems, for which the governing PDE is fully known and the goal is to find a control variable that minimizes a desired cost objective. We provide a set of guidelines for obtaining a good optimal control solution; first by selecting an appropriate PINN architecture and training parameters based on a forward problem, second by choosing the best value for a critical scalar weight in the loss function using a simple but effective two-step line search strategy. We then validate the performance of the PINN framework by comparing it to adjoint-based nonlinear optimal control, which performs gradient descent on the discretized control variable while satisfying the discretized PDE. This comparison is carried out on several distributed control examples based on the Laplace, Burgers, Kuramoto-Sivashinsky, and Navier-Stokes equations. Finally, we discuss the advantages and caveats of using the PINN and adjoint-based approaches for solving optimal control problems constrained by nonlinear PDEs.Comment: 27 pages, 10 figures; refined the guidelines and updated the result

Perception-Based Sampled-Data Optimization of Dynamical Systems

Motivated by perception-based control problems in autonomous systems, this paper addresses the problem of developing feedback controllers to regulate the inputs and the states of a dynamical system to optimal solutions of an optimization problem when one has no access to exact measurements of the system states. In particular, we consider the case where the states need to be estimated from high-dimensional sensory data received only at discrete time intervals. We develop a sampled-data feedback controller that is based on adaptations of a projected gradient descent method, and that includes neural networks as integral components to estimate the state of the system from perceptual information. We derive sufficient conditions to guarantee (local) input-to-state stability of the control loop. Moreover, we show that the interconnected system tracks the solution trajectory of the underlying optimization problem up to an error that depends on the approximation errors of the neural network and on the time-variability of the optimization problem; the latter originates from time-varying safety and performance objectives, input constraints, and unknown disturbances. As a representative application, we illustrate our results with numerical simulations for vision-based autonomous driving.Comment: This is an extended version of the paper accepted to IFAC World Congress 2023 for publication, containing proof

Neural Networks: Training and Application to Nonlinear System Identification and Control

This dissertation investigates training neural networks for system identification and classification. The research contains two main contributions as follow:1. Reducing number of hidden layer nodes using a feedforward componentThis research reduces the number of hidden layer nodes and training time of neural networks to make them more suited to online identification and control applications by adding a parallel feedforward component. Implementing the feedforward component with a wavelet neural network and an echo state network provides good models for nonlinear systems.The wavelet neural network with feedforward component along with model predictive controller can reliably identify and control a seismically isolated structure during earthquake. The network model provides the predictions for model predictive control. Simulations of a 5-story seismically isolated structure with conventional lead-rubber bearings showed significant reductions of all response amplitudes for both near-field (pulse) and far-field ground motions, including reduced deformations along with corresponding reduction in acceleration response. The controller effectively regulated the apparent stiffness at the isolation level. The approach is also applied to the online identification and control of an unmanned vehicle. Lyapunov theory is used to prove the stability of the wavelet neural network and the model predictive controller. 2. Training neural networks using trajectory based optimization approachesTraining neural networks is a nonlinear non-convex optimization problem to determine the weights of the neural network. Traditional training algorithms can be inefficient and can get trapped in local minima. Two global optimization approaches are adapted to train neural networks and avoid the local minima problem. Lyapunov theory is used to prove the stability of the proposed methodology and its convergence in the presence of measurement errors. The first approach transforms the constraint satisfaction problem into unconstrained optimization. The constraints define a quotient gradient system (QGS) whose stable equilibrium points are local minima of the unconstrained optimization. The QGS is integrated to determine local minima and the local minimum with the best generalization performance is chosen as the optimal solution. The second approach uses the QGS together with a projected gradient system (PGS). The PGS is a nonlinear dynamical system, defined based on the optimization problem that searches the components of the feasible region for solutions. Lyapunov theory is used to prove the stability of PGS and QGS and their stability under presence of measurement noise

Direct state feedback optimal control of a double integrator plant implemented by an artificial neural network

The purpose of this paper is to assess the capability of an artificial neural network (ANN) to implement a nonlinear state feedback optimal control law for a double integrator plant. In this case, the cost function to be minimised is the settling time subject to control saturation constraints. The reason for selection of this cost function is that the control law is known in the analytical form and this will be used to form a benchmark. The ultimate aim is to apply the method to form a new direct state feedback optimal position control law for mechanisms in which the frictional energy loss is minimised. An analytical solution is not available in this case so first the time optimal control law is studied to enable straightforward comparison on the ANN and directly implemented closed loop control laws. Since Pontryagin‟s method will be used to compute the optimal state trajectories for the ANN training in the future investigation of the minimum energy loss control, this method is applied to derive the time optimal double integrator state trajectories to illustrate the method. Furthermore, a modification of the time optimal control law is made that avoids the control chatter following a position change that would occur if a practical implementation of the basic control law, which is bang-bang, were to be attempted. Training the ANN with state and control data could be inaccurate due to the discontinuity of the control law on the switching boundary in the state space. This problem is overcome by the authors by instead training the ANN with state and switching function data, as the switching function is nonlinear but continuous, the control function, i.e., the function relating the switching function output to the control variable, being externally implemented. The simulations confirm that the ANN can be trained to accurately reproduce the time optimal control

Optimal Power Management Based on Q-Learning and Neuro-Dynamic Programming for Plug-in Hybrid Electric Vehicles

Energy optimization for plug-in hybrid electric vehicles (PHEVs) is a challenging problem due to its system complexity and various constraints. In this research, we present a Q-learning based in-vehicle model-free solution that can robustly converge to the optimal control. The proposed algorithms combine neuro-dynamic programming (NDP) with future trip information to effectively estimate the expected future energy cost (expected cost-to-go) for a given vehicle state and control actions. The convergence of those learning algorithms is demonstrated on both fixed and randomly selected drive cycles. Based on the characteristics of these learning algorithms, we propose a two-stage deployment solution for PHEV power management applications. We will also introduce a new initialization strategy that combines optimal learning with a properly selected penalty function. Such initialization can reduce the learning convergence time by 70%, which has huge impact on in-vehicle implementation. Finally, we develop a neural network (NN) for the battery state-of-charge (SoC) prediction, rendering our power management controller completely model-free.Ph.D.College of Engineering & Computer ScienceUniversity of Michigan-Dearbornhttps://deepblue.lib.umich.edu/bitstream/2027.42/140754/1/Chang Liu Final Dissertation.pdfDescription of Chang Liu Final Dissertation.pdf : Dissertatio