HJB-RBF Based Approach for the Control of PDEs

Semi-Lagrangian schemes for the discretization of the dynamic programming principle are based on a time discretization projected on a state-space grid. The use of a structured grid makes this approach not feasible for high-dimensional problems due to the curse of dimensionality. Here, we present a new approach for infinite horizon optimal control problems where the value function is computed using radial basis functions by the Shepard moving least squares approximationmethod on scattered grids.We propose a newmethod to generate a scattered mesh driven by the dynamics and the selection of the shape parameter in the RBF using an optimization routine. This mesh will help to localize the problem and approximate the dynamic programming principle in high dimension. Error estimates for the value function are also provided. Numerical tests for high dimensional problems will show the effectiveness of the proposed method

Neural network solution for suboptimal control of non-holonomic chained form system

In this paper, we develop fixed-final time nearly optimal control laws for a class of non-holonomic chained form systems by using neural networks to approximately solve a Hamilton-Jacobi-Bellman equation. A certain time-folding method is applied to recover uniform complete controllability for the chained form system. This method requires an innovative design of a certain dynamic control component. Using this time-folding method, the chained form system is mapped into a controllable linear system for which controllers can systematically be designed to ensure exponential or asymptotic stability as well as nearly optimal performance. The result is a neural network feedback controller that has time-varying coefficients found by a priori offline tuning. The results of this paper are demonstrated in an example

A sparse Markov chain approximation of LQ-type stochastic control problems

We propose a novel Galerkin discretization scheme for stochastic optimal control problems on an indefinite time horizon. The control problems are linear-quadratic in the controls, but possibly nonlinear in the state variables, and the discretization is based on the fact that problems of this kind admit a dual formulation in terms of linear boundary value problems. We show that the discretized linear problem is dual to a Markov decision problem, prove an $L^{2}$ error bound for the general scheme and discuss the sparse discretization using a basis of so-called committor functions as a special case; the latter is particularly suited when the dynamics are metastable, e.g., when controlling biomolecular systems. We illustrate the method with several numerical examples, one being the optimal control of Alanine dipeptide to its helical conformation

Solving Partial Differential Equations Using Artificial Neural Networks

<p>This thesis presents a method for solving partial differential equations (PDEs) using articial neural networks. The method uses a constrained backpropagation (CPROP) approach for preserving prior knowledge during incremental training for solving nonlinear elliptic and parabolic PDEs adaptively, in non-stationary environments. Compared to previous methods that use penalty functions or Lagrange multipliers,</p><p>CPROP reduces the dimensionality of the optimization problem by using direct elimination, while satisfying the equality constraints associated with the boundary and initial conditions exactly, at every iteration of the algorithm. The effectiveness of this method is demonstrated through several examples, including nonlinear elliptic</p><p>and parabolic PDEs with changing parameters and non-homogeneous terms. The computational complexity analysis shows that CPROP compares favorably to existing methods of solution, and that it leads to considerable computational savings when subject to non-stationary environments.</p><p>The CPROP based approach is extended to a constrained integration (CINT) method for solving initial boundary value partial differential equations (PDEs). The CINT method combines classical Galerkin methods with CPROP in order to constrain the ANN to approximately satisfy the boundary condition at each stage of integration. The advantage of the CINT method is that it is readily applicable to PDEs in irregular domains and requires no special modification for domains with complex geometries. Furthermore, the CINT method provides a semi-analytical solution that is infinitely differentiable. The CINT method is demonstrated on two hyperbolic and one parabolic initial boundary value problems (IBVPs). These IBVPs are widely used and have known analytical solutions. When compared with Matlab's nite element (FE) method, the CINT method is shown to achieve significant improvements both in terms of computational time and accuracy. The CINT method is applied to a distributed optimal control (DOC) problem of computing optimal state and control trajectories for a multiscale dynamical system comprised of many interacting dynamical systems, or agents. A generalized reduced gradient (GRG) approach is presented in which the agent dynamics are described by a small system of stochastic dierential equations (SDEs). A set of optimality conditions is derived using calculus of variations, and used to compute the optimal macroscopic state and microscopic control laws. An indirect GRG approach is used to solve the optimality conditions numerically for large systems of agents. By assuming a parametric control law obtained from the superposition of linear basis functions, the agent control laws can be determined via set-point regulation, such</p><p>that the macroscopic behavior of the agents is optimized over time, based on multiple, interactive navigation objectives.</p><p>Lastly, the CINT method is used to identify optimal root profiles in water limited ecosystems. Knowledge of root depths and distributions is vital in order to accurately model and predict hydrological ecosystem dynamics. Therefore, there is interest in accurately predicting distributions for various vegetation types, soils, and climates. Numerical experiments were were performed that identify root profiles that maximize transpiration over a 10 year period across a transect of the Kalahari. Storm types were varied to show the dependence of the optimal profile on storm frequency and intensity. It is shown that more deeply distributed roots are optimal for regions where</p><p>storms are more intense and less frequent, and shallower roots are advantageous in regions where storms are less intense and more frequent.</p>Dissertatio

Feynman-Kac Numerical Techniques for Stochastic Optimal Control

Three significant advancements are proposed for improving numerical methods in the solution of forward-backward stochastic differential equations (FBSDEs) appearing in the Feynman-Kac representation of the value function in stochastic optimal control (SOC) problems. First, we propose a novel characterization of FBSDE estimators as either on-policy or off-policy, highlighting the intuition for these techniques that the distribution over which value functions are approximated should, to some extent, match the distribution the policies generate. Second, two novel numerical estimators are proposed for improving the accuracy of single-timestep updates. In the case of LQR problems, we demonstrate both in theory and in numerical simulation that our estimators result in near machine-precision level accuracy, in contrast to previously proposed methods that can potentially diverge on the same problems. Third, we propose a new method for accelerating the global convergence of FBSDE methods. By the repeated use of the Girsanov change of probability measures, it is demonstrated how a McKean-Markov branched sampling method can be utilized for the forward integration pass, as long as the controlled drift terms are appropriately compensated in the backward integration pass. Subsequently, a numerical approximation of the value function is proposed by solving a series of function approximation problems backwards in time along the edges of a space-filling tree.Ph.D

Formation control of mobile robots and unmanned aerial vehicles

In this dissertation, the nonlinear control of nonholonomic mobile robot formations and unmanned aerial vehicle (UAV) formations is undertaken and presented in six papers. In the first paper, an asymptotically stable combined kinematic/torque control law is developed for leader-follower based formation control of mobile robots using backstepping. A neural network (NN) is introduced along with robust integral of the sign of the error (RISE) feedback to approximate the dynamics of the follower as well as its leader using online weight tuning. Subsequently, in the second paper, a novel NN observer is designed to estimate the linear and angular velocities of both the follower and its leader robot and a NN output feedback control law is developed. On the other hand, in the third paper, a NN-based output feedback control law is presented for the control of an underactuated quad rotor UAV, and a NN virtual control input scheme is proposed which allows all six degrees of freedom to be controlled using only four control inputs. The results of this paper are extended to include the control of quadrotor UAV formations, and a novel three-dimensional leader-follower framework is proposed in the fourth paper. Next, in the fifth paper, the discrete-time nonlinear optimal control is undertaken using two online approximators (OLA\u27s) to solve the infinite horizon Hamilton-Jacobi-Bellman (HJB) equation forward-in-time to achieve nearly optimal regulation and tracking control. In contrast, paper six utilizes a single OLA to solve the infinite horizon HJB and Hamilton-Jacobi-Isaacs (HJI) equations forward-intime for the near optimal regulation and tracking control of continuous affine nonlinear systems. The effectiveness of the optimal tracking controllers proposed in the fifth and sixth papers are then demonstrated using nonholonomic mobile robot formation control --Abstract, page iv

Optimal control of a motor-integrated hybrid powertrain for a two-wheeled vehicle suitable for personal transportation

The present research aims to propose an optimized configuration of the motor integrated power-train with an optimal controller suitable for small power-train based two wheeler automobile which can increase the system level efficiency without affecting drivability. This work will be the foundation for realizing the system in a production ready vehicle for the two wheeler OEM TVS Motor Company in India. A detailed power-train model is developed (from first principles) for the scooter vehicle, which is powered by a 110 cc spark ignition (SI) engine and coupled with two types of transmission, a continuous variable transmission (CVT) and a 4-speed manual transmission (MT). Both models are capable of simulating torque and NOx emission output of the SI engine and dynamic response of the full power-train. The torque production and emission outputs of the model are compared with experimental results available from TVS Motor Company. The CVT gear ratio model is developed using an indirect method and an analytical model. Both types of powertrain models are applied to perform a simulated study of fuel consumption, NOx emission and drivability study for a particular vehicle platform. In the next stage of work, the mathematical model for a brush-less direct current machine (BLDC) with the drive system and Li-Ion battery are developed. The models are verified and calibrated with the experimental results from TVS Motor Company. The BLDC machine is integrated with both the CVT and MT powertrain models in parallel hybrid configurations and a drive cycle simulation is conducted for different static assist levels by the electrical machines. The initial test confirms the need of optimal sizing of the powertrain components as well as an optimal control system. The detailed model of the powertrain is converted to a control-oriented model which is suitable for optimal control. This is followed by multi-objective optimization of different components of the motor-integrated powertrain using a single function as well as Pareto-Optimal methods. The objective function for the multi-objective optimization is proposed to reduce the fuel consumption with battery charge sustainability with least impact on the increase of financial cost and weight of the vehicle. The optimization is conducted by a nested methodology that involves Particle Swarm Optimization and a Non-dominated sorting genetic algorithm where, concurrently, a global optimal control is developed corresponding to the multi-objective design. The global optimal controller is designed using dynamic programming. The research is concluded with an optimal controller developed using the hp-collocation method. The objective function of the dynamic programming method and hp-collocation method is proposed to reduce fuel consumption with battery charge sustainability.Open Acces