Solving optimal control problems governed by random Navier-Stokes equations using low-rank methods

Many problems in computational science and engineering are simultaneously characterized by the following challenging issues: uncertainty, nonlinearity, nonstationarity and high dimensionality. Existing numerical techniques for such models would typically require considerable computational and storage resources. This is the case, for instance, for an optimization problem governed by time-dependent Navier-Stokes equations with uncertain inputs. In particular, the stochastic Galerkin finite element method often leads to a prohibitively high dimensional saddle-point system with tensor product structure. In this paper, we approximate the solution by the low-rank Tensor Train decomposition, and present a numerically efficient algorithm to solve the optimality equations directly in the low-rank representation. We show that the solution of the vorticity minimization problem with a distributed control admits a representation with ranks that depend modestly on model and discretization parameters even for high Reynolds numbers. For lower Reynolds numbers this is also the case for a boundary control. This opens the way for a reduced-order modeling of the stochastic optimal flow control with a moderate cost at all stages.Comment: 29 page

Mitigating the Curse of Dimensionality: Sparse Grid Characteristics Method for Optimal Feedback Control and HJB Equations

We address finding the semi-global solutions to optimal feedback control and the Hamilton--Jacobi--Bellman (HJB) equation. Using the solution of an HJB equation, a feedback optimal control law can be implemented in real-time with minimum computational load. However, except for systems with two or three state variables, using traditional techniques for numerically finding a semi-global solution to an HJB equation for general nonlinear systems is infeasible due to the curse of dimensionality. Here we present a new computational method for finding feedback optimal control and solving HJB equations which is able to mitigate the curse of dimensionality. We do not discretize the HJB equation directly, instead we introduce a sparse grid in the state space and use the Pontryagin's maximum principle to derive a set of necessary conditions in the form of a boundary value problem, also known as the characteristic equations, for each grid point. Using this approach, the method is spatially causality free, which enjoys the advantage of perfect parallelism on a sparse grid. Compared with dense grids, a sparse grid has a significantly reduced size which is feasible for systems with relatively high dimensions, such as the $6$-D system shown in the examples. Once the solution obtained at each grid point, high-order accurate polynomial interpolation is used to approximate the feedback control at arbitrary points. We prove an upper bound for the approximation error and approximate it numerically. This sparse grid characteristics method is demonstrated with two examples of rigid body attitude control using momentum wheels

Computational optimal control of the terminal bunt manoeuvre

This work focuses on a study of missile guidance in the form of trajectory shaping of a generic cruise missile attacking a fixed target which must be struck from above. The problem is reinterpreted using optimal control theory resulting in two formulations: I) minimum time-integrated altitude and 2) minimum flight time. Each formulation entails nonlinear, two-dimensional missile flight dynamics, boundary conditions and path constraints. Since the thus obtained optimal control problems do not admit analytical solutions, a recourse to computational optimal control is made. The focus here is on informed use of the tools of computational optimal control, rather than their development. Each of the formulations is solved using a three-stage approach. In stage I, the problem is discretised, effectively transforming it into a nonlinear programming problem, and hence suitable for approximate solution with the FORTRAN packages DIRCOL and NUDOCCCS. The results of this direct approach are used to discern the structure of the optimal solution, i.e. type of constraints active, time of their activation, switching and jump points. This qualitative analysis, employing the results of stage I and optimal control theory, constitutes stage 2. Finally, in stage 3, the insight of stage 2 are made precise by rigorous mathemati cal formulation of the relevant two-point boundary value problems (TPBVPs), using the appropriate theorems of optimal control theory. The TPBVPs obtained from this indirect approach are then solved using the FORTRAN package BNDSCO and the results compared with the appropriate solutions of stage I. For each formulation (minimum altitude and minimum time) the influence of boundary conditions on the structure of the optimal solution and the performance index is investigated. The results are then interpreted from the operational and computational perspectives. Software implementation employing DIRCOL, NUDOCCCS and BNDSCO, which produced the results, is described and documented. Finally, some conclusions are drawn and recommendations made

Deep neural network approximations for the stable manifolds of the Hamilton-Jacobi equations

As the Riccati equation for control of linear systems, the Hamilton-Jacobi-Bellman (HJB) equations play a fundamental role for optimal control of nonlinear systems. For infinite-horizon optimal control, the stabilizing solution of HJB equation can be represented by the stable manifold of the associated Hamiltonian system. In this paper, we study the neural network (NN) semiglobal approximation of the stable manifold. The main contribution includes two aspects: firstly, from the mathematical point of view, we rigorously prove that if an approximation is sufficiently close to the exact stable manifold of the HJB equation, then the corresponding control derived from this approximation is near optimal. Secondly, we propose a deep learning method to approximate the stable manifolds, and then numerically compute optimal feedback controls. The algorithm is devised from geometric features of the stable manifold, and relies on adaptive data generation by finding trajectories randomly in the stable manifold. The trajectories are found by solving two-point boundary value problems (BVP) locally near the equilibrium and extending the local solution by initial value problems (IVP) for the associated Hamiltonian system. A number of samples are chosen on each trajectory. Some adaptive samples are selected near the points with large errors after the previous round of training. Our algorithm is causality-free basically, hence it has a potential to apply to various high-dimensional nonlinear systems. We illustrate the effectiveness of our method by stabilizing the Reaction Wheel Pendulums.Comment: The algorithm is modified. The main point is that the trajectories on stable manifold are found by a combination of two-point BVP near the equilibrium and initial value problem far away from the equilibrium. The algorithm becomes more effectiv

An Overview of Quasi-Monte Carlo Methods in Control Systems

Many control problems are so complex that analytic techniques fail to solve them [2]. Furthermore, even when analytic solutions are available, they may be computationally costly [2] and generally result in very high-order compensators [3]. Due to these reasons, we tend to accept approximate answers which provide us with certain performance guarantees for such problems. Sampling methods thus come into the picture to try and remedy the “cost of solution” problem by drawing samples from an appropriate space, and providing an approximate answer. For many years, random sampling has dominated the afore mentioned arena [8, 11, 4]. Random sample generation, with a uniform underlying distribution, however tends to cluster the samples on the boundary of the sample space in higher dimensions. It is for this reason that we are interested in presenting a method that distributes the points regularly in the sample space while providing deterministic guarantees on the error involved. Recently, deterministic or quasi-Monte Carlo (QMC) methods have proven superior to random methods in several applications such as the calculation of certain integrals [6], financial derivatives [7] and motion planning in robotics [10]. They have also been used for stability analysis of high speed networks [9]. In this work, we provide an overview of such deterministic quasi-Monte Carlo method of sampling, and their applications to control systems analysis and design. We present the basic concepts pertaining to quasi-Monte Carlo deterministic sampling. Such concepts include the following: Indicator functions, performance objective, generation of point sets, total variation, and error bounds

Optimal Direction-Dependent Path Planning for Autonomous Vehicles

The focus of this thesis is optimal path planning. The path planning problem is posed as an optimal control problem, for which the viscosity solution to the static Hamilton-Jacobi-Bellman (HJB) equation is used to determine the optimal path. The Ordered Upwind Method (OUM) has been previously used to numerically approximate the viscosity solution of the static HJB equation for direction-dependent weights. The contributions of this thesis include an analytical bound on the convergence rate of the OUM for the boundary value problem to the viscosity solution of the HJB equation. The convergence result provided in this thesis is to our knowledge the tightest existing bound on the convergence order of OUM solutions to the viscosity solution of the static HJB equation. Only convergence without any guarantee of rate has been previously shown. Navigation functions are often used to provide controls to robots. These functions can suffer from local minima that are not also global minima, which correspond to the inability to find a path at those minima. Provided the weight function is positive, the viscosity solution to the static HJB equation cannot have local minima. Though this has been discussed in literature, a proof has not yet appeared. The solution of the HJB equation is shown in this work to have no local minima that is not also global. A path can be found using this method. Though finding the shortest path is often considered in optimal path planning, safe and energy efficient paths are required for rover path planning. Reducing instability risk based on tip-over axes and maximizing solar exposure are important to consider in achieving these goals. In addition to obstacle avoidance, soil risk and path length on terrain are considered. In particular, the tip-over instability risk is a direction-dependent criteria, for which accurate approximate solutions to the static HJB equation cannot be found using the simpler Fast Marching Method. An extension of the OUM to include a bi-directional search for the source-point path planning problem is also presented. The solution is found on a smaller region of the environment, containing the optimal path. Savings in computational time are observed. A comparison is made in the path planning problem in both timing and performance between a genetic algorithm rover path planner and OUM. A comparison in timing and number of updates required is made between OUM and several other algorithms that approximate the same static HJB equation. Finally, the OUM algorithm solving the boundary value problem is shown to converge numerically with the rate of the proven theoretical bound