Sub-Optimal Tracking in Switched Systems With Controlled Subsystems and Fixed-Mode Sequence Using Approximate Dynamic Programming

Optimal tracking in switched systems with controlled subsystem and Discrete-time (DT) dynamics is investigated. A feedback control policy is generated such that a) the system tracks the desired reference signal, and b) the optimal switching time instants are sought. For finding the optimal solution, approximate dynamic programming is used. Simulation results are provided to illustrate the effectiveness of the solution

Sub-optimal Tracking in Switched Systems with Controlled Subsystems and Fixed-mode Sequence using Approximate Dynamic Programming

Optimal tracking in switched systems with controlled subsystem and Discrete-time (DT) dynamics is investigated. A feedback control policy is generated such that a) the system tracks the desired reference signal, and b) the optimal switching time instants are sought. For finding the optimal solution, approximate dynamic programming is used. Simulation results are provided to illustrate the effectiveness of the solution.Comment: 6 pages, 2 figures, This article has been accepted for oral presentation at 2019 Dynamic System and Control Conference. The content is the same as the final edition of the accepted paper. However, the presentation might be differen

Near-Optimal Control of a Quadcopter Using Reinforcement Learning

This paper presents a novel control method for quadcopters that achieves near-optimal tracking control for input-affine nonlinear quadcopter dynamics. The method uses a reinforcement learning algorithm called Single Network Adaptive Critics (SNAC), which approximates a solution to the discrete-time Hamilton-Jacobi-Bellman (DT-HJB) equation using a single neural network trained offline. The control method involves two SNAC controllers, with the outer loop controlling the linear position and velocities (position control) and the inner loop controlling the angular position and velocities (attitude control). The resulting quadcopter controller provides optimal feedback control and tracks a trajectory for an infinite-horizon, and it is compared with commercial optimal control software. Furthermore, the closed-loop controller can control the system with any initial conditions within the domain of training. Overall, this research demonstrates the benefits of using SNAC for nonlinear control, showing its ability to achieve near-optimal tracking control while reducing computational complexity. This paper provides insights into a new approach for controlling quadcopters, with potential applications in various fields such as aerial surveillance, delivery, and search and rescue

A radial basis function method for solving optimal control problems.

This work presents two direct methods based on the radial basis function (RBF) interpolation and arbitrary discretization for solving continuous-time optimal control problems: RBF Collocation Method and RBF-Galerkin Method. Both methods take advantage of choosing any global RBF as the interpolant function and any arbitrary points (meshless or on a mesh) as the discretization points. The first approach is called the RBF collocation method, in which states and controls are parameterized using a global RBF, and constraints are satisfied at arbitrary discrete nodes (collocation points) to convert the continuous-time optimal control problem to a nonlinear programming (NLP) problem. The resulted NLP is quite sparse and can be efficiently solved by well-developed sparse solvers. The second proposed method is a hybrid approach combining RBF interpolation with Galerkin error projection for solving optimal control problems. The proposed solution, called the RBF-Galerkin method, applies a Galerkin projection to the residuals of the optimal control problem that make them orthogonal to every member of the RBF basis functions. Also, RBF-Galerkin costate mapping theorem will be developed describing an exact equivalency between the Karush–Kuhn–Tucker (KKT) conditions of the NLP problem resulted from the RBF-Galerkin method and discretized form of the first-order necessary conditions of the optimal control problem, if a set of conditions holds. Several examples are provided to verify the feasibility and viability of the RBF method and the RBF-Galerkin approach as means of finding accurate solutions to general optimal control problems. Then, the RBF-Galerkin method is applied to a very important drug dosing application: anemia management in chronic kidney disease. A multiple receding horizon control (MRHC) approach based on the RBF-Galerkin method is developed for individualized dosing of an anemia drug for hemodialysis patients. Simulation results are compared with a population-oriented clinical protocol as well as an individual-based control method for anemia management to investigate the efficacy of the proposed method

Modeling, Optimization, and Control of Down-Hole Drilling System

This dissertation investigates dynamics modeling, optimization, and control methodologies of the down-hole drilling system, which can enable a more accurate and reliable automated tracking of drilling trajectory, mitigating drilling vibration, improving the drilling rate, etc. Unlike many existing works, which only consider drilling control in the torsional dimension, the proposed research aims to address the drilling dynamics modeling and control considering both coupled axial and torsional drill string dynamics. The dissertation will first address optimization and control for vertical drilling, and then resolve critical modeling and control challenges for the directional drilling process. In Chapter 2, a customized Dynamic Programming (DP) method is proposed to enable a computationally efficient optimization for the vertical down-hole drilling process. The method is enabled by a new customized DP searching scheme based on a partial inversion of the dynamics model. Through extensive simulation, the method is proved to be effective in searching for an optimal drilling control solution. This method can generate an open-loop optimal control solution, which can be used as a guide for drilling control or in a driller-assist system. In Chapter 3, to enable a closed-loop control solution for the vertical drilling, a neutral- delay differential equations (NDDEs) model based control approach is proposed, specifically to address an axial-torsional coupled vertical drilling dynamics capturing more transient dynamics behaviors through the NDDE. An equivalent input disturbance (EID) approach is used to control the NDDEs model by constructing the Lyapunov-Krasovskii functional (LKF) and formulating them into a linear matrix inequality (LMI). The control gains can be obtained to effectively mitigate the undesired vibrations and maintain accurate trajectory tracking performance under different control references. The works on Chapter 2 and Chapter 3 are mostly for vertical drilling, and the remaining of the dissertation will focus on modeling and control for directional drilling. Chapter 4 proposes a dual heuristic programming (DHP) approach for automated directional drilling control. By approximating the derivative of the cost-to-go function using a neural network (NN), the DHP approach solves the “curse of dimensionality” associated with the traditional DP. The result shows that the proposed controller is robust, computationally efficient, and effective for the directional drilling system. To validate the DHP based control method using a high-fidelity directional drilling model, a hybrid drilling dynamics model is proposed in Chapter 5. The philosophy of the proposed modeling approach is to use the finite element method (FEM) to describe curved sections in the drill string and use the transfer matrix method (TMM) to model straight sections in the drill string. By integrating different methods, we can achieve both modeling accuracy and computational efficiency for a geometrically complex structure. Compared to existing directional drilling models used for off-line analysis, this model can be used for real-time testbeds such as software-in-the-loop (SIL) system and hardware-in-the-loop (HIL) system. Finally, a software-in-the-loop real-time simulation testbed is built to test the designed DHP based controller in Chapter 6. A higher-order hybrid model of directional drilling is implemented in the SIL. The SIL results demonstrate that the designed DHP based controller can effectively mitigate harmful vibrations and accurately track the desired references

Approximate dynamic programming based solutions for fixed-final-time optimal control and optimal switching

Optimal solutions with neural networks (NN) based on an approximate dynamic programming (ADP) framework for new classes of engineering and non-engineering problems and associated difficulties and challenges are investigated in this dissertation. In the enclosed eight papers, the ADP framework is utilized for solving fixed-final-time problems (also called terminal control problems) and problems with switching nature. An ADP based algorithm is proposed in Paper 1 for solving fixed-final-time problems with soft terminal constraint, in which, a single neural network with a single set of weights is utilized. Paper 2 investigates fixed-final-time problems with hard terminal constraints. The optimality analysis of the ADP based algorithm for fixed-final-time problems is the subject of Paper 3, in which, it is shown that the proposed algorithm leads to the global optimal solution providing certain conditions hold. Afterwards, the developments in Papers 1 to 3 are used to tackle a more challenging class of problems, namely, optimal control of switching systems. This class of problems is divided into problems with fixed mode sequence (Papers 4 and 5) and problems with free mode sequence (Papers 6 and 7). Each of these two classes is further divided into problems with autonomous subsystems (Papers 4 and 6) and problems with controlled subsystems (Papers 5 and 7). Different ADP-based algorithms are developed and proofs of convergence of the proposed iterative algorithms are presented. Moreover, an extension to the developments is provided for online learning of the optimal switching solution for problems with modeling uncertainty in Paper 8. Each of the theoretical developments is numerically analyzed using different real-world or benchmark problems --Abstract, page v