Optimal Adaptive Tracking Control Of Partially Uncertain Nonlinear Discrete-Time Systems Using Lifelong Hybrid Learning

This article addresses a multilayer neural network (MNN)-based optimal adaptive tracking of partially uncertain nonlinear discrete-time (DT) systems in affine form. By employing an actor–critic neural network (NN) to approximate the value function and optimal control policy, the critic NN is updated via a novel hybrid learning scheme, where its weights are adjusted once at a sampling instant and also in a finite iterative manner within the instants to enhance the convergence rate. Moreover, to deal with the persistency of excitation (PE) condition, a replay buffer is incorporated into the critic update law through concurrent learning. To address the vanishing gradient issue, the actor and critic MNN weights are tuned using control input and temporal difference errors (TDEs), respectively. In addition, a weight consolidation scheme is incorporated into the critic MNN update law to attain lifelong learning and overcome catastrophic forgetting, thus lowering the cumulative cost. The tracking error, and the actor and critic weight estimation errors are shown to be bounded using the Lyapunov analysis. Simulation results using the proposed approach on a two-link robot manipulator show a significant reduction in tracking error by $44\%$ and cumulative cost by $31\%$ in a multitask environment

Online parameter estimation under non-persistent excitations for high-rate dynamic systems

High-rate dynamic systems are defined as systems experiencing dynamic events of typical amplitudes higher than 100 gn for a duration of less than 100 ms. They are characterized by 1) large uncertainties on the external loads; 2) high levels of nonstationarity and heavy disturbance; and 3) generation of unmodeled dynamics from changes in mechanical configuration. To fully enable these systems, feedback capabilities must be developed. This includes computationally fast software and low latency hardware. This paper presents a pure time-based online parameter estimation algorithm for high-rate dynamic systems with real-time applicability. The algorithm is based on a model reference adaptive system architecture consisting of a reference system and an adaptive model. The adaptive model is built on a reduced order physical representation of the system and uncertainties are linearized. Uncertain coefficients are adapted leveraging instantaneous measurements and historical input–output data sets, termed history stack data, based on concurrent learning theory for coping with the lack of persistent excitation. The history stack is sequentially modified based on a singular value maximizing algorithm to accelerate convergence. The algorithm is numerically verified and experimentally validated on a testbed consisting of a cantilever beam with a moving cart. Numerical verifications show that the algorithm provides fast and accurate convergence when concurrent learning is used. Experimental validations show that the algorithm can successfully identify static positions of the cart, and can also track its movement relatively well, with large chattering and overshoots during travel time. The average computation speed of the algorithm per sample step, implemented in MATLAB, is 93 μs. It is envisioned that the implementation of the algorithm on an FPGA, along with refined coding, will greatly reduce computation time

Control and safety of fully actuated and underactuated nonlinear systems: from adaptation to robustness to optimality

The state-of-the-art quadratic program-based control Lyapunov-control barrier function (QP-CLBF) is a powerful control approach to balance safety and stability in a pointwise optimal fashion. However, under this approach, modeling inaccuracies may degrade the performance of closed-loop systems and cause a violation of safety-critical constraints. This thesis extends the recently-developed QP-CLBF through the derivation of five novel robust quadratic program-based adaptive control approaches for fully actuated and underactuated nonlinear systems with a view toward adapting to unknown parameters, being robust to unmodeled dynamics and disturbances, ensuring the system remains in safe sets and being optimal with respect in a pointwise fashion. Simulation and quantitative results demonstrate the superiority of proposed approaches over the baseline methods.Ph.D