Reinforcement Learning, Intelligent Control and their Applications in Connected and Autonomous Vehicles

Reinforcement learning (RL) has attracted large attention over the past few years. Recently, we developed a data-driven algorithm to solve predictive cruise control (PCC) and games output regulation problems. This work integrates our recent contributions to the application of RL in game theory, output regulation problems, robust control, small-gain theory and PCC. The algorithm was developed for $H_\infty$ adaptive optimal output regulation of uncertain linear systems, and uncertain partially linear systems to reject disturbance and also force the output of the systems to asymptotically track a reference. In the PCC problem, we determined the reference velocity for each autonomous vehicle in the platoon using the traffic information broadcasted from the lights to reduce the vehicles\u27 trip time. Then we employed the algorithm to design an approximate optimal controller for the vehicles. This controller is able to regulate the headway, velocity and acceleration of each vehicle to the desired values. Simulation results validate the effectiveness of the algorithms

Fast and accurate trajectory tracking control of an autonomous surface vehicle with unmodeled dynamics and disturbances

In this paper, fast and accurate trajectory tracking control of an autonomous surface vehicle (ASV) with complex unknowns including unmodeled dynamics, uncertainties and/or unknown disturbances is addressed within a proposed homogeneity-based finite-time control (HFC) framework. Major contributions are as follows: (1) In the absence of external disturbances, a nominal HFC framework is established to achieve exact trajectory tracking control of an ASV, whereby global finitetime stability is ensured by combining homogeneous analysis and Lyapunov approach; (2) Within the HFC scheme, a finite-time disturbance observer (FDO) is further nested to rapidly and accurately reject complex disturbances, and thereby contributing to an FDO-based HFC (FDO-HFC) scheme which can realize exactness of trajectory tracking and disturbance observation; (3) Aiming to exactly deal with complicated unknowns including unmodeled dynamics and/or disturbances, a finite-time unknown observer (FUO) is deployed as a patch for the nominal HFC framework, and eventually results in an FUO-based HFC (FUOHFC) scheme which guarantees that accurate trajectory tracking can be achieved for an ASV under harsh environments. Simulation studies and comprehensive comparisons conducted on a benchmark ship demonstrate the effectiveness and superiority of the proposed HFC schemes

Robust Neural Network RISE Observer Based Fault Diagnostics And Prediction

A novel fault diagnostics and prediction scheme in continuous time is introduced for a class of nonlinear systems. The proposed method uses a novel neural network (NN) based robust integral sign of the error (RISE) observer, or estimator, allowing for semi-global asymptotic stability in the presence of NN approximation errors, disturbances and unmodeled dynamics. This is in comparison to typical results presented in the literature that show only boundedness in the presence of uncertainties. The output of the observer/estimator is compared with that of the nonlinear system and a residual is used for declaring the presence of a fault when the residual exceeds a user defined threshold. The NN weights are tuned online with no offline tuning phase. The output of the RISE observer is utilized for diagnostics. Additionally, a method for time-to-failure (TTF) prediction, a first step in prognostics, is developed by projecting the developed parameter-update law under the assumption that the nonlinear system satisfies a linear-in-the-parameters (LIP) assumption. The TTF method uses known critical values of a system to predict when an estimated parameter will reach a known failure threshold. The performance of the NN/RISE observer system is evaluated on a nonlinear system and a simply supported beam finite element analysis (FEA) simulation based on laboratory experiments. Results show that the proposed method provides as much as 25% increased accuracy while the TTF scheme renders a more accurate prediction. © 2010 IEEE

Discrete-time neural network based state observer with neural network based control formulation for a class of systems with unmatched uncertainties

An observer is a dynamic system that estimates the state variables of another system using noisy measurements, either to estimate unmeasurable states, or to improve the accuracy of the state measurements. The Modified State Observer (MSO) is a technique that uses a standard observer structure modified to include a neural network to estimate system states as well as system uncertainty. It has been used in orbit uncertainty estimation and atmospheric reentry uncertainty estimation problems to correctly estimate unmodeled system dynamics. A form of the MSO has been used to control a nonlinear electrohydraulic system with parameter uncertainty using a simplified linear model. In this paper an extension of the MSO into discrete-time is developed using Lyapunov stability theory. Discrete-time systems are found in all digital hardware implementations, such as that found in a Martian rover, a quadcopter UAV, or digital flight control systems, and have the added benefit of reduced computation time compared to continuous systems. The derived adaptive update law guarantees stability of the error dynamics and boundedness of the neural network weights. To prove the validity of the discrete-time MSO (DMSO) simulation studies are performed using a two wheeled inverted pendulum (TWIP) robot, an unstable nonlinear system with unmatched uncertainties. Using a linear model with parameter uncertainties, the DMSO is shown to correctly estimate the state of the system as well as the system uncertainty, providing state estimates orders of magnitude more accurate, and in periods of time up to 10 times faster than the Discrete Kalman Filter. The DMSO is implemented on an actual TWIP robot to further validate the performance and demonstrate the applicability to discrete-time systems found in many aerospace applications. Additionally, a new form of neural network control is developed to compensate for the unmatched uncertainties that exist in the TWIP system using a state variable as a virtual control input. It is shown that in all cases the neural network based control assists with the controller effectiveness, resulting in the most effective controller, performing on average 53.1% better than LQR control alone --Abstract, page iii

Investigation of Air Transportation Technology at Princeton University, 1989-1990

The Air Transportation Technology Program at Princeton University proceeded along six avenues during the past year: microburst hazards to aircraft; machine-intelligent, fault tolerant flight control; computer aided heuristics for piloted flight; stochastic robustness for flight control systems; neural networks for flight control; and computer aided control system design. These topics are briefly discussed, and an annotated bibliography of publications that appeared between January 1989 and June 1990 is given