Mitigating the Curse of Dimensionality: Sparse Grid Characteristics Method for Optimal Feedback Control and HJB Equations

We address finding the semi-global solutions to optimal feedback control and the Hamilton--Jacobi--Bellman (HJB) equation. Using the solution of an HJB equation, a feedback optimal control law can be implemented in real-time with minimum computational load. However, except for systems with two or three state variables, using traditional techniques for numerically finding a semi-global solution to an HJB equation for general nonlinear systems is infeasible due to the curse of dimensionality. Here we present a new computational method for finding feedback optimal control and solving HJB equations which is able to mitigate the curse of dimensionality. We do not discretize the HJB equation directly, instead we introduce a sparse grid in the state space and use the Pontryagin's maximum principle to derive a set of necessary conditions in the form of a boundary value problem, also known as the characteristic equations, for each grid point. Using this approach, the method is spatially causality free, which enjoys the advantage of perfect parallelism on a sparse grid. Compared with dense grids, a sparse grid has a significantly reduced size which is feasible for systems with relatively high dimensions, such as the $6$-D system shown in the examples. Once the solution obtained at each grid point, high-order accurate polynomial interpolation is used to approximate the feedback control at arbitrary points. We prove an upper bound for the approximation error and approximate it numerically. This sparse grid characteristics method is demonstrated with two examples of rigid body attitude control using momentum wheels

Fault detection in nonlinear systems: an observer-based approach

An un-permitted deviation of at least one characteristic property or parameter of a system from standard condition is referred as a fault. Faults result in reduced efficiency of the system, reduced quality of the product, and sometimes complete breakdown of the process. This not only causes economic losses but may also result in fatalities. An early detection of faults can assist to avert these losses. Therefore, fault detection and process monitoring is becoming an essential part of modern control systems. Fault detection in linear dynamical systems has been extensively studied and well established techniques exist in the literature. However, fault detection for nonlinear dynamical systems is yet an active field of research. This work is motivated by the fact that most of real systems are nonlinear in nature and there is a need to develop fault detection techniques for nonlinear systems. Observer-based methods for fault detection have proven to be among the most capable approaches, therefore, this research is focused towards these methods. The first step in observer-based fault detection is to generate a symptom signal, called the residual signal, which carries the information of faults. This is done by comparing the measurements from the process to their estimates generated by an observer (filter). It is desired that the residual signal is sensitive to faults and robust against disturbances. This research presents new methods for designing observer (filter) to generate residual signal which is sensitive to faults and robust against disturbances. Three types of filters are proposed in this dissertation; these include a fault sensitive filter, disturbance attenuating filter, and a filter to achieve simultaneous attenuation of disturbances and amplification of faults. Despite the disturbance attenuation property of the proposed filters, the residual signal is not completely decoupled from the effect of disturbances and uncertainties. Therefore, a threshold is needed to care for the effect of disturbances and uncertainties. Selection of threshold plays an important role in the performance of the fault detection system. If it is selected too high, some faults will not be detected. Conversely, if it is selected too low, disturbances and uncertainties will result in false alarms. This research presents a new method to determine the threshold to avoid false-alarms and to minimize missed-detections. A threshold generator is proposed which is itself a dynamic system and produces a variable threshold. This threshold changes with the effects of uncertainties and disturbances and fits more tightly to the fault-free residual signal and, hence, the performance of fault detection system is improved. In addition to the residual generation stage, the efficiency of a fault detection system can also be optimized by post-filtering. A further contribution of this research is in proposing a post-filter which operates on the residual signal to generate a modified residual signal. This modified residual signal is simultaneously sensitive to faults and robust against disturbances. Together with this post-filter, a strategy is adopted to select a threshold which maximizes the fault detectability and minimizes the number of false-alarms

Integrated Optimal and Robust Control of Spacecraft in Proximity Operations

With the rapid growth of space activities and advancement of aerospace science and technology, many autonomous space missions have been proliferating in recent decades. Control of spacecraft in proximity operations is of great importance to accomplish these missions. The research in this dissertation aims to provide a precise, efficient, optimal, and robust controller to ensure successful spacecraft proximity operations. This is a challenging control task since the problem involves highly nonlinear dynamics including translational motion, rotational motion, and flexible structure deformation and vibration. In addition, uncertainties in the system modeling parameters and disturbances make the precise control more difficult. Four control design approaches are integrated to solve this challenging problem. The first approach is to consider the spacecraft rigid body translational and rotational dynamics together with the flexible motion in one unified optimal control framework so that the overall system performance and constraints can be addressed in one optimization process. The second approach is to formulate the robust control objectives into the optimal control cost function and prove the equivalency between the robust stabilization problem and the transformed optimal control problem. The third approach is to employ the è-D technique, a novel optimal control method that is based on a perturbation solution to the Hamilton-Jacobi-Bellman equation, to solve the nonlinear optimal control problem obtained from the indirect robust control formulation. The resultant optimal control law can be obtained in closedorm, and thus facilitates the onboard implementation. The integration of these three approaches is called the integrated indirect robust control scheme. The fourth approach is to use the inverse optimal adaptive control method combined with the indirect robust control scheme to alleviate the conservativeness of the indirect robust control scheme by using online parameter estimation such that adaptive, robust, and optimal properties can all be achieved. To show the effectiveness of the proposed control approaches, six degree-offreedom spacecraft proximity operation simulation is conducted and demonstrates satisfying performance under various uncertainties and disturbances

Optimization-Based Control Methodologies with Applications to Autonomous Vehicle

This thesis includes two main parts. In the first part, the main contribution is to develop nonsingular rigid-body attitude control laws using a convex formulation, and implement them in an experimental set up. The attitude recovery problem is first parameterized in terms of quaternions, and then two polynomial controllers using an SoS Lyapunov function and an SoS density function are developed. A quaternion-based polynomial controller using backstepping is also designed to make the closed-loop system asymptotically stable. Moreover, the proposed quaternion-based controllers are implemented in a Quanser helicopter, and compared to the polynomial controllers and a PID controller experimentally. The main contribution of the second part of this thesis is to analytically solve the Hamilton-Jacobi-Bellman equation for a class of third order nonlinear optimal control problems for which the dynamics are affine and the cost is quadratic in the input. One special advantage of this work is that the solution is directly obtained for the control input without the computation of a value function first. The value function can however also be obtained based on the control input. Furthermore, a Lyapunov function can be constructed for a subclass of optimal control problems, yielding a proof certificate for stability. Using the proposed methodology, experimental results of a path following problem implemented in a Wheeled Mobile Robot (WMR) are then presented to verify the effectiveness of the proposed methodology

Adaptive, Neural and Robust Control of Wing-Rock and Aeroelastic System

Modern aircraft exhibit wing-rock phenomenon and aeroelastic instability. Wingrock (roll single degree of freedom motion) and aeroelastic systems\u27 (two degrees of freedom) behavior are described by complex nonlinear differential equations. The nonlinearities in the dynamics of these systems give rise to limit cycle oscillations beyond critical speed of aircraft. The onset of wing-rock and aeroelastic instability limits the performance of aircraft and can even lead to catastrophic consequences. Therefore, control of wing-rock motion and stabilization of aeroelastic systems are important. In the past, several studies have been made and experimental and analytical results have been obtained to explain the wing-rock and aeroelastic phenomena in wind-tunnel tests, and also control systems have been derived. Motivation for this research is the importance of flying aircraft in a large flight envelope in which complex uncertain aerodynamic nonlinearities appear, causing instabilities and flutter in the aircraft wings. For the control of wing-rock motion and the stabilization of aeroelastic instabilities, new control systems are designed. Because modeling of nonlinear dynamics of wing-rock motion and aeroelastic systems are imprecise, the control algorithms must be insensitive to model uncertainties. Apparently control theory for deterministic systems is not applicable to uncertain systems. For the stabilization of wing-rock, two non-certainity equivalent adaptive (NCEA) laws are designed. The first control system includes a finite form realization of a speed-gradient adaptation law, and the second controller is based on the Immersion and Invariance (I&I) theory. For the nonlinear multi-input multi-output (MIMO) aeroelastic systems, equipped with leading- and trailing-edge control surfaces, four distinct control systems are designed. First, a Chebyshev neural adaptive control law is derived for the suppression of limit cycle oscillations (LCOs) of the prototypical wing. For this derivation SDU decomposition of the high-frequency constant gain matrix is utilized for obtaining a singularity free controller. Then for a multi-input aeroelastic system with state dependent input matrix, a higher-order robust sliding mode control law for finite-time stabilization is derived. This is followed by the design of a suboptimal controller based on the state-dependent Riccati equation (SDRE) method. Finally, a suboptimal control law is designed for the control of the aerolelastic system, based dierential game theory. In this approach, the wind gust is treated as an adversary which tries to destabilize system. These control algorithms are simulated using MATLAB and SIMULINK to verify their performance. Results show that the designed controllers are effective in suppressing the limit cycle oscillations

Cooperative Control Reconfiguration in Networked Multi-Agent Systems

Development of a network of autonomous cooperating vehicles has attracted significant attention during the past few years due to its broad range of applications in areas such as autonomous underwater vehicles for exploring deep sea oceans, satellite formations for space missions, and mobile robots in industrial sites where human involvement is impossible or restricted, to name a few. Motivated by the stringent specifications and requirements for depth, speed, position or attitude of the team and the possibility of having unexpected actuators and sensors faults in missions for these vehicles have led to the proposed research in this thesis on cooperative fault-tolerant control design of autonomous networked vehicles. First, a multi-agent system under a fixed and undirected network topology and subject to actuator faults is studied. A reconfigurable control law is proposed and the so-called distributed Hamilton-Jacobi-Bellman equations for the faulty agents are derived. Then, the reconfigured controller gains are designed by solving these equations subject to the faulty agent dynamics as well as the network structural constraints to ensure that the agents can reach a consensus even in presence of a fault while simultaneously the team performance index is minimized. Next, a multi-agent network subject to simultaneous as well as subsequent actuator faults and under directed fixed topology and subject to bounded energy disturbances is considered. An H∞ performance fault recovery control strategy is proposed that guarantees: the state consensus errors remain bounded, the output of the faulty system behaves exactly the same as that of the healthy system, and the specified H∞ performance bound is guaranteed to be minimized. Towards this end, the reconfigured control law gains are selected first by employing a geometric control approach where a set of controllers guarantees that the output of the faulty agent imitates that of the healthy agent and the consensus achievement objectives are satisfied. Then, the remaining degrees of freedom in the selection of the control law gains are used to minimize the bound on a specified H∞ performance index. Then, control reconfiguration problem in a team subject to directed switching topology networks as well as actuator faults and their severity estimation uncertainties is considered. The consensus achievement of the faulty network is transformed into two stability problems, in which one can be solved offline while the other should be solved online and by utilizing information that each agent has received from the fault detection and identification module. Using quadratic and convex hull Lyapunov functions the control gains are designed and selected such that the team consensus achievement is guaranteed while the upper bound of the team cost performance index is minimized. Finally, a team of non-identical agents subject to actuator faults is considered. A distributed output feedback control strategy is proposed which guarantees that agents outputs’ follow the outputs of the exo-system and the agents states remains stable even when agents are subject to different actuator faults

Safety-aware model-based reinforcement learning using barrier transformation

The ability to learn and execute optimal control policies safely is critical to the realization of complex autonomy, especially where task restarts are not available and/or when the systems are safety-critical. Safety requirements are often expressed in terms of state and/or control constraints. Methods such as barrier transformation and control barrier functions have been successfully used for safe learning in systems under state constraints and/or control constraints, in conjunction with model-based reinforcement learning to learn the optimal control policy. However, existing barrier-based safe learning methods rely on fully known models and full state feedback. In this thesis, two different safe model-based reinforcement learning techniques are developed. One of the techniques utilizes a novel filtered concurrent learning method to realize simultaneous learning and control in the presence of model uncertainties for safety-critical systems, and the other technique utilizes a novel dynamic state estimator to realize simultaneous learning and control for safety-critical systems with a partially observable state. The applicability of the developed techniques is demonstrated through simulations, and to illustrate their effectiveness, comparative simulations are presented wherever alternate methods exist to solve the problem under consideration. The thesis concludes with a discussion about the limitations of the developed techniques. Extensions of the developed techniques are also proposed along with the possible approaches to achieve them