Multi-H∞ controls for unknown input-interference nonlinear system with reinforcement learning

This article studies the multi-H∞ controls for the input-interference nonlinear systems via adaptive dynamic programming (ADP) method, which allows for multiple inputs to have the individual selfish component of the strategy to resist weighted interference. In this line, the ADP scheme is used to learn the Nash-optimization solutions of the input-interference nonlinear system such that multiple H∞ performance indices can reach the defined Nash equilibrium. First, the input-interference nonlinear system is given and the Nash equilibrium is defined. An adaptive neural network (NN) observer is introduced to identify the input-interference nonlinear dynamics. Then, the critic NNs are used to learn the multiple H∞ performance indices. A novel adaptive law is designed to update the critic NN weights by minimizing the Hamiltonian-Jacobi-Isaacs (HJI) equation, which can be used to directly calculate the multi-H∞ controls effectively by using input-output data such that the actor structure is avoided. Moreover, the control system stability and updated parameter convergence are proved. Finally, two numerical examples are simulated to verify the proposed ADP scheme for the input-interference nonlinear system

Intelligent control of nonlinear systems with actuator saturation using neural networks

Common actuator nonlinearities such as saturation, deadzone, backlash, and hysteresis are unavoidable in practical industrial control systems, such as computer numerical control (CNC) machines, xy-positioning tables, robot manipulators, overhead crane mechanisms, and more. When the actuator nonlinearities exist in control systems, they may exhibit relatively large steady-state tracking error or even oscillations, cause the closed-loop system instability, and degrade the overall system performance. Proportional-derivative (PD) controller has observed limit cycles if the actuator nonlinearity is not compensated well. The problems are particularly exacerbated when the required accuracy is high, as in micropositioning devices. Due to the non-analytic nature of the actuator nonlinear dynamics and the fact that the exact actuator nonlinear functions, namely operation uncertainty, are unknown, the saturation compensation research is a challenging and important topic with both theoretical and practical significance. Adaptive control can accommodate the system modeling, parametric, and environmental structural uncertainties. With the universal approximating property and learning capability of neural network (NN), it is appealing to develop adaptive NN-based saturation compensation scheme without explicit knowledge of actuator saturation nonlinearity. In this dissertation, intelligent anti-windup saturation compensation schemes in several scenarios of nonlinear systems are investigated. The nonlinear systems studied within this dissertation include the general nonlinear system in Brunovsky canonical form, a second order multi-input multi-output (MIMO) nonlinear system such as a robot manipulator, and an underactuated system-flexible robot system. The abovementioned methods assume the full states information is measurable and completely known. During the NN-based control law development, the imposed actuator saturation is assumed to be unknown and treated as the system input disturbance. The schemes that lead to stability, command following and disturbance rejection is rigorously proved, and verified using the nonlinear system models. On-line NN weights tuning law, the overall closed-loop performance, and the boundedness of the NN weights are rigorously derived and guaranteed based on Lyapunov approach. The NN saturation compensator is inserted into a feedforward path. The simulation conducted indicates that the proposed schemes can effectively compensate for the saturation nonlinearity in the presence of system uncertainty

Development of Robust Control Strategies for Autonomous Underwater Vehicles

The resources of the energy and chemical balance in the ocean sustain mankind in many ways. Therefore, ocean exploration is an essential task that is accomplished by deploying Underwater Vehicles. An Underwater Vehicle with autonomy feature for its navigation and control is called Autonomous Underwater Vehicle (AUV). Among the task handled by an AUV, accurately positioning itself at a desired position with respect to the reference objects is called set-point control. Similarly, tracking of the reference trajectory is also another important task. Battery recharging of AUV, positioning with respect to underwater structure, cable, seabed, tracking of reference trajectory with desired accuracy and speed to avoid collision with the guiding vehicle in the last phase of docking are some significant applications where an AUV needs to perform the above tasks. Parametric uncertainties in AUV dynamics and actuator torque limitation necessitate to design robust control algorithms to achieve motion control objectives in the face of uncertainties. Sliding Mode Controller (SMC), H / μ synthesis, model based PID group controllers are some of the robust controllers which have been applied to AUV. But SMC suffers from less efficient tuning of its switching gains due to model parameters and noisy estimated acceleration states appearing in its control law. In addition, demand of high control effort due to high frequency chattering is another drawback of SMC. Furthermore, real-time implementation of H / μ synthesis controller based on its stability study is restricted due to use of linearly approximated dynamic model of an AUV, which hinders achieving robustness. Moreover, model based PID group controllers suffer from implementation complexities and exhibit poor transient and steady-state performances under parametric uncertainties. On the other hand model free Linear PID (LPID) has inherent problem of narrow convergence region, i.e.it can not ensure convergence of large initial error to zero. Additionally, it suffers from integrator-wind-up and subsequent saturation of actuator during the occurrence of large initial error. But LPID controller has inherent capability to cope up with the uncertainties. In view of addressing the above said problem, this work proposes wind-up free Nonlinear PID with Bounded Integral (BI) and Bounded Derivative (BD) for set-point control and combination of continuous SMC with Nonlinear PID with BI and BD namely SM-N-PID with BI and BD for trajectory tracking. Nonlinear functions are used for all P,I and D controllers (for both of set-point and tracking control) in addition to use of nonlinear tan hyperbolic function in SMC(for tracking only) such that torque demand from the controller can be kept within a limit. A direct Lyapunov analysis is pursued to prove stable motion of AUV. The efficacies of the proposed controllers are compared with other two controllers namely PD and N-PID without BI and BD for set-point control and PD plus Feedforward Compensation (FC) and SM-NPID without BI and BD for tracking control. Multiple AUVs cooperatively performing a mission offers several advantages over a single AUV in a non-cooperative manner; such as reliability and increased work efficiency, etc. Bandwidth limitation in acoustic medium possess challenges in designing cooperative motion control algorithm for multiple AUVs owing to the necessity of communication of sensors and actuator signals among AUVs. In literature, undirected graph based approach is used for control design under communication constraints and thus it is not suitable for large number of AUVs participating in a cooperative motion plan. Formation control is a popular cooperative motion control paradigm. This thesis models the formation as a minimally persistent directed graph and proposes control schemes for maintaining the distance constraints during the course of motion of entire formation. For formation control each AUV uses Sliding Mode Nonlinear PID controller with Bounded Integrator and Bounded Derivative. Direct Lyapunov stability analysis in the framework of input-to-state stability ensures the stable motion of formation while maintaining the desired distance constraints among the AUVs

Virtual Structure Based Formation Tracking of Multiple Wheeled Mobile Robots: An Optimization Perspective

Today, with the increasing development of science and technology, many systems need to be optimized to find the optimal solution of the system. this kind of problem is also called optimization problem. Especially in the formation problem of multi-wheeled mobile robots, the optimization algorithm can help us to find the optimal solution of the formation problem. In this paper, the formation problem of multi-wheeled mobile robots is studied from the point of view of optimization. In order to reduce the complexity of the formation problem, we first put the robots with the same requirements into a group. Then, by using the virtual structure method, the formation problem is reduced to a virtual WMR trajectory tracking problem with placeholders, which describes the expected position of each WMR formation. By using placeholders, you can get the desired track for each WMR. In addition, in order to avoid the collision between multiple WMR in the group, we add an attraction to the trajectory tracking method. Because MWMR in the same team have different attractions, collisions can be easily avoided. Through simulation analysis, it is proved that the optimization model is reasonable and correct. In the last part, the limitations of this model and corresponding suggestions are given

Advances in Reinforcement Learning

Reinforcement Learning (RL) is a very dynamic area in terms of theory and application. This book brings together many different aspects of the current research on several fields associated to RL which has been growing rapidly, producing a wide variety of learning algorithms for different applications. Based on 24 Chapters, it covers a very broad variety of topics in RL and their application in autonomous systems. A set of chapters in this book provide a general overview of RL while other chapters focus mostly on the applications of RL paradigms: Game Theory, Multi-Agent Theory, Robotic, Networking Technologies, Vehicular Navigation, Medicine and Industrial Logistic

Brachiating power line inspection robot: controller design and implementation

The prevalence of electrical transmission networks has led to an increase in productivity and prosperity. In 2014, estimates showed that the global electric power transmission network consisted of 5.5 million circuit kilometres (Ckm) of high-voltage transmission lines with a combined capacity of 17 million mega-volt ampere. The vastness of the global transmission grid presents a significant problem for infrastructure maintenance. The high maintenance costs, coupled with challenging terrain, provide an opportunity for autonomous inspection robots. The Brachiating Power Line Inspection Robot (BPLIR) with wheels [73] is a transmission line inspection robot. The BPLIR is the focus of this research and this dissertation tackles the problem of state estimation, adaptive trajectory generation and robust control for the BPLIR. A kinematics-based Kalman Filter state estimator was designed and implemented to determine the full system state. Instrumentation used for measurement consisted of 2 Inertial Measurement Units (IMUs). The advantages of utilising IMUs is that they are less susceptible to drift, have no moving parts and are not prone to misalignment errors. The use of IMU's in the design meant that absolute angles (link angles measured with respect to earth) could be estimated, enabling the BPLIR to navigate inclined slopes. Quantitative Feedback Control theory was employed to address the issue of parameter uncertainty during operation. The operating environment of the BPLIR requires it to be robust to environmental factors such as wind disturbance and uncertainty in joint friction over time. The resulting robust control system was able to compensate for uncertain system parameters and reject disturbances in simulation. An online trajectory generator (OTG), inspired by Raibert-style reverse-time symmetry[10], fed into the control system to drive the end effector to the power line by employing brachiation. The OTG produced two trajectories; one of which was reverse time symmetrical and; another which minimised the perpendicular distance between the end gripper and the power line. Linear interpolation between the two trajectories ensured a smooth bump-less trajectory for the BPLIR to follow

Control and game-theoretic methods for secure cyber-physical-human systems

This work focuses on systems comprising tightly interconnected physical and digital components. Those, aptly named, cyber-physical systems will be the core of the Fourth Industrial Revolution. Thus, cyber-physical systems will be called upon to interact with humans, either in a cooperative fashion, or as adversaries to malicious human agents that will seek to corrupt their operation. In this work, we will present methods that enable an autonomous system to operate safely among human agents and to gain an advantage in cyber-physical security scenarios by employing tools from control, game and learning theories. Our work revolves around three main axes: unpredictability-based defense, operation among agents with bounded rationality and verification of safety properties for autonomous systems. In taking advantage of the complex nature of cyber-physical systems, our unpredictability-based defense work will focus both on attacks on actuating and sensing components, which will be addressed via a novel switching-based Moving Target Defense framework, and on Denial-of-Service attacks on the underlying network via a zero-sum game exploiting redundant communication channels. Subsequently, we will take a more abstract view of complex system security by exploring the principles of bounded rationality. We will show how attackers of bounded rationality can coordinate in inducing erroneous decisions to a system while they remain stealthy. Methods of cognitive hierarchy will be employed for decision prediction, while closed form solutions of the optimization problem and the conditions of convergence to the Nash equilibrium will be investigated. The principles of bounded rationality will be brought to control systems via the use of policy iteration algorithms, enabling data-driven attack prediction in a more realistic fashion than what can be offered by game equilibrium solutions. The issue of intelligence in security scenarios will be further considered via concepts of learning manipulation through a proposed framework where bounded rationality is understood as a hierarchy in learning, rather than optimizing, capability. This viewpoint will allow us to propose methods of exploiting the learning process of an imperfect opponent in order to affect their cognitive state via the use of tools from optimal control theory. Finally, in the context of safety, we will explore verification and compositionality properties of linear systems that are designed to be added to a cascade network of similar systems. To obfuscate the need for knowledge of the system's dynamics, we will state decentralized conditions that guarantee a specific dissipativity properties for the system, which are shown to be solved by reinforcement learning techniques. Subsequently, we will propose a framework that employs a hierarchical solution of temporal logic specifications and reinforcement learning problems for optimal tracking.Ph.D