Excessive disturbance rejection control of autonomous underwater vehicle using reinforcement learning

© 2018 Australasian Robotics and Automation Association. All rights reserved. Small Autonomous Underwater Vehicles (AUV) in shallow water might not be stabilized well by feedback or model predictive control. This is because wave and current disturbances may frequently exceed AUV thrust capabilities and disturbance estimation and prediction models available are not sufficiently accurate. In contrast to classical model-free Reinforcement Learning (RL), this paper presents an improved RL for Excessive disturbance rejection Control (REC) that is able to learn and utilize disturbance behaviour, through formulating the disturbed AUV dynamics as a multi-order Markov chain. The unobserved disturbance behaviour is then encoded in the AUV state-action history of fixed length, its embeddings are learned within the policy optimization. The proposed REC is further enhanced by a base controller that is pre-trained on iterative Linear Quadratic Regulator (iLQR) solutions for a reduced AUV dynamic model, resulting in hybrid-REC. Numerical simulations on pose regulation tasks have demonstrated that REC significantly outperforms a canonical controller and classical RL, and that the hybrid-REC leads to more efficient and safer sampling and motion than REC

Plug-and-play and coordinated control for bus-connected AC islanded microgrids

This paper presents a distributed control architecture for voltage and frequency stabilization in AC islanded microgrids. In the primary control layer, each generation unit is equipped with a local controller acting on the corresponding voltage-source converter. Following the plug-and-play design approach previously proposed by some of the authors, whenever the addition/removal of a distributed generation unit is required, feasibility of the operation is automatically checked by designing local controllers through convex optimization. The update of the voltage-control layer, when units plug -in/-out, is therefore automatized and stability of the microgrid is always preserved. Moreover, local control design is based only on the knowledge of parameters of power lines and it does not require to store a global microgrid model. In this work, we focus on bus-connected microgrid topologies and enhance the primary plug-and-play layer with local virtual impedance loops and secondary coordinated controllers ensuring bus voltage tracking and reactive power sharing. In particular, the secondary control architecture is distributed, hence mirroring the modularity of the primary control layer. We validate primary and secondary controllers by performing experiments with balanced, unbalanced and nonlinear loads, on a setup composed of three bus-connected distributed generation units. Most importantly, the stability of the microgrid after the addition/removal of distributed generation units is assessed. Overall, the experimental results show the feasibility of the proposed modular control design framework, where generation units can be added/removed on the fly, thus enabling the deployment of virtual power plants that can be resized over time

LQG control for dynamic positioning of floating caissons based on the Kalman filter

This paper presents the application of an linear quadratic gaussian (LQG) control strategy for concrete caisson deployment for marine structures. Currently these maneuvers are carried out manually with the risk that this entails. Control systems for these operations with classical regulators have begun to be implemented. They try to reduce risks, but they still need to be optimized due to the complexity of the dynamics involved during the sinking process and the contact with the sea bed. A linear approximation of the dynamic model of the caisson is obtained and an LQG control strategy is implemented based on the Kalman filter (KF). The results of the proposed LQG control strategy are compared to the ones given by a classic controller. It is noted that the proposed system is positioned with greater precision and accuracy, as shown in the different simulations and in the Monte Carlo study. Furthermore, the control efforts are less than with classical regulators. For all the reasons cited above, it is concluded that there is a clear improvement in performance with the control system proposed.The Spanish FEDER/Ministry of Science, Innovation and Universities—State Research Agency is greatly acknowledged for partially funding our research through the SAFE Project (Desarrollo de un Sistema Autónomo para el Fondeo de Estructuras para Obras Marítimas), GrantAgreement: RTC-2017-6603-4. The Regional Ministry of Universities, Equality, Culture and Sports of the Gov-ernment of Cantabria has supported this work through the ControlFond project (Control De Ve-hículos Subacuáticos No Tripulados Para Supervisión De Estructuras Para Obras Marítimas Fondeadas). The authors would like to thank FCC Construcción CO as a collaborator in the de-velopment of the SAFE Project, specially Victor Florez Casillas and Nuria Cotallo Aguado (Tech-nical Direction/Hydraulic and Maritime Works) and Alvaro de Toro Mingo (Machinery Direction). R. Guanche also acknowledges financial support from the Ramon y Cajal Program (RYC-2017-23260) of the Spanish Ministry of Science, Innovation and Universities

Adaptive Output-Feedback Model Predictive Control of Hammerstein Systems with Unknown Linear Dynamics

This paper considers model predictive control of Hammerstein systems, where the linear dynamics are a priori unknown and the input nonlinearity is known. Predictive cost adaptive control (PCAC) is applied to this system using recursive least squares for online, closed-loop system identification with optimization over a receding horizon performed by quadratic programming (QP). In order to account for the input nonlinearity, the input matrix is defined to be control dependent, and the optimization is performed iteratively. This technique is applied to output stabilization of a chain of integrators with unknown dynamics under control saturation and deadzone input nonlinearity.Comment: arXiv admin note: text overlap with arXiv:2309.1158

Data-Driven Nonlinear Control Designs for Constrained Systems

Systems with nonlinear dynamics are theoretically constrained to the realm of nonlinear analysis and design, while explicit constraints are expressed as equalities or inequalities of state, input, and output vectors of differential equations. Few control designs exist for systems with such explicit constraints, and no generalized solution has been provided. This dissertation presents general techniques to design stabilizing controls for a specific class of nonlinear systems with constraints on input and output, and verifies that such designs are straightforward to implement in selected applications. Additionally, a closed-form technique for an open-loop problem with unsolvable dynamic equations is developed. Typical optimal control methods cannot be readily applied to nonlinear systems without heavy modification. However, by embedding a novel control framework based on barrier functions and feedback linearization, well-established optimal control techniques become applicable when constraints are imposed by the design in real-time. Applications in power systems and aircraft control often have safety, performance, and hardware restrictions that are combinations of input and output constraints, while cryogenic memory applications have design restrictions and unknown analytic solutions. Most applications fall into a broad class of systems known as passivity-short, in which certain properties are utilized to form a structural framework for system interconnection with existing general stabilizing control techniques. Previous theoretical contributions are extended to include constraints, which can be readily applied to the development of scalable system networks in practical systems, even in the presence of unknown dynamics. In cases such as these, model identification techniques are used to obtain estimated system models which are guaranteed to be at least passivity-short. With numerous analytic tools accessible, a data-driven nonlinear control design framework is developed using model identification resulting in passivity-short systems which handles input and output saturations. Simulations are presented that prove to effectively control and stabilize example practical systems