Multi-objective reinforcement learning for AUV thruster failure recovery

This paper investigates learning approaches for discovering fault-tolerant control policies to overcome thruster failures in Autonomous Underwater Vehicles (AUV). The proposed approach is a model-based direct policy search that learns on an on-board simulated model of the vehicle. When a fault is detected and isolated the model of the AUV is reconfigured according to the new condition. To discover a set of optimal solutions a multi-objective reinforcement learning approach is employed which can deal with multiple conflicting objectives. Each optimal solution can be used to generate a trajectory that is able to navigate the AUV towards a specified target while satisfying multiple objectives. The discovered policies are executed on the robot in a closed-loop using AUVs state feedback. Unlike most existing methods which disregard the faulty thruster, our approach can also deal with partially broken thrusters to increase the persistent autonomy of the AUV. In addition, the proposed approach is applicable when the AUV either becomes under-actuated or remains redundant in the presence of a fault. We validate the proposed approach on the model of the Girona500 AUV

Online Discovery of AUV Control Policies to Overcome Thruster Failures

We investigate methods to improve fault-tolerance of Autonomous Underwater Vehicles (AUVs) to increase their reliability and persistent autonomy. We propose a learning-based approach that is able to discover new control policies to overcome thruster failures as they happen. The proposed approach is a model-based direct policy search that learns on an on-board simulated model of the AUV. The model is adapted to a new condition when a fault is detected and isolated. Since the approach generates an optimal trajectory, the learned fault-tolerant policy is able to navigate the AUV towards a specified target with minimum cost. Finally, the learned policy is executed on the real robot in a closed-loop using the state feedback of the AUV. Unlike most existing methods which rely on the redundancy of thrusters, our approach is also applicable when the AUV becomes under-actuated in the presence of a fault. To validate the feasibility and efficiency of the presented approach, we evaluate it with three learning algorithms and three policy representations with increasing complexity. The proposed method is tested on a real AUV, Girona500

An energy-aware architecture : a practical implementation for autonomous underwater vehicles

Energy awareness, fault tolerance and performance estimation are important aspects for extending the autonomy levels of today’s autonomous vehicles. Those are related to the concepts of survivability and reliability, two important factors that often limit the trust of end users in conducting large-scale deployments of such vehicles. With the aim of preparing the way for persistent autonomous operations this work focuses its efforts on investigating those effects on underwater vehicles capable of long-term missions. A novel energy-aware architecture for autonomous underwater vehicles (AUVs) is presented. This, by monitoring at runtime the vehicle’s energy usage, is capable of detecting and mitigating failures in the propulsion subsystem, one of the most common sources of mission-time problems. Furthermore it estimates the vehicle’s performance when operating in unknown environments and in the presence of external disturbances. These capabilities are a great contribution for reducing the operational uncertainty that most underwater platforms face during their deployment. Using knowledge collected while conducting real missions the proposed architecture allows the optimisation of on-board resource usage. This improves the vehicle’s effectiveness when operating in unknown stochastic scenarios or when facing the problem of resource scarcity. The architecture has been implemented on a real vehicle, Nessie AUV, used for real sea experiments as part of multiple research projects. These gave the opportunity of evaluating the improvements of the proposed system when considering more complex autonomous tasks. Together with Nessie AUV, the commercial platform IVER3 AUV has been involved in the evaluating the feasibility of this approach. Results and operational experience, gathered both in real sea scenarios and in controlled environment experiments, are discussed in detail showing the benefits and the operational constraints of the introduced architecture, alongside suggestions for future research directions

Formation Control and Fault Accommodation for a Team of Autonomous Underwater Vehicles

The purpose of this thesis is the development of efficient formation control and fault accommodation algorithms for a team of autonomous underwater vehicles (AUVs). The team of AUVs are capable of performing a wide range of deep water marine applications such as seabed mapping and surveying, oil and gas exploration and extraction, and oil and gas pipeline inspection. However, communication limitations and the presence of undesirable events such as component faults in any of the team members can prevent the whole team to achieve safe, reliable, and efficient performance while executing underwater mission tasks. In this regard, the semi-decentralized control scheme is developed to achieve trajectory tracking and formation keeping while requiring information exchange only among neighboring agents. To this end, model predictive control (MPC) technique and dynamic game theory are utilized to formulate and solve the formation control problem. Moreover, centralized and decentralized control schemes are developed to assess the performance of the proposed semi-decentralized control scheme in the simulation studies. The simulation results verify that the performance of the proposed semi-decentralized scheme is very close to the centralized scheme with lower control effort cost while it does not impose stringent communication requirements as in the centralized scheme. Moreover, the semi-decentralized active fault recovery scheme is developed to maintain a graceful degraded performance and to ensure that the team of autonomous underwater vehicles can satisfy mission objectives when an actuator fault occurs in any of the team members. In this regard, online fault information provided by fault detection and isolation (FDI) modules of each agent and its neighbors are incorporated to redesign the nominal controllers based on the MPC technique and dynamic game theory. Additionally, FDI imperfections such as fault estimation error and time delay are taken into account, and a performance index is derived to show the impact of FDI imperfections on the performance of team members. Moreover, centralized and decentralized active fault recovery schemes are developed to evaluate the performance of the proposed semi-decentralized recovery scheme through comparative simulation studies with various fault scenarios. The comparative simulation studies justify that the proposed semi-decentralized fault recovery scheme meets the design specifications even if the performance of the FDI module is not ideal

Deep Learning-Based Machinery Fault Diagnostics

This book offers a compilation for experts, scholars, and researchers to present the most recent advancements, from theoretical methods to the applications of sophisticated fault diagnosis techniques. The deep learning methods for analyzing and testing complex mechanical systems are of particular interest. Special attention is given to the representation and analysis of system information, operating condition monitoring, the establishment of technical standards, and scientific support of machinery fault diagnosis

Risk analysis and decision making for autonomous underwater vehicles

Risk analysis for autonomous underwater vehicles (AUVs) is essential to enable AUVs to explore extreme and dynamic environments. This research aims to augment existing risk analysis methods for AUVs, and it proposes a suite of methods to quantify mission risks and to support the implementation of safety-based decision making strategies for AUVs in harsh marine environments. This research firstly provides a systematic review of past progress of risk analysis research for AUV operations. The review answers key questions including fundamental concepts and evolving methods in the domain of risk analysis for AUVs, and it highlights future research trends to bridge existing gaps. Based on the state-of-the-art research, a copula-based approach is proposed for predicting the risk of AUV loss in underwater environments. The developed copula Bayesian network (CBN) aims to handle non-linear dependencies among environmental variables and inherent technical failures for AUVs, and therefore achieve accurate risk estimation for vehicle loss given various environmental observations. Furthermore, path planning for AUVs is an effective decision making strategy for mitigating risks and ensuring safer routing. A further study presents an offboard risk-based path planning approach for AUVs, considering a challenging environment with oil spill scenarios incorporated. The proposed global Risk-A* planner combines a Bayesian-based risk model for probabilistic risk reasoning and an A*-based algorithm for path searching. However, global path planning designed for static environments cannot handle the unpredictable situations that may emerge, and real-time replanned solutions are required to account for dynamic environmental observations. Therefore, a hybrid risk-aware decision making strategy is investigated for AUVs to combine static global planning with dynamic local re-planning. A dynamic risk analysis model based on the system theoretic process analysis (STPA) and BN is applied for generating a real-time risk map in target mission areas. The dynamic window algorithm (DWA) serves for local path planning to avoid moving obstacles. The proposed hybrid risk-aware decisionmaking architecture is essential for the real-life implementation of AUVs, leading eventually to a real-time adaptive path planning process onboard the AUV

A Meta-Learning-based Trajectory Tracking Framework for UAVs under Degraded Conditions

Due to changes in model dynamics or unexpected disturbances, an autonomous robotic system may experience unforeseen challenges during real-world operations which may affect its safety and intended behavior: in particular actuator and system failures and external disturbances are among the most common causes of degraded mode of operation. To deal with this problem, in this work, we present a meta-learning-based approach to improve the trajectory tracking performance of an unmanned aerial vehicle (UAV) under actuator faults and disturbances which have not been previously experienced. Our approach leverages meta-learning to train a model that is easily adaptable at runtime to make accurate predictions about the system's future state. A runtime monitoring and validation technique is proposed to decide when the system needs to adapt its model by considering a data pruning procedure for efficient learning. Finally, the reference trajectory is adapted based on future predictions by borrowing feedback control logic to make the system track the original and desired path without needing to access the system's controller. The proposed framework is applied and validated in both simulations and experiments on a faulty UAV navigation case study demonstrating a drastic increase in tracking performance.Comment: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (to appear) 2021 copyright IEE