Online Discovery of AUV Control Policies to Overcome Thruster Failures

We investigate methods to improve fault-tolerance of Autonomous Underwater Vehicles (AUVs) to increase their reliability and persistent autonomy. We propose a learning-based approach that is able to discover new control policies to overcome thruster failures as they happen. The proposed approach is a model-based direct policy search that learns on an on-board simulated model of the AUV. The model is adapted to a new condition when a fault is detected and isolated. Since the approach generates an optimal trajectory, the learned fault-tolerant policy is able to navigate the AUV towards a specified target with minimum cost. Finally, the learned policy is executed on the real robot in a closed-loop using the state feedback of the AUV. Unlike most existing methods which rely on the redundancy of thrusters, our approach is also applicable when the AUV becomes under-actuated in the presence of a fault. To validate the feasibility and efficiency of the presented approach, we evaluate it with three learning algorithms and three policy representations with increasing complexity. The proposed method is tested on a real AUV, Girona500

Towards semi-episodic learning for robot damage recovery

International audienceThe recently introduced Intelligent Trial and Error algorithm (IT&E) enables robots to creatively adapt to damage in a matter of minutes by combining an off-line evolutionary algorithm and an on-line learning algorithm based on Bayesian Optimization. We extend the IT&E algorithm to allow for robots to learn to compensate for damages while executing their task(s). This leads to a semi-episodic learning scheme that increases the robot's lifetime autonomy and adaptivity. Preliminary experiments on a toy simulation and a 6-legged robot locomotion task show promising results

Increasing the robustness of autonomous systems to hardware degradation using machine learning

Autonomous systems perform predetermined tasks (missions) with minimum supervision. In most applications, the state of the world changes with time. Sensors are employed to measure part or whole of the world’s state. However, sensors often fail amidst operation; feeding as such decision-making with wrong information about the world. Moreover, hardware degradation may alter dynamic behaviour, and subsequently the capabilities, of an autonomous system; rendering the original mission infeasible. This thesis applies machine learning to yield powerful and robust tools that can facilitate autonomy in modern systems. Incremental kernel regression is used for dynamic modelling. Algorithms of this sort are easy to train and are highly adaptive. Adaptivity allows for model adjustments, whenever the environment of operation changes. Bayesian reasoning provides a rigorous framework for addressing uncertainty. Moreover, using Bayesian Networks, complex inference regarding hardware degradation can be answered. Specifically, adaptive modelling is combined with Bayesian reasoning to yield recursive estimation algorithms that are robust to sensor failures. Two solutions are presented by extending existing recursive estimation algorithms from the robotics literature. The algorithms are deployed on an underwater vehicle and the performance is assessed in real-world experiments. A comparison against standard filters is also provided. Next, the previous algorithms are extended to consider sensor and actuator failures jointly. An algorithm that can detect thruster failures in an Autonomous Underwater Vehicle has been developed. Moreover, the algorithm adapts the dynamic model online to compensate for the detected fault. The performance of this algorithm was also tested in a real-world application. One step further than hardware fault detection, prognostics predict how much longer can a particular hardware component operate normally. Ubiquitous sensors in modern systems render data-driven prognostics a viable solution. However, training is based on skewed datasets; datasets where the samples from the faulty region of operation are much fewer than the ones from the healthy region of operation. This thesis presents a prognostic algorithm that tackles the problem of imbalanced (skewed) datasets

An energy-aware architecture : a practical implementation for autonomous underwater vehicles

Energy awareness, fault tolerance and performance estimation are important aspects for extending the autonomy levels of today’s autonomous vehicles. Those are related to the concepts of survivability and reliability, two important factors that often limit the trust of end users in conducting large-scale deployments of such vehicles. With the aim of preparing the way for persistent autonomous operations this work focuses its efforts on investigating those effects on underwater vehicles capable of long-term missions. A novel energy-aware architecture for autonomous underwater vehicles (AUVs) is presented. This, by monitoring at runtime the vehicle’s energy usage, is capable of detecting and mitigating failures in the propulsion subsystem, one of the most common sources of mission-time problems. Furthermore it estimates the vehicle’s performance when operating in unknown environments and in the presence of external disturbances. These capabilities are a great contribution for reducing the operational uncertainty that most underwater platforms face during their deployment. Using knowledge collected while conducting real missions the proposed architecture allows the optimisation of on-board resource usage. This improves the vehicle’s effectiveness when operating in unknown stochastic scenarios or when facing the problem of resource scarcity. The architecture has been implemented on a real vehicle, Nessie AUV, used for real sea experiments as part of multiple research projects. These gave the opportunity of evaluating the improvements of the proposed system when considering more complex autonomous tasks. Together with Nessie AUV, the commercial platform IVER3 AUV has been involved in the evaluating the feasibility of this approach. Results and operational experience, gathered both in real sea scenarios and in controlled environment experiments, are discussed in detail showing the benefits and the operational constraints of the introduced architecture, alongside suggestions for future research directions

A Hybrid Nonlinear Model Predictive Control and Recurrent Neural Networks for Fault-Tolerant Control of an Autonomous Underwater Vehicle

The operation of Autonomous Unmanned Vehicles (AUVs) that is used for environment protection, risk evaluation and plan determination for emergency, are among the most important and challenging problems. An area that has received much attention for use of AUVs is in underwater applications where much work remains to be done to equip AUVs with systems to steer them accurately and reliably in harsh marine environments. Design of control strategies for AUVs is very challenging as compared to other systems due to their operational environment (ocean). Particularly when hydrodynamic parameters uncertainties are to be integrated into both the controller design as well as AUVs nonlinear dynamics. On the other hand, AUVs like all other mechanical systems are prone to faults. Dealing effectively with faulty situations for mechanical systems is an important consideration since faults can result in abnormal operation or even a failure. Hence, fault tolerant and fault-accommodating methods in the controller design are among active research topics for maintaining the reliability of complex AUV control systems. The objective of this thesis is to develop a nonlinear Model Predictive Control (MPC) that requires solving an online Quadratic Programming (QP) problem by using a Recurrent Neural Network (RNN). Also, an Extended Kalman Filter (EKF) is integrated with the developed scheme to provide the MPC algorithm with the system states estimates as well as a nonlinear prediction. This hybrid control approach utilizes both the mathematical model of the system as well as the adaptive nature of the intelligent technique through neural networks. The reason behind the selection of MPC is to benefit from its main capability in optimization within the current time slots while taking future time slots into consideration. The proposed control method is integrated with EKF which is an appropriate method for state estimation and data reconciliation of nonlinear systems. In order to address the high performance runtime cost of solving the MPC problem (formulated as a quadratic programming problem), an RNN is developed that has a low model complexity as well as good performance in real-time implementation. The proposed method is first developed to control an AUV following a desired trajectory. Since the problem of trajectory tracking and path following of AUVs exhibit nonlinear behavior, the effectiveness of the developed MPC-RNN algorithm is studied in comparison with two other control system methods, namely the linear MPC using Kalman Filter (KF) and the conventional nonlinear MPC using the EKF. In order to guarantee the fault-tolerant features of our proposed control method when faced with severe actuator faults, the developed MPC-RNN scheme is integrated with a dual Extended Kalman Filter that is used for a combined estimation of AUV states and parameters. The actuator faults are defined as the system parameters that are to be estimated online by the dual-EKF. Therefore, the developed Active Fault-Tolerant Control (AFTC) strategy is then applied to an AUV faced with loss of effectiveness (LOE) actuator fault scenarios while following a trajectory. Analysis and discussions regarding the comparison of the proposed AFTC method with Fault-Tolerant Nonlinear Model Predictive Control (FTNMPC) algorithm are presented in this work. The proposed approach to AFTC exploits the advantages of the MPC-RNN algorithm properties as well as accounting explicitly for severe control actuator faults in the nonlinear AUV model with uncertainties that are formulated by the MPC