Reliability Quantification of Deep Reinforcement Learning-based Control

Reliability quantification of deep reinforcement learning (DRL)-based control is a significant challenge for the practical application of artificial intelligence (AI) in safety-critical systems. This study proposes a method for quantifying the reliability of DRL-based control. First, an existing method, random noise distillation, was applied to the reliability evaluation to clarify the issues to be solved. Second, a novel method for reliability quantification was proposed to solve these issues. The reliability is quantified using two neural networks: reference and evaluator. They have the same structure with the same initial parameters. The outputs of the two networks were the same before training. During training, the evaluator network parameters were updated to maximize the difference between the reference and evaluator networks for trained data. Thus, the reliability of the DRL-based control for a state can be evaluated based on the difference in output between the two networks. The proposed method was applied to DQN-based control as an example of a simple task, and its effectiveness was demonstrated. Finally, the proposed method was applied to the problem of switching trained models depending on the state. Con-sequently, the performance of the DRL-based control was improved by switching the trained models according to their reliability.Comment: 18 pages and 17 figure

An overview of the current research on stability of ships and ocean vehicles : the STAB2018 perspective

The paper provides an insight into contemporary research on ship stability and identifies the possible directions for future research by reviewing a selection of papers published in STAB 2015, ISSW 2016 & 2017. These works have been organized in different sections, according to the main thematic areas of research, covering intact and damage stability, regulatory issues including probabilistic approaches, advanced numerical methods for ship motion and stability failure prediction including roll damping, operational issues related to ship stability and environmental modelling. Furthermore, the educational potential of STAB/ISSW is exemplified. This review paper is a joint effort within the SRDC (Stability R&D Committee)

A Reliability Quantification Method for Deep Reinforcement Learning-Based Control

Reliability quantification of deep reinforcement learning (DRL)-based control is a significant challenge for the practical application of artificial intelligence (AI) in safety-critical systems. This study proposes a method for quantifying the reliability of DRL-based control. First, an existing method, random network distillation, was applied to the reliability evaluation to clarify the issues to be solved. Second, a novel method for reliability quantification was proposed to solve these issues. The reliability is quantified using two neural networks: a reference and an evaluator. They have the same structure with the same initial parameters. The outputs of the two networks were the same before training. During training, the evaluator network parameters were updated to maximize the difference between the reference and evaluator networks for trained data. Thus, the reliability of the DRL-based control for a state can be evaluated based on the difference in output between the two networks. The proposed method was applied to DRL-based controls as an example of a simple task, and its effectiveness was demonstrated. Finally, the proposed method was applied to the problem of switching trained models depending on the state. Consequently, the performance of the DRL-based control was improved by switching the trained models according to their reliability

Experimental Validation of Smoothed Particle Hydrodynamics on Generation and Propagation of Water Waves

This paper is aimed to validate smoothed particle hydrodynamics (SPH) on the generation and propagation of water waves. It is a classical problem in marine engineering but a still important problem because there is a strong demand to generate intended nonlinear water waves and to predict complicated interactions between nonlinear water waves and fixed/floating bodies, which is indispensable for further ocean utilization and development. A dedicated experiment was conducted in a large wave basin of Kobe University equipped with a piston-type wavemaker, at three water depths using several amplitudes and periods of piston motion for the validation of SPH mainly on the long-distance propagation of water waves. An SPH-based two-dimensional numerical wave tank (NWT) is used for numerical simulation and is accelerated by a graphics processing units (GPU), assuming future applications to realistic engineering problems. In addition, comparison of large-deformation of shallow water waves, when passing over a fixed box-shape obstacle, is also investigated to discuss the applicability to wave-structure interaction problems. Finally, an SPH-based three-dimensional NWT is also validated by comparing with an experiment and two-dimensional simulation. Through these validation studies, detailed discussion on the accuracy of SPH simulation of water waves is made as well as providing a recommended set of SPH parameters

Comparison of MPS and SPH methods for solving forced motion ship flooding problems

International audienc

NONLINEAR DYNAMICS ON PARAMETRIC ROLL RESONANCE WITH REALISTIC NUMERICAL MODELLING

Abstract This paper describes latest collaborative researches between Japan and the UK on parametric roll resonance of a container ship in following and head seas with realistic modelling of restoring moment as a nonlinear function of wave steepness in experimental, geometrical and analytical approaches. Firstly, captive model experiments were conducted, and demonstrated that the Froude-Krylov prediction could overestimate the amplitude of its variation. Secondly, Poincare mapping technique applied to the numerical model with measured time-varying restoring moment was used for tracing stable steady states, and revealed symmetry-breaking, period-doubling, chaos and capsizing associated with parametric roll resonance. Thirdly, an averaging method was applied to the same numerical model, and confirmed good agreement with the Poincare mapping as well as subcritical bifurcation. Finally, by utilising the present numerical model and methodology, it is shown that a requirement of higher metacentric height does not always improve safety for capsizing associated with parametric roll resonance