Constrained Discrete Phase Control of a Heaving Wave Energy Converter in Irregular Seas Using Reinforcement Learning

Designed for offshore deployment in irregular seas, the point absorber wave energy conversion (WEC) system is promisingly attractive amongst the currently available WEC technologies. The effectiveness of phase control when applied to a heaving point absorber through a hydraulic power take-off (PTO) system is systematically investigated in both regular and irregular waves. For this purpose, two phase control accumulators are utilized in the hydraulic PTO system. Simulations are performed in MATLAB® using the Cummins equation to model the dynamics of the heaving point absorber in the time domain. For a given sea state, the opening instant of the control valves of the phase control accumulators relative to the wave excitation peak and the volumetric displacement of the hydraulic motor are utilized as parameters in a number of simulation runs. In regular waves, the parametric investigation demonstrates that in most cases there is a trade off between maximizing the mean generated power and minimizing the maximum motion amplitude. In fully developed irregular seas, a parametric investigation of different sea states in the North Atlantic demonstrates that by utilizing phase control a significant increase in the power absorption efficiency can be obtained compared to the WEC system operation without phase control. The problem of providing an effective phase-control strategy that maximizes the mean generated power of the WEC system subject to motion amplitude constraints is formulated and solved using a Reinforcement Learning (RL) approach based on the Q-learning algorithm. The RL-based controller chooses actions that determine the opening instant of the phase control accumulator valves and the volumetric displacement of the hydraulic motor. As demonstrated in both regular and irregular waves, the RL-based controller is successful in finding the optimal phase-control strategy. Finally, the prediction of the wave excitation force is performed using a Radial Basis Function (RBF) network ensemble in order to evaluate the impact of the prediction accuracy on the RL-controller\u27s performance. The results show that the computed mean generated power and maximum motion amplitude values using the RBF network ensemble predictions compare very well with the corresponding values computed assuming perfect knowledge of the future wave excitation