Wind Turbine Fault-Tolerant Control via Incremental Model-Based Reinforcement Learning

A reinforcement learning (RL) based fault-tolerant control strategy is developed in this paper for wind turbine torque & pitch control under actuator & sensor faults subject to unknown system models. An incremental model-based heuristic dynamic programming (IHDP) approach, along with a critic-actor structure, is designed to enable fault-tolerance capability and achieve optimal control. Particularly, an incremental model is embedded in the critic-actor structure to quickly learn the potential system changes, such as faults, in real-time. Different from the current IHDP methods that need the intensive evaluation of the state and input matrices, only the input matrix of the incremental model is dynamically evaluated and updated by an online recursive least square estimation procedure in our proposed method. Such a design significantly enhances the online model evaluation efficiency and control performance, especially under faulty conditions. In addition, a value function and a target critic network are incorporated into the main critic-actor structure to improve our method’s learning effectiveness. Case studies for wind turbines under various working conditions are conducted based on the fatigue, aerodynamics, structures, and turbulence (FAST) simulator to demonstrate the proposed method’s solid fault-tolerance capability and adaptability. Note to Practitioners —This work achieves high-performance wind turbine control under unknown actuator & sensor faults. Such a task is still an open problem due to the complexity of turbine dynamics and potential uncertainties in practical situations. A novel data-driven and model-free control strategy based on reinforcement learning is proposed to handle these issues. The designed method can quickly capture the potential changes in the system and adjust its control policy in real-time, rendering strong adaptability and fault-tolerant abilities. It provides data-driven innovations for complex operational tasks of wind turbines and demonstrates the feasibility of applying reinforcement learning to handle fault-tolerant control problems. The proposed method has a generic structure and has the potential to be implemented in other renewable energy systems

The integration of digital art into online museum exhibitions design

This article explores the impact of digital culture on museum exhibitions design, particularly in the realm of digital art. The article showcases the Museum of Modern Art (MoMA) in New York City and Tate Modern (TATE) in London as examples of institutions adapting to the digital era by offering online exhibitions and interactive experiences. The article discusses the benefits of digital art and how it enhances the exhibition experience, providing visitors with a new perspective and allowing artists to approach their work from a different angle. The online museum exhibition is described as interactive, open, and dynamic, bringing new opportunities for museums to engage with their public

Accurate Reconstruction of Molecular Phylogenies for Proteins Using Codon and Amino Acid Unified Sequence Alignments (CAUSA)

Based on molecular clock hypothesis, and neutral theory of molecular evolution, molecular phylogenies have been widely used for inferring evolutionary history of organisms and individual genes. Traditionally, alignments and phylogeny trees of proteins and their coding DNA sequences are constructed separately, thus often different conclusions were drawn. Here we present a new strategy for sequence alignment and phylogenetic tree reconstruction, codon and amino acid unified sequence alignment (CAUSA), which aligns DNA and protein sequences and draw phylogenetic trees in a unified manner. We demonstrated that CAUSA improves both the accuracy of multiple sequence alignments and phylogenetic trees by solving a variety of molecular evolutionary problems in virus, bacteria and mammals. Our results support the hypothesis that the molecular clock for proteins has two pointers existing separately in DNA and protein sequences. It is more accurate to read the molecular clock by combination (additive) of these two pointers, since the ticking rates of them are sometimes consistent, sometimes different. CAUSA software were released as Open Source under GNU/GPL license, and are downloadable free of charge from the website www.dnapluspro.com

Data-driven torque and pitch control of wind turbines via reinforcement learning

This paper addresses the torque and pitch control problems of wind turbines. The main contribution of this work is the development of an innovative reinforcement learning (RL)-based control method targeting wind turbine applications. Our RL-based control framework synergistically combines the advantages of deep neural networks (DNNs) and model predictive control (MPC) technologies. The proposed control strategy is data-driven, adapting to real-time changes in system dynamics and enhancing control performance and robustness. Additionally, the incorporation of an MPC structure within our design improves learning efficiency and reduces the high computational complexity typically found in deep RL algorithms. Specifically, a DNN is designed to approximate the wind turbine dynamics based on a continuously updated dataset composed of state and action measurements taken at specified sampling intervals. The real-time control policy is generated by integrating the online trained DNN into an MPC architecture. The proposed method iteratively updates the DNN and control policy in real-time to optimize performance. As a primary result of this work, the proposed method demonstrates superior robustness and control performance compared to commonly-employed MPC and other baseline wind turbine controllers in the presence of uncertainties and unexpected actuator faults. This effectiveness is showcased through simulations with a high-fidelity wind turbine simulator

Power regulation and load mitigation of floating wind turbines via reinforcement learning

Floating offshore wind turbines (FOWTs) are often subjected to heavy structural loads due to challenging operating conditions, which can negatively impact power generation and lead to structural fatigue. This paper proposes a novel reinforcement learning (RL)-based control scheme to address this issue. It combines individual pitch control (IPC) and collective pitch control (CPC) to balance two key objectives: load reduction and power regulation. Specifically, a novel incremental model-based dual heuristic programming (IDHP) strategy is developed as the IPC solution to reduce structural loads. It integrates the online-learned FOWT dynamics into the dual heuristic programming process, making the entire control scheme data-driven and free from dependence on analytical models. Furthermore, the proposed method differs from existing IDHP methods in that only partial system dynamics need to be learned, resulting in a simplified design structure and improved training efficiency. Tests using a high-fidelity FOWT simulator demonstrate the effectiveness of the proposed method

Learning-based nonlinear model predictive control with accurate uncertainty compensation

A learning-based nonlinear model predictive control (LBNMPC) method is proposed in this paper for general nonlinear systems under system uncertainties and subject to state and input constraints. The proposed LBNMPC strategy decouples the robustness and performance requirements by employing an additional learned model and introducing it into the MPC framework along with the nominal model. The nominal model helps to ensure the closed-loop system’s safety and stability, and the learned model aims to improve the tracking behaviors. As a core of the learned model construction, an online parameter estimator is designed to deal with system uncertainties. This estimation process effectively evaluates both the current and historical effects of uncertainties, leading to superior estimating performance compared with conventional methods. By constructing an invariant terminal constraint set, we prove that the LBNMPC is recursively feasible and robustly asymptotically stable. Numerical verifications for a two-link manipulator are conducted to validate the effectiveness and robustness of the proposed control scheme

Wind farm control technologies : from classical control to reinforcement learning

Wind power plays a vital role in the global effort towards net zero. The recent figure shows that 93GW new wind capacity was installed worldwide in 2020, leading to a 53% year-on-year increase. Control system is the core in wind farm operations and has essential influences on the farm’s power capture efficiency, economic profitability, and operation & maintenance cost. However, wind farms’ inherent system complexities and the aerodynamic interactions among wind turbines bring significant barriers to control systems design. The wind industry has recognized that new technologies are needed to handle wind farm control tasks, especially for large-scale offshore wind farms. This paper provides a comprehensive review of the development and most recent advances of wind farm control technologies. This covers the introduction of fundamental aspects in wind farm control in terms of system modelling, main challenges, and control objectives. Existing wind farm control methods for different purposes, including layout optimization, power generation maximization, fatigue loads minimization, and power reference tracking, are investigated. Moreover, a detailed discussion regarding the differences and connections among model-based, model-free and data-driven wind farm approaches is presented. In addition, highlights are made on the state-of-the-art wind farm control technologies based on reinforcement learning - a booming machine learning technique that has drawn worldwide attention. Future challenges and research avenues in wind farm control are also analysed

Wind farm power generation control via double-network-based deep reinforcement learning

A model-free deep reinforcement learning (DRL) method is proposed in this paper to maximize the total power generation of wind farms through the combination of induction control and yaw control. Specifically, a novel double-network-based DRL approach is designed to generate control policies for thrust coefficients and yaw angles simultaneously and separately. Two sets of critic-actor networks are constructed to this end. They are linked by a central power-related reward, providing a coordinated control structure while inheriting the critic-actor mechanism's advantages. Compared with conventional DRL methods, the proposed double-network-based DRL strategy can adapt to the distinctive and incompatible features of different control inputs, guaranteeing a reliable training process and ensuring superior performance. Also, the prioritized experience replay strategy is utilized to improve the training efficiency of deep neural networks. Simulation tests based on a dynamic wind farm simulator show that the proposed method can significantly increase the power generation for wind farms with different layouts

Efficacy and safety of a combination of miglitol, metformin and insulin aspart in the treatment of type 2 diabetes

Purpose: To study the clinical effect of combining insulin aspart with different drugs in the treatment oftype 2 diabetes mellitus (T2DM).Methods: Two hundred and thirty-seven T2DM patients admitted to the Endocrinology Department of the Second Affiliated Hospital of Kunming Medical University from March to September 2018 were selected as subjects in this study. Miglitol and metformin were used in combination with insulin aspart in the treatment of T2DM. In addition, data on the effectiveness and safety of different treatment options,such as patient’s weight, waist circumference, blood glucose indicators, indices of heart, liver and kidney functions, and incidence of complications were recorded and compared between the two groups.Results: The use of a combination of miglitol and insulin aspart produced an excellent hypoglycaemic effect, and it significantly reduced the incidence of sensory neuropathy in the eyes and distal limbs (p < 0.05). The use of combination of metformin and insulin aspart effectively protected the heart and kidney, and prevented hypoglycaemia (p < 0.05).Conclusion: These results suggest that treatment with a combination of miglitol and insulin aspart is suitable for patients with T2DM whose blood sugar levels are out of control, while combined treatment with metformin and insulin aspart is more suited for patients who desire to reduce blood sugar and blood lipids through weight loss, and patients with cardiac and renal insufficiency