Reinforcement learning for condition-based control of gas turbine engines

A condition-based control framework is proposed for gas turbine engines using reinforcement learning and adaptive dynamic programming (RL-ADP). The system behaviour, specifically the fuel efficiency function and constraints, exhibit unknown degradation patterns which vary from engine to engine. Due to these variations, accurate system models to describe the true system states over the life of the engines are difficult to obtain. Consequently, model-based control techniques are unable to fully compensate for the effects of the variations on the system performance. The proposed RL-ADP control framework is based on Q-learning and uses measurements of desired performance quantities as reward signals to learn and adapt the system efficiency maps. This is achieved without knowledge of the system variation or degradation dynamics, thus providing a through life adaptation strategy that delivers improved system performance. In order to overcome the long standing difficulties associated with the application of adaptive techniques in a safety critical setting, a dual-control loop structure is proposed in the implementation of the RL-ADP scheme. The overall control framework maintains guarantees on the main thrust control loop whilst extracting improved performance as the engine degrades by tuning sets of variable geometry components in the RL-ADP control loop. Simulation results on representative engine data sets demonstrate the effectiveness of this approach as compared to an industry standard gain scheduling

Optimized synthesis of cost-effective, controllable oil system architectures for turbofan engines

Turbofan oil systems are used to provide lubrication and cooling in the engine . There is an increasing interest in oil system architectures which utilize electric pumps and/or valves to give optimized control of flows to individual oil chambers, leading to improved thermal management of oil and lubrication efficiency. The challenges here lie in the trade-off between increasing controllability and minimizing the addition of new components, which adds unwanted production and maintenance costs. This paper formulates the oil system architecture design as a constrained, multiobjective optimization problem. An architecture is described using a graph with nodes representing components and edges representing interconnections between components. A fixed set of nodes called the architecture template is provided as an input and the edges are optimized for a multicriteria objective function. A heuristic method for determining similarities between the different oil chamber flow requirements is presented. This is used in the optimization to evaluate the controllability objective based on the structure of the valve architecture. The methodology provides benefits to system designers by selecting cheaper architectures with fewer valves when the need to control oil chambers separately is small. The effect of manipulating the cost/controllability criteria weightings is investigated to show the impact on the resulting architecture