Approximate Q-learning approach for Health Aware Control Design

International audienceHealth-aware control (HAC) has emerged as one of the domains where the control is formulated based upon the failure prognostics of system/component or the Remaining Useful Life (RUL) predictions of critical components. Usually, the degradation models are available for components under degradation, but accurate dynamics of the global system is not available. Moreover, mathematical dynamic (transition) models of RUL are rarely available, making the incorporation of RUL information in the control paradigm a difficult task. This paper proposes use of Approximate Reinforcement Learning based algorithms for control formulation. In particular, the approximate Q-learning based algorithm is developed to obtain optimal control law that manages the degradation speed in order to satisfy desired RUL requirements. The proposed method is studied using simulation of a DC motor and degradation of bearing mounted on shaft

Advanced data-driven methods for prognostics and life extension of assets using condition monitoring and sensor data.

A considerable number of engineering assets are fast reaching and operating beyond their orignal design lives. This is the case across various industrial sectors, including oil and gas, wind energy, nuclear energy, etc. Another interesting evolution is the on-going advancement in cyber-physical systems (CPS), where assets within an industrial plant are now interconnected. Consequently, conventional ways of progressing engineering assets beyond their original design lives would need to change. This is the fundamental research gap that this PhD sets out to address. Due to the complexity of CPS assets, modelling their failure cannot be simplistically or analytically achieved as was the case with older assets. This research is a completely novel attempt at using advanced analytics techniques to address the core aspects of asset life extension (LE). The obvious challenge in a system with several pieces of disparate equipment under condition monitoring is how to identify those that need attention and prioritise them. To address this gap, a technique which combined machine learning algorithms and practices from reliability-centered maintenance was developed, along with the use of a novel health condition index called the potential failure interval factor (PFIF). The PFIF was shown to be a good indicator of asset health states, thus enabling the categorisation of equipment as “healthy”, “good ” or “soon-to-fail”. LE strategies were then devoted to the vulnerable group labelled “good – monitor” and “soon-to-fail”. Furthermore, a class of artificial intelligence (AI) algorithms known as Bayesian Neural Networks (BNNs) were used in predicting the remaining useful life (RUL) for the vulnerable assets. The novelty in this was the implicit modelling of the aleatoric and epistemic uncertainties in the RUL prediction, thus yielding interpretable predictions that were useful for LE decision-making. An advanced analytics approach to LE decision-making was then proposed, with the novelty of implementing LE as an on-going series of activities, similar to operation and maintenance (O&M). LE strategies would therefore be implemented at the system, sub-system or component level, meshing seamlessly with O&M, albeit with the clear goal of extending the useful life of the overall asset. The research findings buttress the need for a paradigm shift, from conventional ways of implementing LE in the form of a project at the end of design life, to a more systematic approach based on advanced analytics.Shafiee, Mahmood (Associate)PhD in Energy and Powe