A deep reinforcement learning based homeostatic system for unmanned position control

Deep Reinforcement Learning (DRL) has been proven to be capable of designing an optimal control theory by minimising the error in dynamic systems. However, in many of the real-world operations, the exact behaviour of the environment is unknown. In such environments, random changes cause the system to reach different states for the same action. Hence, application of DRL for unpredictable environments is difficult as the states of the world cannot be known for non-stationary transition and reward functions. In this paper, a mechanism to encapsulate the randomness of the environment is suggested using a novel bio-inspired homeostatic approach based on a hybrid of Receptor Density Algorithm (an artificial immune system based anomaly detection application) and a Plastic Spiking Neuronal model. DRL is then introduced to run in conjunction with the above hybrid model. The system is tested on a vehicle to autonomously re-position in an unpredictable environment. Our results show that the DRL based process control raised the accuracy of the hybrid model by 32%.N/

On Bayesian Networks for Structural Health and Condition Monitoring

The first step in data-driven approaches to Structural Health Monitoring (SHM) is that of damage detection. This is a problem that has been well studied in laboratory conditions. Yet, SHM remains an academic topic, not yet widely implemented in industry. One of the main reasons for this is arguably the difficulty in dealing with Environmental and Operational Variabilities (EOVs), which have a tendency to influence damage-sensitive features in ways similar to damage itself. A large number of the methods developed for SHM applications make use of linear Gaussian models for various tasks including dimensionality reduction, density estimation and system identification. A wide range of linear Gaussian models can be formulated as special cases of a general class of probabilistic graphical models, or Bayesian networks. The work presented here discusses how Bayesian networks can be used systematically to approach different types of damage detection problems, through their likelihood function. A likelihood evaluates the probability that an observation belongs to a particular model. If this model correctly captures the undamaged state of the system, then a likelihood can be used as a novelty index, which can point to the presence of damage. Likelihood functions can be systematically exploited for damage detection purposes across the vast range of linear Gaussian models. One of the key benefits of this fact is that simple models can easily be extended to mixtures of linear Gaussian models. It is shown how this approach can be effective in dealing with operational and environmental variabilities. This thesis thus provides a point of view on performing novelty detection under this wide class of models systematically with their likelihood functions. Models that are typically used for other purposes can become powerful novelty detectors in this view. The relationship between Principal Component Analysis (PCA) and Kalman filters is a good example of this. Under the graphical model perspective these two models are a simple variation of each other, where they model data with and without time dependence. Provided these models are trained with representative data from a non-damaged system, their likelihood function presents a useful novelty index. Their limitation to modelling linear Gaussian data can be overcome through the mixture modelling interpretation. Through graphical models, this is a straightforward extension, but one that retains a probabilistic interpretation. The impact of this interpretation is that environmental and operational variability, as well as potential nonlinearity, in SHM features can be captured by these models. Even though the interpretation changes depending on the model, the likelihood function can consistently be used as a damage indicator, throughout models like Gaussian mixtures, PCA, Factor Analysis, Autoregressive models, Kalman filters and switching Kalman filters. The work here focuses around these models. There are various ways in which these models can be used, but here the focus is narrowed to exploring them as novelty detectors, and showing their application in different contexts. The context in this case refers to different types of SHM data and features, as this could be either vibration, acoustics, ultrasound, performance metrics, etc. %The thesis divides into three main sections. The first presents an overview and scope, with introductions to SHM data, machine learning and the use of likelihood functions for novelty detection. This thesis provides a discussion on the theoretical background for probabilistic graphical models, or Bayesian networks. Separate chapters are dedicated to the discussion of Bayesian networks to model static and dynamic data (with and without temporal dependencies, respectively). Furthermore, three different application examples are presented to demonstrate the use of likelihood function inference for damage detection. These systems are a simulated mass-spring-damper system, with varying stiffness in its non-damaged condition, and with a cubic spring nonlinearity. This system presents a challenge from the point of view of the characterisation of the changing environment in terms of global stiffness and excitation energy. It is shown how mixtures of PCA models can be used to tackle this problem if frequency domain features are used, and mixtures of linear dynamical systems (Kalman filters) can be used to successfully characterise the baseline undamaged system and to identify the presence of damage directly from time domain measurements. Another case study involves the detection of damage on the Z-24 bridge. This is a well-studied problem in SHM research, and it is of interest due to the nonlinear stiffness effect due to temperature changes. The features used here are the first four natural frequencies of the bridge. It is demonstrated how a Gaussian mixture model can characterise the undamaged condition, and its likelihood is able to accurately predict the presence of damage. The third case study involves the prediction of various stages of damage on a wind turbine bearing. This is an experimental laboratory investigation - and the problem is also tackled with a Gaussian mixture model. This problem is of interest because the lowest damage level seeded in the bearing was subsurface yield. This is of great relevance to the wind turbine community, as detecting this level of damage is currently not feasible. Features from Acoustic Emission (AE) measurements were used to train a Gaussian mixture model. It is shown that the likelihood function of this model can correctly predict the presence of damage

State Estimation Fusion for Linear Microgrids over an Unreliable Network

Microgrids should be continuously monitored in order to maintain suitable voltages over time. Microgrids are mainly monitored remotely, and their measurement data transmitted through lossy communication networks are vulnerable to cyberattacks and packet loss. The current study leverages the idea of data fusion to address this problem. Hence, this paper investigates the effects of estimation fusion using various machine-learning (ML) regression methods as data fusion methods by aggregating the distributed Kalman filter (KF)-based state estimates of a linear smart microgrid in order to achieve more accurate and reliable state estimates. This unreliability in measurements is because they are received through a lossy communication network that incorporates packet loss and cyberattacks. In addition to ML regression methods, multi-layer perceptron (MLP) and dependent ordered weighted averaging (DOWA) operators are also employed for further comparisons. The results of simulation on the IEEE 4-bus model validate the effectiveness of the employed ML regression methods through the RMSE, MAE and R-squared indices under the condition of missing and manipulated measurements. In general, the results obtained by the Random Forest regression method were more accurate than those of other methods.This research was partially funded by public research projects of Spanish Ministry of Science and Innovation, references PID2020-118249RB-C22 and PDC2021-121567-C22 - AEI/10.13039/ 501100011033, and by the Madrid Government (Comunidad de Madrid-Spain) under the Multiannual Agreement with UC3M in the line of Excellence of University Professors, reference EPUC3M17

FD-ZKF: A Zonotopic Kalman Filter optimizing fault detection rather than state estimation

Enhancing the sensitivity to faults with respect to disturbances, rather than optimizing the precision of the state estimation using Kalman Filters (KF) is the subject of this paper. The stochastic paradigm (KF) is based on minimizing the trace of the state estimation error covariance. In the context of the bounded-error paradigm using Zonotopic Kalman Filters (ZKF), this is analog to minimize the covariation trace. From this analogy and keeping a similar observer-based structure as in ZKF, a criterion jointly inspired by set-membership approaches and approximate decoupling techniques coming from parity-space residual generation is proposed. Its on-line maximization provides an optimal time-varying observer gain leading to the so-called FD-ZKF filter that allows enhancing the fault detection properties. The characterization of minimum detectable fault magnitude is done based on a sensitivity analysis. The effect of the uncertainty is addressed using a set-membership approach and a zonotopic representation reducing set operations to simple matrix calculations. A case study based on a quadruple-tank system is used both to illustrate and compare the effectiveness of the results obtained from the FD-ZKF approach compared to a pure ZKF approachPostprint (author's final draft

Development of a High-Resolution Wind Forecast System Based on the WRF Model and a Hybrid Kalman-Bayesian Filter

S