Memory pattern identification for feedback tracking control in human-machine systems

Objective: The aim of this paper was to identify the characteristics of memory patterns with respect to a visual input, perceived by the human operator during a manual control task, which consisted in following a moving target on a display with a cursor.Background: Manual control tasks involve nondeclarative memory. The memory encodings of different motor skills have been referred to as procedural memories. The procedural memories have a pattern, which this paper sought to identify for the particular case of a onedimensional tracking task. Specifically, data recorded from human subjects controlling dynamical systems with different fractional order were investigated.Method: A Finite Impulse Response (FIR) controller was fitted to the data, and pattern analysis was performed to the fitted parameters. Then, the FIR model was further reduced to a lower order controller; from the simplified model, the stability analysis of the human-machine system in closedloop was conducted.Results: It is shown that the FIR model can be employed to identify and represent patterns in human procedural memories during manual control tasks. The obtained procedural memory pattern presents a time scale of about 650 ms before decay. Furthermore, the fitted controller is stable for systems with fractional order less or equal to 1.Conclusion: For systems of different fractional order, the proposed control scheme – based on a FIR model – can effectively characterize the linear properties of manual control in humans.Application: This research supports a biofidelic approach to human manual control modeling over feedback visual perceptions. Relevant applications of this research are: the development of shared-control systems, where a virtual human model assists the human during a control task, and human operator state monitoring.</div

Acoustic-based UAV detection using late fusion of deep neural networks

Multirotor UAVs have become ubiquitous in commercial and public use. As they become more affordable and more available, the associated security risks further increase, especially in relation to airspace breaches and the danger of drone-to-aircraft collisions. Thus, robust systems must be set in place to detect and deal with hostile drones. This paper investigates the use of deep learning methods to detect UAVs using acoustic signals. Deep neural network models are trained with mel-spectrograms as inputs. In this case, Convolutional Neural Networks (CNNs) are shown to be the better performing network, compared with Recurrent Neural Networks (RNNs) and Convolutional Recurrent Neural Networks (CRNNs). Furthermore, late fusion methods have been evaluated using an ensemble of deep neural networks, where the weighted soft voting mechanism has achieved the highest average accuracy of 94.7%, which has outperformed the solo models. In future work, the developed late fusion technique could be utilized with radar and visual methods to further improve the UAV detection performance

Acoustic-based engine fault diagnosis using WPT, PCA and Bayesian optimization

Engine fault diagnosis aims to assist engineers in undertaking vehicle maintenance in an efficient manner. This paper presents an automatic model and hyperparameter selection scheme for engine combustion fault classification, using acoustic signals captured from cylinder heads of the engine. Wavelet Packet Transform (WPT) is utilized for time–frequency analysis, and statistical features are extracted from both high- and low-level WPT coefficients. Then, the extracted features are used to compare three models: (i) standard classification model; (ii) Bayesian optimization for automatic model and hyperparameters selection; and (iii) Principle Component Analysis (PCA) for feature space dimensionality reduction combined with Bayesian optimization. The latter two models both demonstrated improved accuracy and the other performance metrics compared to the standard model. Moreover, with similar accuracy level, PCA with Bayesian optimized model achieved around 20% less total evaluation time and 8–19% less testing time, compared to the second model, for all fault conditions, which thus shows a promising solution for further development in real-time engine fault diagnosis

Machine hearing for industrial fault diagnosis

This paper proposes to apply a machine hearing framework for industrial fault diagnosis, which is inspired by humans' 'listening and diagnostic' capability in identifying machinery faults. The proposed method combines simplified human auditory functionalities with machine learning, aiming to model in a more biologically plausible way. It includes primarily using cochleagram to extract useful time-frequency information in sound signals-representing the cochlea filtering properties in human hearing. Then, a recurrent neural network with long short-term memory layers is constructed to learn and classify the cochleagrams for fault diagnosis-this is to incorporate memory elements in temporal information processing. The proposed method is validated with an experimental study on bearing fault diagnosis using acoustic measurements, while the developed machine hearing scheme could be beneficial to many industrial fault diagnosis applications, e.g., for aeronautical, automotive, marine, railway and manufacturing industry

Human response delay estimation and monitoring using gamma distribution analysis

The aim of this paper is to estimate and monitor the human response delay in manual control tasks. A probability distribution analysis is applied on the response delay, based on experimental data collected from human subjects controlling a dynamic system and responding to visually perceived errors via joystick or steering wheel. The distribution analysis includes firstly a sliding segment method, to extract the delay time for each slice of data. Then, probability distributions of the delay time are fitted by using a bootstrap based goodness-of-fit test. For both manual-control cases, with a joystick and a steering wheel respectively, the experimental data can be explained reasonably by a Gamma distribution. Consequently, the Gamma distribution parameters for different human subjects are compared. Based on these findings, an online monitoring method of the level of attention in the human-operator - or applied workload - is proposed, which could be of interest for relevant shared-control applications

Mitigating threshold effects in human control by stochastic resonance with fractional colored noise

In industrial applications, mechanical and physiological thresholds may limit the capability of human manipulating machine via control devices, such as joysticks and steering wheels. These thresholds can result in loss of information in the control signals that are kept below the threshold of detection of the device or the human operator. One approach to mitigate these effects is stochastic resonance, i.e., by injecting additive noise into a signal to raise its energy content over the threshold of detection. Though this noise partially corrupts the signal, it can increase the detectability of the signal by the control device. This article provides, for the first time, research towards using stochastic resonance to improve human performance in control tasks. In particular, it shows that using adaptive colored noise can improve the detectability of the steering control signals recorded from human participants. The approach converts a signal processing task to an optimization problem, where particle swarm optimization is employed to obtain the optimal color (or spectral exponent) of the injected additive noise, generated through an intelligent technique with fractional order filters. The results have shown that the proposed method improves the detectability of subthreshold steering control signals. This method can be widely applicable to other industrial domains, such as energy harvesting and enhancing sensory perception

Enhancing stochastic resonance by adaptive colored noise and particle swarm optimization: an application to steering control

In this paper, an intelligent signal processing approach is applied to enhance the detectability of weak signals - i.e., signals which are partially below a theoretical threshold of detection. Mechanical and physiological thresholds limit the capability of humans when manipulating machines via control devices, such as steering wheels. One approach to tackle the shortcomings of lost subthreshold information is stochastic resonance, which consists in adding noise to a signal, to raise its energy content over the threshold of detection. In particular, this paper shows that using adaptive colored can noise improve the detectability of steering control signals recorded from human participants. The approach converts a signal processing task to a machine learning problem; particle swarm optimization is employed to obtain the optimal color (or spectral exponent) of the injected noise, generated through fractional order filters. The results have shown that the proposed method improves the detectability of subthreshold steering control signals, which can be further applicable to many other domains, such as improving tactile sensation or acoustic perception through noise and energy harvesting from vehicle tires

Gas path fault and degradation modelling in twin-shaft gas turbines

In this study, an assessment of degradation and failure modes in the gas-path components of twin-shaft industrial gas turbines (IGTs) has been carried out through a model-based analysis. Measurements from twin-shaft IGTs operated in the field and denoting reduction in engine performance attributed to compressor fouling conditions, hot-end blade turbine damage, and failure in the variable stator guide vane (VSGV) mechanism of the compressor have been considered for the analysis. The measurements were compared with simulated data from a thermodynamic model constructed in a Simulink environment, which predicts the physical parameters (pressure and temperature) across the different stations of the IGT. The model predicts engine health parameters, e.g., component efficiencies and flow capacities, which are not available in the engine field data. The results show that it is possible to simulate the change in physical parameters across the IGT during degradation and failure in the components by varying component efficiencies and flow capacities during IGT simulation. The results also demonstrate that the model can predict the measured field data attributed to failure in the gas-path components of twin-shaft IGTs. The estimated health parameters during degradation or failure in the gas-path components can assist the development of health-index prognostic methods for operational engine performance prediction

Enhancing stochastic resonance by adaptive colored noise and particle swarm optimization: an application to steering control

In this paper, an intelligent signal processing approach is applied to enhance the detectability of weak signals - i.e., signals which are partially below a theoretical threshold of detection. Mechanical and physiological thresholds limit the capability of humans when manipulating machines via control devices, such as steering wheels. One approach to tackle the shortcomings of lost subthreshold information is stochastic resonance, which consists in adding noise to a signal, to raise its energy content over the threshold of detection. In particular, this paper shows that using adaptive colored can noise improve the detectability of steering control signals recorded from human participants. The approach converts a signal processing task to a machine learning problem; particle swarm optimization is employed to obtain the optimal color (or spectral exponent) of the injected noise, generated through fractional order filters. The results have shown that the proposed method improves the detectability of subthreshold steering control signals, which can be further applicable to many other domains, such as improving tactile sensation or acoustic perception through noise and energy harvesting from vehicle tires

Mitigating threshold effects in human control by stochastic resonance with fractional colored noise

In industrial applications, mechanical and physiological thresholds may limit the capability of human manipulating machine via control devices, such as joysticks and steering wheels. These thresholds can result in loss of information in the control signals that are kept below the threshold of detection of the device or the human operator. One approach to mitigate these effects is stochastic resonance, i.e., by injecting additive noise into a signal to raise its energy content over the threshold of detection. Though this noise partially corrupts the signal, it can increase the detectability of the signal by the control device. This article provides, for the first time, research towards using stochastic resonance to improve human performance in control tasks. In particular, it shows that using adaptive colored noise can improve the detectability of the steering control signals recorded from human participants. The approach converts a signal processing task to an optimization problem, where particle swarm optimization is employed to obtain the optimal color (or spectral exponent) of the injected additive noise, generated through an intelligent technique with fractional order filters. The results have shown that the proposed method improves the detectability of subthreshold steering control signals. This method can be widely applicable to other industrial domains, such as energy harvesting and enhancing sensory perception