Implementation of statistical process control framework with machine learning on waveform profiles with no gold standard reference

[[abstract]]Condensation water temperature profiles are collected from a curing process for high-pressure hose products. The shape of those profiles resembles sine waves with diminishing amplitudes. A gold standard wave profile does not exist. Instead some wave profiles with various frequency and amplitudes are deemed normal for the water release operation. To the best of our knowledge, the current practice and research on SPC do not provide a solution for monitoring wave profiles of this kind. We leveraged existing methods, tools, algorithms that can be found in open source or commercial software for quick response to this type of problem. The proposed SPC implementation framework first converts waveform profiles from the time domain to the frequency domain. Then a set of phase I IX control charts is constructed based on a Partition Around Medoids (PAM) clustering method. A Support Vector Machine (SVM) classifier is then used to label a new profile to its associated group for phase II monitoring so that the IX chart associated with a homogeneous group can provide better process monitoring. Overall 146 water temperature profiles were collected in phase I process, while 39 profiles were captured in phase II process. Out of those 39 profiles, 6 of which were recognized as abnormal waveform profiles by quality engineers and our judgements. The proposed framework with machine learning and SPC implementation in the frequency domain works well during phase I control charting with low false alarm rates. The proposed framework also outperforms the other profile analysis methods in phase II control charting in term of high detection rate of abnormal profiles.[[notice]]補正完

Learning Unmanned Aerial Vehicle Control for Autonomous Target Following

While deep reinforcement learning (RL) methods have achieved unprecedented successes in a range of challenging problems, their applicability has been mainly limited to simulation or game domains due to the high sample complexity of the trial-and-error learning process. However, real-world robotic applications often need a data-efficient learning process with safety-critical constraints. In this paper, we consider the challenging problem of learning unmanned aerial vehicle (UAV) control for tracking a moving target. To acquire a strategy that combines perception and control, we represent the policy by a convolutional neural network. We develop a hierarchical approach that combines a model-free policy gradient method with a conventional feedback proportional-integral-derivative (PID) controller to enable stable learning without catastrophic failure. The neural network is trained by a combination of supervised learning from raw images and reinforcement learning from games of self-play. We show that the proposed approach can learn a target following policy in a simulator efficiently and the learned behavior can be successfully transferred to the DJI quadrotor platform for real-world UAV control

End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks

Reinforcement Learning (RL) algorithms have found limited success beyond simulated applications, and one main reason is the absence of safety guarantees during the learning process. Real world systems would realistically fail or break before an optimal controller can be learned. To address this issue, we propose a controller architecture that combines (1) a model-free RL-based controller with (2) model-based controllers utilizing control barrier functions (CBFs) and (3) on-line learning of the unknown system dynamics, in order to ensure safety during learning. Our general framework leverages the success of RL algorithms to learn high-performance controllers, while the CBF-based controllers both guarantee safety and guide the learning process by constraining the set of explorable polices. We utilize Gaussian Processes (GPs) to model the system dynamics and its uncertainties. Our novel controller synthesis algorithm, RL-CBF, guarantees safety with high probability during the learning process, regardless of the RL algorithm used, and demonstrates greater policy exploration efficiency. We test our algorithm on (1) control of an inverted pendulum and (2) autonomous car-following with wireless vehicle-to-vehicle communication, and show that our algorithm attains much greater sample efficiency in learning than other state-of-the-art algorithms and maintains safety during the entire learning process.Comment: Published in AAAI 201

Vehicle Steering control: A model of learning

A hierarchy of strategies were postulated to describe the process of learning steering control. Vehicle motion and steering control data were recorded for twelve novices who drove an instrumented car twice a week during and after a driver training course. Car-driver describing functions were calculated, the probable control structure determined, and the driver-alone transfer function modelled. The data suggested that the largest changes in steering control with learning were in the way the driver used the lateral position cue

A MRAS-based Learning Feed-forward Controller

Inspired by learning feed–forward control structures, this paper considers the adaptation of the parameters of a model–reference based learning feed–forward controller that realizes an inverse model of the process. The actual process response is determined by a setpoint generator. For linear systems it can be proved that the controlled system is asymptotically stable in the sense of Liapunov. Compared with more standard model reference configurations this system has a superior performance. It is fast, robust and relatively insensitive for noisy measurements. Simulations with an arbitrary second–order process and with a model of a typical fourth–ordermechatronics process demonstrate this