Data analytics for stochastic control and prognostics in cyber-physical systems

In this dissertation, several novel cyber fault diagnosis and prognosis and defense methodologies for cyber-physical systems have been proposed. First, a novel routing scheme for wireless mesh network is proposed. An effective capacity estimation for P2P and E2E path is designed to guarantee the vital transmission safety. This scheme can ensure a high quality of service (QoS) under imperfect network condition, even cyber attacks. Then, the imperfection, uncertainties, and dynamics in the cyberspace are considered both in system model and controller design. A PDF identifier is proposed to capture the time-varying delays and its distribution. With the modification of traditional stochastic optimal control using PDF of delays, the assumption of full knowledge of network imperfection in priori is relaxed. This proposed controller is considered a novel resilience control strategy for cyber fault diagnosis and prognosis. After that, we turn to the development of a general framework for cyber fault diagnosis and prognosis schemes for CPSs wherein the cyberspace performance affect the physical system and vice versa. A novel cyber fault diagnosis scheme is proposed. It is capable of detecting cyber fault by monitoring the probability of delays. Also, the isolation of cyber and physical system fault is achieved with cooperating with the traditional observer based physical system fault detection. Next, a novel cyber fault prognosis scheme, which can detect and estimate cyber fault and its negative effects on system performance ahead of time, is proposed. Moreover, soft and hard cyber faults are isolated depending on whether potential threats on system stability is predicted. Finally, one-class SVM is employed to classify healthy and erroneous delays. Then, another cyber fault prognosis based on OCSVM is proposed --Abstract, page iv

Bayesian Nonparametric Adaptive Control using Gaussian Processes

This technical report is a preprint of an article submitted to a journal.Most current Model Reference Adaptive Control (MRAC) methods rely on parametric adaptive elements, in which the number of parameters of the adaptive element are fixed a priori, often through expert judgment. An example of such an adaptive element are Radial Basis Function Networks (RBFNs), with RBF centers pre-allocated based on the expected operating domain. If the system operates outside of the expected operating domain, this adaptive element can become non-effective in capturing and canceling the uncertainty, thus rendering the adaptive controller only semi-global in nature. This paper investigates a Gaussian Process (GP) based Bayesian MRAC architecture (GP-MRAC), which leverages the power and flexibility of GP Bayesian nonparametric models of uncertainty. GP-MRAC does not require the centers to be preallocated, can inherently handle measurement noise, and enables MRAC to handle a broader set of uncertainties, including those that are defined as distributions over functions. We use stochastic stability arguments to show that GP-MRAC guarantees good closed loop performance with no prior domain knowledge of the uncertainty. Online implementable GP inference methods are compared in numerical simulations against RBFN-MRAC with preallocated centers and are shown to provide better tracking and improved long-term learning.This research was supported in part by ONR MURI Grant N000141110688 and NSF grant ECS #0846750

Research on parallel nonlinear control system of PD and RBF neural network based on U model

The modelling problem of nonlinear control system is studied, and a higher generality nonlinear U model is established. Based on the nonlinear U model, RBF neural network and PD parallel control algorithm are proposed. The difference between the control input value and the output value of the neural network is taken as the learning target by using the online learning ability of the neural network. The gradient descent method is used to adjust the PD output value, and ultimately track the ideal output. The Newton iterative algorithm is used to complete the transformation of the nonlinear model, and the nonlinear characteristic of the plant is reduced without loss of modelling precision, consequently, the control performance of the system is improved. The simulation results show that RBF neural network and PD parallel control system can control the nonlinear system. Moreover, the control system with Newton iteration can improve the control effect and anti-interference performance of the system

Reinforcement Learning Adaptive PID Controller for an Under-actuated Robot Arm

Abstract: An adaptive PID controller is used to control of a two degrees of freedom under actuated manipulator. An actor-critic based reinforcement learning is employed for tuning of parameters of the adaptive PID controller. Reinforcement learning is an unsupervised scheme wherein no reference exists to which convergence of algorithm is anticipated. Thus, it is appropriate for real time applications. Controller structure and learning equations as well as update rules are provided. Simulations are performed in SIMULINK and performance of the controller is compared with NARMA-L2 controller. The results verified good performance of the controller in tracking and disturbance rejection tests

Adaptive Model Learning of Neural Networks with UUB Stability for Robot Dynamic Estimation

Since batch algorithms suffer from lack of proficiency in confronting model mismatches and disturbances, this contribution proposes an adaptive scheme based on continuous Lyapunov function for online robot dynamic identification. This paper suggests stable updating rules to drive neural networks inspiring from model reference adaptive paradigm. Network structure consists of three parallel self-driving neural networks which aim to estimate robot dynamic terms individually. Lyapunov candidate is selected to construct energy surface for a convex optimization framework. Learning rules are driven directly from Lyapunov functions to make the derivative negative. Finally, experimental results on 3-DOF Phantom Omni Haptic device demonstrate efficiency of the proposed method.Comment: 6 pages, 12 figure

Hierarchical modeling of multi-scale dynamical systems using adaptive radial basis function neural networks: application to synthetic jet actuator wing

To obtain a suitable mathematical model of the input-output behavior of highly nonlinear, multi-scale, nonparametric phenomena, we introduce an adaptive radial basis function approximation approach. We use this approach to estimate the discrepancy between traditional model areas and the multiscale physics of systems involving distributed sensing and technology. Radial Basis Function Networks offers the possible approach to nonparametric multi-scale modeling for dynamical systems like the adaptive wing with the Synthetic Jet Actuator (SJA). We use the Regularized Orthogonal Least Square method (Mark, 1996) and the RAN-EKF (Resource Allocating Network-Extended Kalman Filter) as a reference approach. The first part of the algorithm determines the location of centers one by one until the error goal is met and regularization is achieved. The second process includes an algorithm for the adaptation of all the parameters in the Radial Basis Function Network, centers, variances (shapes) and weights. To demonstrate the effectiveness of these algorithms, SJA wind tunnel data are modeled using this approach. Good performance is obtained compared with conventional neural networks like the multi layer neural network and least square algorithm. Following this work, we establish Model Reference Adaptive Control (MRAC) formulations using an off-line Radial Basis Function Networks (RBFN). We introduce the adaptive control law using a RBFN. A theory that combines RBFN and adaptive control is demonstrated through the simple numerical simulation of the SJA wing. It is expected that these studies will provide a basis for achieving an intelligent control structure for future active wing aircraft