NARX-based nonlinear system identification using orthogonal least squares basis hunting

An orthogonal least squares technique for basis hunting (OLS-BH) is proposed to construct sparse radial basis function (RBF) models for NARX-type nonlinear systems. Unlike most of the existing RBF or kernel modelling methods, whichplaces the RBF or kernel centers at the training input data points and use a fixed common variance for all the regressors, the proposed OLS-BH technique tunes the RBF center and diagonal covariance matrix of individual regressor by minimizing the training mean square error. An efficient optimization method isadopted for this basis hunting to select regressors in an orthogonal forward selection procedure. Experimental results obtained using this OLS-BH technique demonstrate that it offers a state-of-the-art method for constructing parsimonious RBF models with excellent generalization performance

Identification of nonlinear time-varying systems using an online sliding-window and common model structure selection (CMSS) approach with applications to EEG

The identification of nonlinear time-varying systems using linear-in-the-parameter models is investigated. A new efficient Common Model Structure Selection (CMSS) algorithm is proposed to select a common model structure. The main idea and key procedure is: First, generate K 1 data sets (the first K data sets are used for training, and theK 1 th one is used for testing) using an online sliding window method; then detect significant model terms to form a common model structure which fits over all the K training data sets using the new proposed CMSS approach. Finally, estimate and refine the time-varying parameters for the identified common-structured model using a Recursive Least Squares (RLS) parameter estimation method. The new method can effectively detect and adaptively track the transient variation of nonstationary signals. Two examples are presented to illustrate the effectiveness of the new approach including an application to an EEG data set

A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction

The Nonlinear autoregressive exogenous (NARX) model, which predicts the current value of a time series based upon its previous values as well as the current and past values of multiple driving (exogenous) series, has been studied for decades. Despite the fact that various NARX models have been developed, few of them can capture the long-term temporal dependencies appropriately and select the relevant driving series to make predictions. In this paper, we propose a dual-stage attention-based recurrent neural network (DA-RNN) to address these two issues. In the first stage, we introduce an input attention mechanism to adaptively extract relevant driving series (a.k.a., input features) at each time step by referring to the previous encoder hidden state. In the second stage, we use a temporal attention mechanism to select relevant encoder hidden states across all time steps. With this dual-stage attention scheme, our model can not only make predictions effectively, but can also be easily interpreted. Thorough empirical studies based upon the SML 2010 dataset and the NASDAQ 100 Stock dataset demonstrate that the DA-RNN can outperform state-of-the-art methods for time series prediction.Comment: International Joint Conference on Artificial Intelligence (IJCAI), 201

Study on identification of nonlinear systems using Quasi-ARX models

制度:新 ; 報告番号:甲3660号 ; 学位の種類:博士(工学) ; 授与年月日:2012/9/15 ; 早大学位記番号:新6026Waseda Universit

Safety-Critical Controller Verification via Sim2Real Gap Quantification

The well-known quote from George Box states that: "All models are wrong, but some are useful." To develop more useful models, we quantify the inaccuracy with which a given model represents a system of interest, so that we may leverage this quantity to facilitate controller synthesis and verification. Specifically, we develop a procedure that identifies a sim2real gap that holds with a minimum probability. Augmenting the nominal model with our identified sim2real gap produces an uncertain model which we prove is an accurate representor of system behavior. We leverage this uncertain model to synthesize and verify a controller in simulation using a probabilistic verification approach. This pipeline produces controllers with an arbitrarily high probability of realizing desired safe behavior on system hardware without requiring hardware testing except for those required for sim2real gap identification. We also showcase our procedure working on two hardware platforms - the Robotarium and a quadruped

Precision Control of a Sensorless Brushless Direct Current Motor System

Sensorless control strategies were first suggested well over a decade ago with the aim of reducing the size, weight and unit cost of electrically actuated servo systems. The resulting algorithms have been successfully applied to the induction and synchronous motor families in applications where control of armature speeds above approximately one hundred revolutions per minute is desired. However, sensorless position control remains problematic. This thesis provides an in depth investigation into sensorless motor control strategies for high precision motion control applications. Specifically, methods of achieving control of position and very low speed thresholds are investigated. The developed grey box identification techniques are shown to perform better than their traditional white or black box counterparts. Further, fuzzy model based sliding mode control is implemented and results demonstrate its improved robustness to certain classes of disturbance. Attempts to reject uncertainty within the developed models using the sliding mode are discussed. Novel controllers, which enhance the performance of the sliding mode are presented. Finally, algorithms that achieve control without a primary feedback sensor are successfully demonstrated. Sensorless position control is achieved with resolutions equivalent to those of existing stepper motor technology. The successful control of armature speeds below sixty revolutions per minute is achieved and problems typically associated with motor starting are circumvented.Research Instruments Ltd

Bidirectional Encoder-Decoder with Dual-Stage Attention for Multivariate Time-Series Prediction

학위논문(석사)--서울대학교 대학원 :공과대학 협동과정 기술경영·경제·정책전공,2019. 8. 조성준.자연어처리에 주로 사용되는 RNN 계열의 모델들은 순차적인 데이터를 다루는 데 적합하기 때문에 시계열 데이터 분석에도 다양하게 활용되고 있다. RNN 계열의 모델들이 가지고 있는 취약점은 기울기 값의 소실이라는 문제이다. 이를 해결하기 위해 배치 정규화 같은 기법들이 사용되고 있지만, 시계열 데이터의 경우는 과거에서부터 이어져 오는 추세를 잃어버릴 수 있다. 그리고 적절한 외생변수들을 선택하는 사항도 고려되어야 한다. 본 연구에서는 위에서 언급한 문제들을 해결하기 위해 듀얼 어텐션 메커니즘을 기반으로 하는 양방향 Encoder-Decoder LSTM 모델을 제안한다. Encoder에서 작동하는 어텐션 메커니즘은 이전 단계의 양방향 LSTM에서 전달받은 hidden state와 cell state를 참조하여 예측에 도움이 되는 외생변수들을 파악한다. Decoder는 시계열적인 특성을 반영한 모델 구조로 되어 있다. 먼저 타겟변수의 과거 값들과 추세를 나타내는 통계치를 입력으로 받아서 양방향 LSTM을 통해 학습하고, 지정한 시간 간격까지 매번 hidden state를 업데이트시킨다. Decoder에서의 어텐션은 업데이트된 hidden state와 Encoder에서 나온 출력값을 연결하여 입력으로 받는다. 따라서 예측에 적합한 외생변수의 시점을 파악하는 것뿐만 아니라 타겟변수의 긴 추세를 반영할 수 있다. 모델 평가에 사용된 데이터는 약 4년 치의 KODEX 200(ETF)과 KODEX 200에 포함된 회사들의 5분 단위 개별 주가 거래 데이터이다.Recurrent neural network has been widely applied for time-series prediction. However, the vanishing gradient is still a problem and only a few of them select the relevant dependent variables appropriately. In this paper, I propose a bidirectional encoder-decoder model using dual-stage attention to address the above problems. In the encoder, the input attention mechanism extracts relevant dependent variables by referring to the hidden state and cell state from the bidirectional LSTM of the previous time step. In the decoder, the attention mechanism is applied to the past values of the independent variable but it works differently with the first stage (encoder). A bidirectional LSTM runs through until the defined time step and the hidden state in the decoder is updated at each time step. The updated hidden state combines with the encoded input are used as input in the decoder. With the proposed method, the decoder can capture the information throughout the encoder. It learns a trend of independent variable efficiently and can make a better prediction in comparison with other encoder-decoder models. For the evaluation, the Korean stock market 5-minute trading data is used.제 1장 서론 1 제 2장 관련 연구 4 2.1 시계열 분석에 대한 선행 연구 4 2.2 Encoder-Decoder 모델에 대한 선행 연구 6 2.3 어텐션 메커니즘에 대한 선행 연구 9 제 3장 제안하는 방법 13 3.1 제안하는 Encoder-Decoder의 구조 13 3.2 Encoder의 내부구조 15 3.3 Decoder의 내부구조 17 제 4장 실험 결과 20 4.1 데이터 설명 20 4.2 데이터 전처리 및 학습 방법 22 4.3 성능 평가 및 비교 25 4.4 어텐션 가중치 분석 28 제 5장 결론 35 참고문헌 37 Abstract 42Maste

Aircraft Jet Engine Health Monitoring Through System Identification Using Ensemble Neural Networks

In this thesis a new approach for jet engine Fault Detection and Isolation (FDI) is proposed using ensemble neural networks. Ensemble methods combine various model predictions to reduce the modeling error and increase the prediction accuracy. By combining individual models, more robust and accurate representations are almost always achievable without the need of ad-hoc fine tunings that are required for single model-based solutions. For the purpose of jet engine health monitoring, the model of the jet engine dynamics is represented using three different stand-alone or individual neural network learning algorithms. Specifically, a dynamic multi-layer perceptron (MLP), a dynamic radial-basis function (RBF) neural network, and a dynamic support vector machine (SVM) are trained to individually model the jet engine dynamics. The accuracy of each stand-alone model in identification of the jet engine dynamics is evaluated. Next, three ensemble-based techniques are employed to represent jet engine dynamics. Namely, two heterogenous ensemble models (an ensemble model is heterogeneous when different learning algorithms (neural networks) are used for training its members) and a homogeneous ensemble model (all the models are generated using the same learning algorithm (neural network)). It is concluded that the ensemble models improve the modeling accuracy when compared to stand-alone solutions. The best selected stand-alone model (i.e the dynamic radial-basis function neural network in this application) and the best selected ensemble model (i.e. a heterogenous ensemble) in term of the jet engine modeling accuracy are selected for performing the FDI study. Engine residual signals are generated using both single model-based and ensemble-based solutions under various engine health conditions. The obtained residuals are evaluated in order to detect engine faults. Our simulation results demonstrate that the fault detection task using residuals that are obtained from the ensemble model results in more accurate performance. The fault isolation task is performed by evaluating variations in residual signals (before and after a fault detection flag) using a neural network classifier. As in the fault detection results, it is observed that the ensemble-based fault isolation task results in a more promising performance