System Identification for Nonlinear Control Using Neural Networks

An approach to incorporating artificial neural networks in nonlinear, adaptive control systems is described. The controller contains three principal elements: a nonlinear inverse dynamic control law whose coefficients depend on a comprehensive model of the plant, a neural network that models system dynamics, and a state estimator whose outputs drive the control law and train the neural network. Attention is focused on the system identification task, which combines an extended Kalman filter with generalized spline function approximation. Continual learning is possible during normal operation, without taking the system off line for specialized training. Nonlinear inverse dynamic control requires smooth derivatives as well as function estimates, imposing stringent goals on the approximating technique

Online Natural Gradient as a Kalman Filter

We cast Amari's natural gradient in statistical learning as a specific case of Kalman filtering. Namely, applying an extended Kalman filter to estimate a fixed unknown parameter of a probabilistic model from a series of observations, is rigorously equivalent to estimating this parameter via an online stochastic natural gradient descent on the log-likelihood of the observations. In the i.i.d. case, this relation is a consequence of the "information filter" phrasing of the extended Kalman filter. In the recurrent (state space, non-i.i.d.) case, we prove that the joint Kalman filter over states and parameters is a natural gradient on top of real-time recurrent learning (RTRL), a classical algorithm to train recurrent models. This exact algebraic correspondence provides relevant interpretations for natural gradient hyperparameters such as learning rates or initialization and regularization of the Fisher information matrix.Comment: 3rd version: expanded intr

Aircraft Parameter Estimation using Feedforward Neural Networks With Lyapunov Stability Analysis

Aerodynamic parameter estimation is critical in the aviation sector, especially in design and development programs of defense-military aircraft. In this paper, new results of the application of Artificial Neural Networks (ANN) to the field of aircraft parameter estimation are presented. The performances of Feedforward Neural Network (FFNN) with Backpropagation and FFNN with Backpropagation using Recursive Least Square (RLS) are investigated for aerodynamic parameter estimation. The methods are validated on flight data simulated using MATLAB implementations. The normalized Lyapunov energy functional has been used to derive the convergence conditions for both the ANN-based estimation algorithms. The estimation results are compared on the basis of performance metrics and computation time. The performance of FFNN-RLS has been observed to be approximately 10% better than FFNN-BPN. Simulation results from both algorithms have been found to be highly satisfactory and pave the way for further applications to real flight test data

Recurrent Neural Networks For Accurate RSSI Indoor Localization

This paper proposes recurrent neuron networks (RNNs) for a fingerprinting indoor localization using WiFi. Instead of locating user's position one at a time as in the cases of conventional algorithms, our RNN solution aims at trajectory positioning and takes into account the relation among the received signal strength indicator (RSSI) measurements in a trajectory. Furthermore, a weighted average filter is proposed for both input RSSI data and sequential output locations to enhance the accuracy among the temporal fluctuations of RSSI. The results using different types of RNN including vanilla RNN, long short-term memory (LSTM), gated recurrent unit (GRU) and bidirectional LSTM (BiLSTM) are presented. On-site experiments demonstrate that the proposed structure achieves an average localization error of $0.75$ m with $80\%$ of the errors under $1$ m, which outperforms the conventional KNN algorithms and probabilistic algorithms by approximately $30\%$ under the same test environment.Comment: Received signal strength indicator (RSSI), WiFi indoor localization, recurrent neuron network (RNN), long shortterm memory (LSTM), fingerprint-based localizatio

A Recursive Least Squares Training Algorithm for Multilayer Recurrent Neural Networks

Recurrent neural networks have the potential to perform significantly better than the commonly used feedforward neural networks due to their dynamical nature. However, they have received less attention because training algorithms/architectures have not been well developed. In this study, a recursive least squares algorithm to train recurrent neural networks with an arbitrary number of hidden layers is developed. The training algorithm is developed as an extension of the standard recursive estimation problem. Simulated results obtained for identification of the dynamics of a nonlinear dynamical system show promising results