Deep Probabilistic Time Series Forecasting using Augmented Recurrent Input for Dynamic Systems

The demand of probabilistic time series forecasting has been recently raised in various dynamic system scenarios, for example, system identification and prognostic and health management of machines. To this end, we combine the advances in both deep generative models and state space model (SSM) to come up with a novel, data-driven deep probabilistic sequence model. Specially, we follow the popular encoder-decoder generative structure to build the recurrent neural networks (RNN) assisted variational sequence model on an augmented recurrent input space, which could induce rich stochastic sequence dependency. Besides, in order to alleviate the issue of inconsistency between training and predicting as well as improving the mining of dynamic patterns, we (i) propose using a hybrid output as input at next time step, which brings training and predicting into alignment; and (ii) further devise a generalized auto-regressive strategy that encodes all the historical dependencies at current time step. Thereafter, we first investigate the methodological characteristics of the proposed deep probabilistic sequence model on toy cases, and then comprehensively demonstrate the superiority of our model against existing deep probabilistic SSM models through extensive numerical experiments on eight system identification benchmarks from various dynamic systems. Finally, we apply our sequence model to a real-world centrifugal compressor sensor data forecasting problem, and again verify its outstanding performance by quantifying the time series predictive distribution.Comment: 25 pages, 7 figures, 4 tables, preprint under revie

Freeze-drying modeling and monitoring using a new neuro-evolutive technique

This paper is focused on the design of a black-box model for the process of freeze-drying of pharmaceuticals. A new methodology based on a self-adaptive differential evolution scheme is combined with a back-propagation algorithm, as local search method, for the simultaneous structural and parametric optimization of the model represented by a neural network. Using the model of the freeze-drying process, both the temperature and the residual ice content in the product vs. time can be determine off-line, given the values of the operating conditions (the temperature of the heating shelf and the pressure in the drying chamber). This makes possible to understand if the maximum temperature allowed by the product is trespassed and when the sublimation drying is complete, thus providing a valuable tool for recipe design and optimization. Besides, the black box model can be applied to monitor the freeze-drying process: in this case, the measurement of product temperature is used as input variable of the neural network in order to provide in-line estimation of the state of the product (temperature and residual amount of ice). Various examples are presented and discussed, thus pointing out the strength of the too

Online system identification development based on recursive weighted least square neural networks of nonlinear hammerstein and wiener models.

The realistic dynamics mathematical model of a system is very important for analyzing a system. The mathematical system model can be derived by applying physical, thermodynamic, and chemistry laws. But this method has some drawbacks, among which is difficult for complex systems, sometimes is untraceable for nonlinear behavior that almost all systems have in the real world, and requires much knowledge. Another method is system identification which is also called experimental modeling. System identification can be made offline, but this method has a disadvantage because the features of a dynamic system may change over time. The parameters may vary as environmental conditions change. It requires big data and consumes a long time. This research introduces a developed method for online system identification based on the Hammerstein and Wiener nonlinear block-oriented structure with the artificial neural networks (NN) advantages and recursive weighted least squares algorithm for optimizing neural network learning in real-time. The proposed method aimed to obtain a maximally informative mathematical model that can describe the actual dynamic behaviors of a system, using the DC motor as a case study. The goodness of fit validation based on the normalized root-mean-square error (NRMSE) and normalized mean square error, and Theil’s inequality coefficient are used to evaluate the performance of models. Based on experimental results, for best Wiener parallel NN model and series-parallel NN model are 93.7% and 89.48%, respectively. Best Hammerstein parallel NN polynomial based model and series-parallel NN polynomial model are 88.75% and 93.9% respectively, for best Hammerstein parallel NN sigmoid based model and series-parallel NN sigmoid based model 78.26% and 95.95% respectively, and for best Hammerstein parallel NN hyperbolic tangent based model and series-parallel NN hyperbolic tangent based model 70.7% and 96.4% respectively. The best model of the developed method outperformed the conventional NARX and NARMAX methods best model by 3.26% in terms of NRMSE goodness of fit

On Applications of New Soft and Evolutionary Computing Techniques to Direct and Inverse Modeling Problems

Adaptive direct modeling or system identification and adaptive inverse modeling or channel equalization find extensive applications in telecommunication, control system, instrumentation, power system engineering and geophysics. If the plants or systems are nonlinear, dynamic, Hammerstein and multiple-input and multiple-output (MIMO) types, the identification task becomes very difficult. Further, the existing conventional methods like the least mean square (LMS) and recursive least square (RLS) algorithms do not provide satisfactory training to develop accurate direct and inverse models. Very often these (LMS and RLS) derivative based algorithms do not lead to optimal solutions in pole-zero and Hammerstein type system identification problem as they have tendency to be trapped by local minima. In many practical situations the output data are contaminated with impulsive type outliers in addition to measurement noise. The density of the outliers may be up to 50%, which means that about 50% of the available data are affected by outliers. The strength of these outliers may be two to five times the maximum amplitude of the signal. Under such adverse conditions the available learning algorithms are not effective in imparting satisfactory training to update the weights of the adaptive models. As a result the resultant direct and inverse models become inaccurate and improper. Hence there are three important issues which need attention to be resolved. These are : (i) Development of accurate direct and inverse models of complex plants using some novel architecture and new learning techniques. (ii) Development of new training rules which alleviates local minima problem during training and thus help in generating improved adaptive models. (iii) Development of robust training strategy which is less sensitive to outliers in training and thus to create identification and equalization models which are robust against outliers. These issues are addressed in this thesis and corresponding contribution are outlined in seven Chapters. In addition, one Chapter on introduction, another on required architectures and algorithms and last Chapter on conclusion and scope for further research work are embodied in the thesis. A new cascaded low complexity functional link artificial neural network (FLANN) structure is proposed and the corresponding learning algorithm is derived and used to identify nonlinear dynamic plants. In terms of identification performance this model is shown to outperform the multilayer perceptron and FLANN model. A novel method of identification of IIR plants is proposed using comprehensive learning particle swarm optimization (CLPSO) algorithm. It is shown that the new approach is more accurate in identification and takes less CPU time compared to those obtained by existing recursive LMS (RLMS), genetic algorithm (GA) and PSO based approaches. The bacterial foraging optimization (BFO) and PSO are used to develop efficient learning algorithms to train models to identify nonlinear dynamic and MIMO plants. The new scheme takes less computational effort, more accurate and consumes less input samples for training. Robust identification and equalization of complex plants have been carried out using outliers in training sets through minimization of robust norms using PSO and BFO based methods. This method yields robust performance both in equalization and identification tasks. Identification of Hammerstein plants has been achieved successfully using PSO, new clonal PSO (CPSO) and immunized PSO (IPSO) algorithms. Finally the thesis proposes a distributed approach to identification of plants by developing two distributed learning algorithms : incremental PSO and diffusion PSO. It is shown that the new approach is more efficient in terms of accuracy and training time compared to centralized PSO based approach. In addition a robust distributed approach for identification is proposed and its performance has been evaluated. In essence the thesis proposed many new and efficient algorithms and structure for identification and equalization task such as distributed algorithms, robust algorithms, algorithms for ploe-zero identification and Hammerstein models. All these new methods are shown to be better in terms of performance, speed of computation or accuracy of results

Neuro-fuzzy modeling of multi-field surface neuroprostheses for hand grasp

154 p.Las neuroprótesis aplican pulsos eléctricos a los nervios periféricos con el objetivo de sustituir funciones motrices/sensoriales perdidas, dando asistencia e influyendo positivamente en la rehabilitación motriz de personas con disfunciones motrices causadas por trastornos neurológicos. La complejidad de la neuroanatomía del antebrazo y la mano, su dimensionalidad, las diversas tareas no-cíclicas, la variabilidad de movimientos entre sujetos y la reducida selectividad de las neuroprótesis superficiales, ha dado lugar al diseño de un número reducido de neuroprótesis orientadas a agarres básicos. La posibilidad de hacer más selectiva la estimulación mediante los electrodos multi-campo, junto con el conocimiento sobre la incomodidad y los movimientos que genera la aplicación de la estimulación eléctrica funcional (FES por sus siglas en inglés) en miembro superior, podrían ser base fundamental para el desarrollo de neuroprótesis de agarre más avanzadas. La presente tesis describe un análisis de incomodidad como resultado de FES en el miembro superior, y propone modelos neuro-difusos para neuroprótesis de agarre tanto para personas sanas como para personas con trastornos neurológicos. El conocimiento generado respecto a la incomodidad puede ser utilizado como guía para desarrollar aplicaciones de FES de miembro superior más cómodas. Del mismo modo, los modelos propuestos en esta tesis pueden ser utilizados para apoyar el diseño y la validación de sistemas de control avanzados en neuroprótesis dirigidas a la función de agarre.Tecnalia; Intelligent Control Research Grou

Who wrote this scientific text?

The IEEE bibliographic database contains a number of proven duplications with indication of the original paper(s) copied. This corpus is used to test a method for the detection of hidden intertextuality (commonly named "plagiarism"). The intertextual distance, combined with the sliding window and with various classification techniques, identifies these duplications with a very low risk of error. These experiments also show that several factors blur the identity of the scientific author, including variable group authorship and the high levels of intertextuality accepted, and sometimes desired, in scientific papers on the same topic

Identification of continuous-time model of hammerstein system using modified multi-verse optimizer

his thesis implements a novel nature-inspired metaheuristic optimization algorithm, namely the modified Multi-Verse Optimizer (mMVO) algorithm, to identify the continuous-time model of Hammerstein system. Multi-Verse Optimizer (MVO) is one of the most recent robust nature-inspired metaheuristic algorithm. It has been successfully implemented and used in various areas such as machine learning applications, engineering applications, network applications, parameter control, and other similar applications to solve optimization problems. However, such metaheuristics had some limitations, such as local optima problem, low searching capability and imbalance between exploration and exploitation. By considering these limitations, two modifications were made upon the conventional MVO in our proposed mMVO algorithm. Our first modification was an average design parameter updating mechanism to solve the local optima issue of the traditional MVO. The essential feature of the average design parameter updating mechanism is that it helps any trapped design parameter jump out from the local optima region and continue a new search track. The second modification is the hybridization of MVO with the Sine Cosine Algorithm (SCA) to improve the low searching capability of the conventional MVO. Hybridization aims to combine MVO and SCA algorithms advantages and minimize the disadvantages, such as low searching capability and imbalance between exploration and exploitation. In particular, the search capacity of the MVO algorithm has been improved using the sine and cosine functions of the Sine Cosine Algorithm (SCA) that will be able to balance the processes of exploration and exploitation. The mMVO based method is then used for identifying the parameters of linear and nonlinear subsystems in the Hammerstein model using the given input and output data. Note that the structure of the linear and nonlinear subsystems is assumed to be known. Moreover, a continuous-time linear subsystem is considered in this study, while there are a few methods that utilize such models. Two numerical examples and one real-world application, such as the Twin Rotor System (TRS) are used to illustrate the efficiency of the mMVO-based method. Various nonlinear subsystems such as quadratic and hyperbolic functions (sine and tangent) are used in those experiments. Numerical and experimental results are analyzed to focus on the convergence curve of the fitness function, the parameter variation index, frequency and time domain response and the Wilcoxon rank test. For the numerical identifications, three different levels of white noise variances were taken. The statistical analysis value (mean) was taken from the parameter deviation index to see how much our proposed algorithm has improved. For Example 1, the improvements are 29%, 33.15% and 36.68%, and for the noise variances, 0.01, 0.25, and 1.0 improvements can be found. For Example 2, the improvements are 39.36%, 39.61% and 66.18%, and for noise variances, the improvements are by 0.01, 0.25 and 1.0, respectively. Finally, for the real TRS application, the improvement is 7%. The numerical and experimental results also showed that both Hammerstein model subsystems are defined effectively using the mMVO-based method, particularly in quadratic output estimation error and a differentiation parameter index. The results further confirmed that the proposed mMVObased method provided better solutions than other optimization techniques, such as PSO, GWO, ALO, MVO and SCA

Artificial Immune Systems: Principle, Algorithms and Applications

The present thesis aims to make an in-depth study of adaptive identification, digital channel equalization, functional link artificial neural network (FLANN) and Artificial Immune Systems (AIS).Two learning algorithms CPSO and IPSO are also developed in this thesis. These new algorithms are employed to train the weights of a low complexity FLANN structure by way of minimizing the squared error cost function of the hybrid model. These new models are applied for adaptive identification of complex nonlinear dynamic plants and equalization of nonlinear digital channel. Investigation has been made for identification of complex Hammerstein models. To validate the performance of these new models simulation study is carried out using benchmark complex plants and nonlinear channels. The results of simulation are compared with those obtained with FLANN-GA, FLANN-PSO and MLP-BP based hybrid approaches. Improved identification and equalization performance of the proposed method have been observed in all cases

Model-predictive control for non-domestic buildings: a critical review and prospects

Model-predictive control (MPC) has recently excited a great deal of interest as a new control paradigm for non-domestic buildings. Since it is based on the notion of optimisation, MPC is, in principle, well-placed to deliver significant energy savings and reduction in carbon emissions compared to existing rule-based control systems. In this paper, we critically review the prospects for buildings MPC and, in particular, the central role of the predictive mathematical model that lies at its heart; our clear emphasis is on practical implementation rather than control-theoretic aspects, and covers the role of occupants as well as the form of the predictive model. The most appropriate structure for such a model is still an open question, which we discuss alongside the development of the initial model, and the process of updating the model during the building’s operational life. The importance of sensor placement is highlighted alongside the possibility of updating the model with occupants’ comfort perception. We conclude that there is an urgent need for research on the automated creation and updating of predictive models if MPC is to become an economically-viable control methodology for non-domestic buildings. Finally, more evidence through operating full scale buildings with MPC is required to demonstrate the viability of this method