A Novel Progressive Multi-label Classifier for Classincremental Data

In this paper, a progressive learning algorithm for multi-label classification to learn new labels while retaining the knowledge of previous labels is designed. New output neurons corresponding to new labels are added and the neural network connections and parameters are automatically restructured as if the label has been introduced from the beginning. This work is the first of the kind in multi-label classifier for class-incremental learning. It is useful for real-world applications such as robotics where streaming data are available and the number of labels is often unknown. Based on the Extreme Learning Machine framework, a novel universal classifier with plug and play capabilities for progressive multi-label classification is developed. Experimental results on various benchmark synthetic and real datasets validate the efficiency and effectiveness of our proposed algorithm.Comment: 5 pages, 3 figures, 4 table

Training of a neural network with using deterministic transforms

Deep neural networks have been a leading research topic within the machine learning field for the past few years. The introduction of graphical processing units (GPUs) and hardware ad- vances made possible the training of deep neural networks. Previously the training procedure was impossible due to the huge amount of training samples required. The new trained introduced architectures have outperformed the classical methods in different classification and regression problems. With the introduction of 5G technology, related to low-latency and online applica- tions, the research on decreasing the computational cost of deep learning architectures while maintaining state-of-art performance has gained huge interest. This thesis focuses on the use of Self Size-estimating Feedforward Network (SSFN), a feed- forward multilayer network. SSFN presents low complexity on the training procedure due to a random matrix instance used in its weights. Its weight matrices are trained using a layer-wise convex optimization approach (a supervised training) combined with a random matrix instance (an unsupervised training). The use of deterministic transforms is explored to replace random matrix instances on the SSFN weight matrices. The use of deterministic transforms automat- ically reduces the computational complexity, as its structure allows to compute them by fast algorithms. Several deterministic transforms such as discrete cosine transform, Hadamard trans- form and wavelet transform, among others, are investigated. To this end, two methods based on features statistical parameters are developed. The proposed methods are implemented on each layer to decide the deterministic transform to use. The effectiveness of the proposed approach is illustrated by SSFN for object classification tasks using several benchmark datasets. The results show a proper performance, similar to the original SSFN, and also consistency across the different datasets. Therefore, the possibility of introducing deterministic transformations in machine learning research is demonstrated

Adaptive Control Using Fully Online Sequential-Extreme Learning Machine and a Case Study on Engine Air-Fuel Ratio Regulation

Most adaptive neural control schemes are based on stochastic gradient-descent backpropagation (SGBP), which suffers from local minima problem. Although the recently proposed regularized online sequential-extreme learning machine (ReOS-ELM) can overcome this issue, it requires a batch of representative initial training data to construct a base model before online learning. The initial data is usually difficult to collect in adaptive control applications. Therefore, this paper proposes an improved version of ReOS-ELM, entitled fully online sequential-extreme learning machine (FOS-ELM). While retaining the advantages of ReOS-ELM, FOS-ELM discards the initial training phase, and hence becomes suitable for adaptive control applications. To demonstrate its effectiveness, FOS-ELM was applied to the adaptive control of engine air-fuel ratio based on a simulated engine model. Besides, controller parameters were also analyzed, in which it is found that large hidden node number with small regularization parameter leads to the best performance. A comparison among FOS-ELM and SGBP was also conducted. The result indicates that FOS-ELM achieves better tracking and convergence performance than SGBP, since FOS-ELM tends to learn the unknown engine model globally whereas SGBP tends to “forget” what it has learnt. This implies that FOS-ELM is more preferable for adaptive control applications

A new linear parametrization for peak friction coefficient estimation in real time

The correct estimation of the friction coefficient in automotive applications is of paramount importance in the design of effective vehicle safety systems. In this article a new parametrization for estimating the peak friction coefficient, in the tire-road interface, is presented. The proposed parametrization is based on a feedforward neural network (FFNN), trained by the Extreme Learning Machine (ELM) method. Unlike traditional learning techniques for FFNN, typically based on backpropagation and inappropriate for real time implementation, the ELM provides a learning process based on random assignment in the weights between input and the hidden layer. With this approach, the network training becomes much faster, and the unknown parameters can be identified through simple and robust regression methods, such as the Recursive Least Squares. Simulation results, obtained with the CarSim program, demonstrate a good performance of the proposed parametrization; compared with previous methods described in the literature, the proposed method reduces the estimation errors using a model with a lower number of parameters

Adaptive Control of Nonlinear Discrete-Time Systems by Using OS-ELM Neural Networks

As a kind of novel feedforward neural network with single hidden layer, ELM (extreme learning machine) neural networks are studied for the identification and control of nonlinear dynamic systems. The property of simple structure and fast convergence of ELM can be shown clearly. In this paper, we are interested in adaptive control of nonlinear dynamic plants by using OS-ELM (online sequential extreme learning machine) neural networks. Based on data scope division, the problem that training process of ELM neural network is sensitive to the initial training data is also solved. According to the output range of the controlled plant, the data corresponding to this range will be used to initialize ELM. Furthermore, due to the drawback of conventional adaptive control, when the OS-ELM neural network is used for adaptive control of the system with jumping parameters, the topological structure of the neural network can be adjusted dynamically by using multiple model switching strategy, and an MMAC (multiple model adaptive control) will be used to improve the control performance. Simulation results are included to complement the theoretical results