Adaptive Multi-Output Gradient RBF Tracker For Nonlinear and Nonstationary Regression

Multioutput regression of nonlinear and nonstationary data is largely understudied in both machine learning and control communities. This article develops an adaptive multioutput gradient radial basis function (MGRBF) tracker for online modeling of multioutput nonlinear and nonstationary processes. Specifically, a compact MGRBF network is first constructed with a new two-step training procedure to produce excellent predictive capacity. To improve its tracking ability in fast time-varying scenarios, an adaptive MGRBF (AMGRBF) tracker is proposed, which updates the MGRBF network structure online by replacing the worst performing node with a new node that automatically encodes the newly emerging system state and acts as a perfect local multioutput predictor for the current system state. Extensive experimental results confirm that the proposed AMGRBF tracker significantly outperforms existing state-of-the-art online multioutput regression methods as well as deep-learning-based models, in terms of adaptive modeling accuracy and online computational complexity.</p

Protein Superfamily Classification using Computational Intelligence Techniques

The problem of protein superfamily classification is a challenging research area in Bioinformatics and has its major application in drug discovery. If a newly discovered protein which is responsible for the cause of new disease gets correctly classified to its superfamily, then the task of the drug analyst becomes much easier. The analyst can perform molecular docking to find the correct relative orientation of ligand for the protein. The ligand database can be searched for all possible orientations and conformations of the protein belonging to that superfamily paired with the ligand. Thus, the search space is reduced enormously as the protein-ligand pair is searched for a particular protein superfamily. Therefore, correct classification of proteins becomes a very challenging task as it guides the analysts to discover appropriate drugs. In this thesis, Neural Networks (NN), Multiobjective Genetic Algorithm (MOGA),and Support Vector Machine (SVM) are applied to perform the classification task.Adaptive MultiObjective Genetic Algorithm (AMOGA), which is a variation of MOGA is implemented for the structure optimization of Radial Basis Function Network (RBFN). The modification to MOGA is done based on the two key controlling parameters such as probability of crossover and probability of mutation. These values are adaptively varied based upon the performance of the algorithm, i.e., based upon the percentage of the total population present in the best non-domination level. The problem of finding the number of hidden centers remains a critical issue for the design of RBFN. The most optimal RBF network with good generalization ability can be derived from the pareto optimal set. Therefore, every solution of the pareto optimal set gives information regarding the specific samples to be chosen as hidden centers as well as the update weight matrix connecting the hidden and output layer. Principal Component Analysis (PCA) has been used for dimension reduction and significant feature extraction from long feature vector of amino acid sequences.In two-stage approach for protein superfamily classification, feature extraction process is carried in the first stage and design of the classifier has been proposed in the second stage with an overall objective to maximize the performance accuracy of the classifier. In the feature extraction phase, Genetic Algorithm(GA) based wrapper approach is used to select few eigen vectors from the PCA space which are encoded as binary strings in the chromosome. Using PCA-NSGA-II (non-dominated sorting GA), the non-dominated solutions obtained from the pareto front solves the trade-off problem by compromising between the number of eigen vectors selected and the accuracy obtained by the classifier. In the second stage, Recursive Orthogonal Least Square Algorithm (ROLSA) is used for training RBFN. ROLSA selects the optimal number o

Improving time efficiency of feedforward neural network learning

Feedforward neural networks have been widely studied and used in many applications in science and engineering. The training of this type of networks is mainly undertaken using the well-known backpropagation based learning algorithms. One major problem with this type of algorithms is the slow training convergence speed, which hinders their applications. In order to improve the training convergence speed of this type of algorithms, many researchers have developed different improvements and enhancements. However, the slow convergence problem has not been fully addressed. This thesis makes several contributions by proposing new backpropagation learning algorithms based on the terminal attractor concept to improve the existing backpropagation learning algorithms such as the gradient descent and Levenberg-Marquardt algorithms. These new algorithms enable fast convergence both at a distance from and in a close range of the ideal weights. In particular, a new fast convergence mechanism is proposed which is based on the fast terminal attractor concept. Comprehensive simulation studies are undertaken to demonstrate the effectiveness of the proposed backpropagataion algorithms with terminal attractors. Finally, three practical application cases of time series forecasting, character recognition and image interpolation are chosen to show the practicality and usefulness of the proposed learning algorithms with comprehensive comparative studies with existing algorithms

Development of Fuzzy System Based Channel Equalisers

Channel equalisers are used in digital communication receivers to mitigate the effects of inter symbol interference (ISI) and inter user interference in the form of co-channel interference (CCI) and adjacent channel interference (ACI) in the presence of additive white Gaussian noise (AWGN). An equaliser uses a large part of the computations involved in the receiver. Linear equalisers based on adaptive filtering techniques have long been used for this application. Recently, use of nonlinear signal processing techniques like artificial neural networks (ANN) and radial basis functions (RBF) have shown encouraging results in this application. This thesis presents the development of a nonlinear fuzzy system based equaliser for digital communication receivers. The fuzzy equaliser proposed in this thesis provides a parametric implementation of symbolby-symbol maximum a-posteriori probability (MAP) equaliser based on Bayes’s theory. This MAP equaliser is also called Bayesian equaliser. Its decision function uses an estimate of the noise free received vectors, also called channel states or channel centres. The fuzzy equaliser developed here can be implemented with lower computational complexity than the RBF implementation of the MAP equaliser by using scalar channel states instead of channel states. It also provides schemes for performance tradeoff with complexity and schemes for subset centre selection. Simulation studies presented in this thesis suggests that the fuzzy equaliser by using only 10%-20% of the Bayesian equaliser channel states can provide near optimal performance. Subsequently, this fuzzy equaliser is modified for CCI suppression and is termed fuzzy–CCI equaliser. The fuzzy–CCI equaliser provides a performance comparable to the MAP equaliser designed for channels corrupted with CCI. However the structure of this equaliser is similar to the MAP equaliser that treats CCI as AWGN. A decision feedback form of this equaliser which uses a subset of channel states based on the feedback state is derived. Simulation studies presented in this thesis demonstrate that the fuzzy–CCI equaliser can effectively remove CCI without much increase in computational complexity. This equaliser is also successful in removing interference from more than one CCI sources, where as the MAP equalisers treating CCI as AWGN fail. This fuzzy–CCI equaliser can be treated as a fuzzy equaliser with a preprocessor for CCI suppression, and the preprocessor can be removed under high signal to interference ratio condition

Large Scale Kernel Methods for Fun and Profit

Kernel methods are among the most flexible classes of machine learning models with strong theoretical guarantees. Wide classes of functions can be approximated arbitrarily well with kernels, while fast convergence and learning rates have been formally shown to hold. Exact kernel methods are known to scale poorly with increasing dataset size, and we believe that one of the factors limiting their usage in modern machine learning is the lack of scalable and easy to use algorithms and software. The main goal of this thesis is to study kernel methods from the point of view of efficient learning, with particular emphasis on large-scale data, but also on low-latency training, and user efficiency. We improve the state-of-the-art for scaling kernel solvers to datasets with billions of points using the Falkon algorithm, which combines random projections with fast optimization. Running it on GPUs, we show how to fully utilize available computing power for training kernel machines. To boost the ease-of-use of approximate kernel solvers, we propose an algorithm for automated hyperparameter tuning. By minimizing a penalized loss function, a model can be learned together with its hyperparameters, reducing the time needed for user-driven experimentation. In the setting of multi-class learning, we show that – under stringent but realistic assumptions on the separation between classes – a wide set of algorithms needs much fewer data points than in the more general setting (without assumptions on class separation) to reach the same accuracy. The first part of the thesis develops a framework for efficient and scalable kernel machines. This raises the question of whether our approaches can be used successfully in real-world applications, especially compared to alternatives based on deep learning which are often deemed hard to beat. The second part aims to investigate this question on two main applications, chosen because of the paramount importance of having an efficient algorithm. First, we consider the problem of instance segmentation of images taken from the iCub robot. Here Falkon is used as part of a larger pipeline, but the efficiency afforded by our solver is essential to ensure smooth human-robot interactions. In the second instance, we consider time-series forecasting of wind speed, analysing the relevance of different physical variables on the predictions themselves. We investigate different schemes to adapt i.i.d. learning to the time-series setting. Overall, this work aims to demonstrate, through novel algorithms and examples, that kernel methods are up to computationally demanding tasks, and that there are concrete applications in which their use is warranted and more efficient than that of other, more complex, and less theoretically grounded models

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available