Evolutionary Learning of Fuzzy Rules for Regression

The objective of this PhD Thesis is to design Genetic Fuzzy Systems (GFS) that learn Fuzzy Rule Based Systems to solve regression problems in a general manner. Particularly, the aim is to obtain models with low complexity while maintaining high precision without using expert-knowledge about the problem to be solved. This means that the GFSs have to work with raw data, that is, without any preprocessing that help the learning process to solve a particular problem. This is of particular interest, when no knowledge about the input data is available or for a first approximation to the problem. Moreover, within this objective, GFSs have to cope with large scale problems, thus the algorithms have to scale with the data

Enhancing soft computing techniques to actively address imbalanced regression problems

This paper has been supported in part by the ERDF A way of making Europe/Health Institute Carlos III/Spanish Ministry of Science, Innovation and Universities (grant number PI20/00711), by the ERDF A way of making Europe/Regional Government of Andalusia/Ministry of Economic Transformation, Industry, Knowledge and Universities (grant numbers P18-RT-2248 and B-CTS-536-UGR20) and by the MCIN/AEI/10.13039/50110001103 (grant numbers PID2019-107793GB-I00 and PID2020-119478GB-I00). Funding for open access charge: Universidad de Granada / CBUA

Model fusion using fuzzy aggregation: Special applications to metal properties

To improve the modelling performance, one should either propose a new modelling methodology or make the best of existing models. In this paper, the study is concentrated on the latter solution, where a structure-free modelling paradigm is proposed. It does not rely on a fixed structure and can combine various modelling techniques in ‘symbiosis’ using a ‘master fuzzy system’. This approach is shown to be able to include the advantages of different modelling techniques altogether by requiring less training and by minimising the efforts relating optimisation of the final structure. The proposed approach is then successfully applied to the industrial problems of predicting machining induced residual stresses for aerospace alloy components as well as modelling the mechanical properties of heat-treated alloy steels, both representing complex, non-linear and multi-dimensional environments

Ensemble of heterogeneous flexible neural trees using multiobjective genetic programming

Machine learning algorithms are inherently multiobjective in nature, where approximation error minimization and model's complexity simplification are two conflicting objectives. We proposed a multiobjective genetic programming (MOGP) for creating a heterogeneous flexible neural tree (HFNT), tree-like flexible feedforward neural network model. The functional heterogeneity in neural tree nodes was introduced to capture a better insight of data during learning because each input in a dataset possess different features. MOGP guided an initial HFNT population towards Pareto-optimal solutions, where the final population was used for making an ensemble system. A diversity index measure along with approximation error and complexity was introduced to maintain diversity among the candidates in the population. Hence, the ensemble was created by using accurate, structurally simple, and diverse candidates from MOGP final population. Differential evolution algorithm was applied to fine-tune the underlying parameters of the selected candidates. A comprehensive test over classification, regression, and time-series datasets proved the efficiency of the proposed algorithm over other available prediction methods. Moreover, the heterogeneous creation of HFNT proved to be efficient in making ensemble system from the final population

Multiobjective programming for type-2 hierarchical fuzzy inference trees

This paper proposes a design of hierarchical fuzzy inference tree (HFIT). An HFIT produces an optimum tree-like structure. Specifically, a natural hierarchical structure that accommodates simplicity by combining several low-dimensional fuzzy inference systems (FISs). Such a natural hierarchical structure provides a high degree of approximation accuracy. The construction of HFIT takes place in two phases. Firstly, a nondominated sorting based multiobjective genetic programming (MOGP) is applied to obtain a simple tree structure (low model’s complexity) with a high accuracy. Secondly, the differential evolution algorithm is applied to optimize the obtained tree’s parameters. In the obtained tree, each node has a different input’s combination, where the evolutionary process governs the input’s combination. Hence, HFIT nodes are heterogeneous in nature, which leads to a high diversity among the rules generated by the HFIT. Additionally, the HFIT provides an automatic feature selection because it uses MOGP for the tree’s structural optimization that accept inputs only relevant to the knowledge contained in data. The HFIT was studied in the context of both type-1 and type-2 FISs, and its performance was evaluated through six application problems. Moreover, the proposed multiobjective HFIT was compared both theoretically and empirically with recently proposed FISs methods from the literature, such as McIT2FIS, TSCIT2FNN, SIT2FNN, RIT2FNS-WB, eT2FIS, MRIT2NFS, IT2FNN-SVR, etc. From the obtained results, it was found that the HFIT provided less complex and highly accurate models compared to the models produced by most of the other methods. Hence, the proposed HFIT is an efficient and competitive alternative to the other FISs for function approximation and feature selectio

A Fuzzy Rule-Based System to Predict Energy Consumption of Genetic Programming Algorithms

In recent years, the energy-awareness has become one of the most interesting areas in our environmentally conscious society. Algorithm designers have been part of this, particularly when dealing with networked devices and, mainly, when handheld ones are involved. Although studies in this area has increased, not many of them have focused on Evolutionary Algorithms. To the best of our knowledge, few attempts have been performed before for modeling their energy consumption considering different execution devices. In this work, we propose a fuzzy rulebased system to predict energy comsumption of a kind of Evolutionary Algorithm, Genetic Prohramming, given the device in wich it will be executed, its main parameters, and a measurement of the difficulty of the problem addressed. Experimental results performed show that the proposed model can predict energy consumption with very low error values.We acknowledge support from Spanish Ministry of Economy and Competitiveness under projects TIN2014-56494-C4-[1,2,3]-P and TIN2017-85727-C4- [2,4]-P, Regional Government of Extremadura, Department of Commerce and Economy, conceded by the European Regional Development Fund, a way to build Europe, under the project IB16035, and Junta de Extremadura FEDER, projects GR15068 and GR15130

Dynamic non-linear system modelling using wavelet-based soft computing techniques

The enormous number of complex systems results in the necessity of high-level and cost-efficient modelling structures for the operators and system designers. Model-based approaches offer a very challenging way to integrate a priori knowledge into the procedure. Soft computing based models in particular, can successfully be applied in cases of highly nonlinear problems. A further reason for dealing with so called soft computational model based techniques is that in real-world cases, many times only partial, uncertain and/or inaccurate data is available. Wavelet-Based soft computing techniques are considered, as one of the latest trends in system identification/modelling. This thesis provides a comprehensive synopsis of the main wavelet-based approaches to model the non-linear dynamical systems in real world problems in conjunction with possible twists and novelties aiming for more accurate and less complex modelling structure. Initially, an on-line structure and parameter design has been considered in an adaptive Neuro- Fuzzy (NF) scheme. The problem of redundant membership functions and consequently fuzzy rules is circumvented by applying an adaptive structure. The growth of a special type of Fungus (Monascus ruber van Tieghem) is examined against several other approaches for further justification of the proposed methodology. By extending the line of research, two Morlet Wavelet Neural Network (WNN) structures have been introduced. Increasing the accuracy and decreasing the computational cost are both the primary targets of proposed novelties. Modifying the synoptic weights by replacing them with Linear Combination Weights (LCW) and also imposing a Hybrid Learning Algorithm (HLA) comprising of Gradient Descent (GD) and Recursive Least Square (RLS), are the tools utilised for the above challenges. These two models differ from the point of view of structure while they share the same HLA scheme. The second approach contains an additional Multiplication layer, plus its hidden layer contains several sub-WNNs for each input dimension. The practical superiority of these extensions is demonstrated by simulation and experimental results on real non-linear dynamic system; Listeria Monocytogenes survival curves in Ultra-High Temperature (UHT) whole milk, and consolidated with comprehensive comparison with other suggested schemes. At the next stage, the extended clustering-based fuzzy version of the proposed WNN schemes, is presented as the ultimate structure in this thesis. The proposed Fuzzy Wavelet Neural network (FWNN) benefitted from Gaussian Mixture Models (GMMs) clustering feature, updated by a modified Expectation-Maximization (EM) algorithm. One of the main aims of this thesis is to illustrate how the GMM-EM scheme could be used not only for detecting useful knowledge from the data by building accurate regression, but also for the identification of complex systems. The structure of FWNN is based on the basis of fuzzy rules including wavelet functions in the consequent parts of rules. In order to improve the function approximation accuracy and general capability of the FWNN system, an efficient hybrid learning approach is used to adjust the parameters of dilation, translation, weights, and membership. Extended Kalman Filter (EKF) is employed for wavelet parameters adjustment together with Weighted Least Square (WLS) which is dedicated for the Linear Combination Weights fine-tuning. The results of a real-world application of Short Time Load Forecasting (STLF) further re-enforced the plausibility of the above technique

Multiobjective evolutionary optimization of quadratic Takagi-Sugeno fuzzy rules for remote bathymetry estimation

In this work we tackle the problem of bathymetry estimation using: i) a multispectral optical image of the region of interest, and ii) a set of in situ measurements. The idea is to learn the relation that between the reflectances and the depth using a supervised learning approach. In particular, quadratic Takagi-Sugeno fuzzy rules are used to model this relation. The rule base is optimized by means of a multiobjective evolutionary algorithm. To the best of our knowledge this work represents the first use of a quadratic Takagi-Sugeno fuzzy system optimized by a multiobjective evolutionary algorithm with bounded complexity, i.e., able to control the complexity of the consequent part of second-order fuzzy rules. This model has an outstanding modeling power, without inheriting the drawback of complexity due to the use of quadratic functions (which have complexity that scales quadratically with the number of inputs). This opens the way to the use of the proposed approach even for medium/high dimensional problems, like in the case of hyper-spectral images

Interpretable fuzzy modeling using multi-objective immune-inspired optimization algorithms

Abstract—In this paper, an immune inspired multi-objective fuzzy modeling (IMOFM) mechanism is proposed specifically for high-dimensional regression problems. For such problems, high predictive accuracy is often the paramount requirement. With such a requirement in mind, however, one should also put considerable efforts in making the elicited model as interpretable as possible, which leads to a difficult optimization problem. The proposed modeling approach adopts a multi-stage modeling procedure and a variable length coding scheme to account for the enlarged search space due to the simultaneous optimization of the rule-base structure and its associated parameters. IMOFM can account for both Singleton and Mamdani Fuzzy Rule-Based Systems (FRBS) due to the carefully chosen output membership functions, the inference and the defuzzification methods. The proposed algorithm has been compared with other representatives using a simple benchmark problem, and has also been applied to a high-dimensional problem which models mechanical properties of hot rolled steels. Results confirm that IMOFM can elicit accurate and yet transparent FRBSs from quantitative data