Building an interpretable fuzzy rule base from data using Orthogonal Least Squares Application to a depollution problem

In many fields where human understanding plays a crucial role, such as bioprocesses, the capacity of extracting knowledge from data is of critical importance. Within this framework, fuzzy learning methods, if properly used, can greatly help human experts. Amongst these methods, the aim of orthogonal transformations, which have been proven to be mathematically robust, is to build rules from a set of training data and to select the most important ones by linear regression or rank revealing techniques. The OLS algorithm is a good representative of those methods. However, it was originally designed so that it only cared about numerical performance. Thus, we propose some modifications of the original method to take interpretability into account. After recalling the original algorithm, this paper presents the changes made to the original method, then discusses some results obtained from benchmark problems. Finally, the algorithm is applied to a real-world fault detection depollution problem.Comment: pre-print of final version published in Fuzzy Sets and System

Using rule extraction to improve the comprehensibility of predictive models.

Whereas newer machine learning techniques, like artifficial neural net-works and support vector machines, have shown superior performance in various benchmarking studies, the application of these techniques remains largely restricted to research environments. A more widespread adoption of these techniques is foiled by their lack of explanation capability which is required in some application areas, like medical diagnosis or credit scoring. To overcome this restriction, various algorithms have been proposed to extract a meaningful description of the underlying `blackbox' models. These algorithms' dual goal is to mimic the behavior of the black box as closely as possible while at the same time they have to ensure that the extracted description is maximally comprehensible. In this research report, we first develop a formal definition of`rule extraction and comment on the inherent trade-off between accuracy and comprehensibility. Afterwards, we develop a taxonomy by which rule extraction algorithms can be classiffied and discuss some criteria by which these algorithms can be evaluated. Finally, an in-depth review of the most important algorithms is given.This report is concluded by pointing out some general shortcomings of existing techniques and opportunities for future research.Models; Model; Algorithms; Criteria; Opportunities; Research; Learning; Neural networks; Networks; Performance; Benchmarking; Studies; Area; Credit; Credit scoring; Behavior; Time;

An approach to generate rules from neural networks for regression problems

10.1016/S0377-2217(02)00792-0European Journal of Operational Research1551239-250EJOR

LABORATORY EVALUATION AND NEURAL NETWORK MODELING FOR ROTATIONAL VISCOSITY OF REACTED AND ACTIVATED RUBBER MODIFIED BINDERS

Crumb rubber surface activation and pretreatment are considered as one of the promising newly introduced methods for asphalt rubber production. Reacted and Activated Rubber (RAR) is an elastomeric asphalt extender produced by the hot blending and activation of crumb rubber with asphalt and Activated Mineral Binder Stabilizer (AMBS). Besides RAR ability in enhancing the performance of asphaltic mixtures, its dry granulate industrial form enabled its addition directly into the mixture utilizing pugmill or the dryer drum with very minimal to no modification required on the plant level. This study aims to evaluate the rotational viscosity of RAR modified binders and develop an Artificial Neural Network (ANN) viscosity prediction model for extracting a stand-alone viscosity prediction equation. Three different Performance Graded (PG) asphalt binders modified by ten dosages of RAR were tested and evaluated under this study. Sixty-six samples that generated more than three thousand viscosity data point were utilized in binder performance evaluation and ANN modeling. The study concluded that RAR addition has decreased binder temperature susceptibility in considerable amounts when compared to the virgin binders. Furthermore, it was demonstrated that the testing shearing rate had a significant effect on the measured viscosity values for binders modified with high RAR content. The developed ANN model as well as the extracted stand-alone viscosity prediction equation had a high value of the coefficient of determination and were statistically valid. Both of them has the ability to predict the RAR modified binder viscosity as a function of binder grade, temperature, testing shearing rates, and RAR content

Data mining using neural networks

Data mining is about the search for relationships and global patterns in large databases that are increasing in size. Data mining is beneficial for anyone who has a huge amount of data, for example, customer and business data, transaction, marketing, financial, manufacturing and web data etc. The results of data mining are also referred to as knowledge in the form of rules, regularities and constraints. Rule mining is one of the popular data mining methods since rules provide concise statements of potentially important information that is easily understood by end users and also actionable patterns. At present rule mining has received a good deal of attention and enthusiasm from data mining researchers since rule mining is capable of solving many data mining problems such as classification, association, customer profiling, summarization, segmentation and many others. This thesis makes several contributions by proposing rule mining methods using genetic algorithms and neural networks. The thesis first proposes rule mining methods using a genetic algorithm. These methods are based on an integrated framework but capable of mining three major classes of rules. Moreover, the rule mining processes in these methods are controlled by tuning of two data mining measures such as support and confidence. The thesis shows how to build data mining predictive models using the resultant rules of the proposed methods. Another key contribution of the thesis is the proposal of rule mining methods using supervised neural networks. The thesis mathematically analyses the Widrow-Hoff learning algorithm of a single-layered neural network, which results in a foundation for rule mining algorithms using single-layered neural networks. Three rule mining algorithms using single-layered neural networks are proposed for the three major classes of rules on the basis of the proposed theorems. The thesis also looks at the problem of rule mining where user guidance is absent. The thesis proposes a guided rule mining system to overcome this problem. The thesis extends this work further by comparing the performance of the algorithm used in the proposed guided rule mining system with Apriori data mining algorithm. Finally, the thesis studies the Kohonen self-organization map as an unsupervised neural network for rule mining algorithms. Two approaches are adopted based on the way of self-organization maps applied in rule mining models. In the first approach, self-organization map is used for clustering, which provides class information to the rule mining process. In the second approach, automated rule mining takes the place of trained neurons as it grows in a hierarchical structure

Modelos de previsão de acidentes em rodovia brasileira de pista dupla

Tese (doutorado)—Universidade de Brasília, Faculdade de Tecnologia, Departamento de Engenharia Civil e Ambiental, 2019.A modelagem da segurança viária é uma importante estratégia para o gerenciamento da segurança viária (GSV). O desenvolvimento de modelos de resposta multivariada (consideração simultânea de ocorrência de acidentes de diferentes níveis de severidade) também tem grande utilidade e carece de exploração. Um desafio inicial na previsão de frequência de acidentes é a estratégia de segmentação da rodovia, especialmente quanto a extensão do segmento. Em relação a abordagem metodológica, técnicas de aprendizado de máquina (AM), especialmente redes neurais artificiais (RNA), são apresentadas como potencial alternativa. Diante disso, o estudo desenvolvido buscou avaliar o potencial de utilização de técnicas de AM para o desenvolvimento de modelos de previsão de frequência de acidentes segundo três níveis de severidade, e ainda, avaliar a influência da extensão do segmento no desenvolvimento deste tipo de modelo. O trecho de rodovia de pista dupla analisado foi segmentado em 10 diferentes extensões fixas, sendo caracterizado por variáveis de geometria, operação e pavimento, nos períodos 2011-2014 e 2015-2018. Para fins comparativos, foi empregado o modelo multivariado Poisson lognormal (MVPLN) no processo de modelagem. Todos os modelos desenvolvidos tiveram a validade replicativa investigada e, destes, foram selecionados os melhores modelos que foram, também, verificados em termos da validação preditiva e estrutural. Esse procedimento permitiu confirmar a adequação do uso de RNA para a modelagem proposta – com ligeira superioridade à outra abordagem –, mas também revelou limitações dessa técnica. De igual modo, as vantagens e fragilidades dos modelos MVPLN também foram conhecidas. Assim foi possível concluir que duas técnicas são adequadas para a modelagem da segurança viária, devendo ser empregada aquela que melhor se adeque ao propósito do estudo em questão, ou ainda, considerar a associação entre elasCAPESRoad Safety Modeling is an important strategy for road safety management (RSM). The development of multivariate response models (considering the simultaneous occurrence of crashes with different levels of severity) is also of great usefulness and lacks exploration. An initial challenge in crash prediction models is the highway segmentation method, mainly concerning segment lengths. Regarding the methodological approach, machine learning (ML) techniques, especially artificial neural networks (ANN), are presented as a possible alternative. Therefore, this study aimed to evaluate the potential of using ML techniques to develop crash prediction models in terms of three levels of severity and, besides that, evaluate how length segmentation influences this type of model. The data consisted of a multilane highway, which was divided in 10 different fixed lengths, being characterized by means of geometric, operational and paving characteristics, in two periods: 2011-2014 and 2015-2018. For comparison, the multivariate Poisson lognormal model (MVPLN) was used. All the developed models had their replicative validation tested and, among these, the best models were selected, which were also verified in terms of predictive and structural validations. This procedure allowed to assure the properness of using ANN for the proposed modelling – being slightly superior to the other approach – but also showed limitations of this technique. Similarly to that, the pros and cons of the MVPLN were highlighted. Thus, it was possible to conclude that both techniques are proper for modeling road safety, and it is necessary to apply the one that better fits the purpose of the study in question or even consider an association between them

Abstract

An Approach to Generate Rules from Neural Networks for Regression Problems Artificial neural networks have been successfully applied to a variety of business application problems involving classification and regression. They are especially useful for regression problems as they do not require prior knowledge about the data distribution. In many applications, it is desirable to extract knowledge from trained neural networks so that the users can gain a better understanding of the solution. Existing research works have focused primarily on extracting symbolic rules for classification problems with few methods devised for regression problems. In order to fill this gap, we propose an approach to extract rules from neural networks that have been trained to solve regression problems. The extracted rules divide the data samples into groups. For all samples within a group, a linear function of the relevant input attributes of the data approximates the network output. Experimental results show that the proposed approach generates rules that are more accurate than the existing methods based on decision trees and linear regression. The approach is illustrated with three examples on various application problems