Hybrid rule-extraction from support vector machines

Rule-extraction from artificial neural networks(ANNs) as well as support vector machines (SVMs) provide explanations for the decisions made by these systems. This explanation capability is very important in applications such as medical diagnosis. Over the last decade, a multitude of algorithms for rule-extraction from ANNs have been developed. However, rule-extraction from SVMs is not widely available yet.In this paper, a hybrid approach for rule-extraction from SVMs is outlined. This approach has two basic components: (1) data reduction using a logistic regression model and (2) learning based rule-extraction. The quality of the extracted rules is then evaluated in terms of fidelity, accuracy, consistency and comprehensibility. The rules are also verified against the available knowledge from the domain problem (diabetes) to assure correctness and validity

Eclectic rule-extraction from support vector machines

Support vector machines (SVMs) have shown superior performance compared to other machine learning techniques, especially in classification problems. Yet one limitation of SVMs is the lack of an explanation capability which is crucial in some applications, e.g. in the medical and security domains. In this paper, a novel approach for eclectic rule- extraction from support vector machines is presented. This approach utilizes the knowledge acquired by the SVM and represented in its support vectors as well as the parameters associated with them. The approach includes three stages; training, propositional rule- extraction and rule quality evaluation. Results from four different experiments have demonstrated the value of the approach for extracting comprehensible rules of high accuracy and fidelity

Learning-based Rule-Extraction from Support Vector Machines

In recent years, support vector machines (SVMs) have shown good performance in a number of application areas, including text classification. However, the success of SVMs comes at a cost - an inability to explain the process by which a learning result was reached and why a decision is being made. Rule-extraction from SVMs is important for the acceptance of this machine learning technology, especially for applications such as medical diagnosis. It is crucial for the users to understand how the system makes a decision. In this paper, a novel approach for rule-extraction from support vector machines is presented. This approach handles rule-extraction as a learning task, which proceeds in two steps. The first is to use the labeled patterns from a data set to train an SVM. The second step is to use the generated model to predict the label (class) for an extended data set or different, unlabeled data set. The resulting patterns are then used to train a decision tree learning system and to extract the corresponding rule sets. The output rule sets are verified against available knowledge for the domain problem (e.g. a medical expert), and other classification techniques, to assure correctness and validity of rules

Comprehensible credit scoring models using rule extraction from support vector machines.

In recent years, Support Vector Machines (SVMs) were successfully applied to a wide range of applications. Their good performance is achieved by an implicit non-linear transformation of the original problem to a high-dimensional (possibly infinite) feature space in which a linear decision hyperplane is constructed that yields a nonlinear classifier in the input space. However, since the classifier is described as a complex mathematical function, it is rather incomprehensible for humans. This opacity property prevents them from being used in many real- life applications where both accuracy and comprehensibility are required, such as medical diagnosis and credit risk evaluation. To overcome this limitation, rules can be extracted from the trained SVM that are interpretable by humans and keep as much of the accuracy of the SVM as possible. In this paper, we will provide an overview of the recently proposed rule extraction techniques for SVMs and introduce two others taken from the artificial neural networks domain, being Trepan and G-REX. The described techniques are compared using publicly avail- able datasets, such as Ripley's synthetic dataset and the multi-class iris dataset. We will also look at medical diagnosis and credit scoring where comprehensibility is a key requirement and even a regulatory recommendation. Our experiments show that the SVM rule extraction techniques lose only a small percentage in performance compared to SVMs and therefore rank at the top of comprehensible classification techniques.Credit; Credit scoring; Models; Model; Applications; Performance; Space; Decision; Yield; Real life; Risk; Evaluation; Rules; Neural networks; Networks; Classification; Research;

Learning-Based Rule-Extraction From Support Vector Machines: Performance On Benchmark Data Sets

Over the last decade, rule-extraction from neural networks (ANN) techniques have been developed to explain how classification and regression are realised by the ANN. Yet, this is not the case for support vector machines (SVMs) which also demonstrate an inability to explain the process by which a learning result was reached and why a decision is being made. Rule-extraction from SVMs is important, especially for applications such as medical diagnosis. In this paper, an approach for learning-based rule-extraction from support vector machines is outlined, including an evaluation of the quality of the extracted rules in terms of fidelity, accuracy, consistency and comprehensibility. In addition, the rules are verified by use of knowledge from the problem domains as well as other classification techniques to assure correctness and validity

Rule Extraction from Support Vector Machines: A Geometric Approach. Technical Report

This paper presents a new approach to rule extraction from Support Vector Machines. SVMs have been applied successfully in many areas with excellent generalization results; rule extraction can offer explanation capability to SVMs. We propose to approximate the SVM classification boundary through querying followed by clustering, searching and then to extract rules by solving an optimization problem. Theoretical proof and experimental results then indicate that the rules can be used to validate the SVM results, since maximum fidelity with high accuracy can be achieved

Rule Extraction from Support Vector Machines: A Geometric Approach

Despite the success of connectionist systems in prediction and classi¯cation problems, critics argue that the lack of symbol processing and explanation capability makes them less competitive than symbolic systems. Rule extraction from neural networks makes the interpretation of the behaviour of connectionist networks possible by relating sub-symbolic and symbolic process- ing. However, most rule extraction methods focus only on speci¯c neural network architectures and present limited generalization performance. Support Vector Machine is an unsupervised learning method that has been recently applied successfully in many areas, and o®ers excellent generalization ability in comparison with other neural network, statistical, or symbolic machine learning models. In this thesis, an algorithm called Geometric and Oracle-Based Support Vector Machines Rule Extraction (GOSE) has been proposed to overcome the limitations of other rule-extraction methods by extracting comprehensible models from Support Vector Machines (SVM). This algorithm views the extraction as a geometric task. Given a trained SVM network, GOSE queries the synthetic instances and draws conjunction rules by approximating the optimization problem. The extracted rule set also represents the approximation of the SVM classi¯cation boundary. Unlike previous works in SVM rule-extraction, GOSE is broadly applicable to different networks and problems because it need not rely on training examples and network architectures. Theoretical proof guarantees that GOSE is capable of approximating the behavior of SVM networks. Empirical experiments are conducted on di®erent SVM networks from binary classification networks to multi-class networks in various classi¯cation domains. The result of experiments demonstrates that GOSE can extract comprehensible rules with high levels of accuracy and ¯delity for their corresponding networks. GOSE also exhibits superior consistency. After analyzing and applying several optimizing measures, the complexity of GOSE was improved. In brief, GOSE provides a novel way to explain how an SVM network functions

Rule Extraction from Support Vector Machines: Measuring the Explanation Capability Using the Area under the ROC Curve

Recently, the area of rule extraction from support vector machines (SVMs) has been explored. One important indication of the success of a rule extraction method is the performance of extracted rules as compared to the original SVM. In this paper, we describe the use of the area under the receiver operating characteristics (ROC) curve (AUC) to assess the quality of rules extracted from an SVM. In particular, we directly compare AUC to the more commonly used measures of accuracy and fidelity and show that AUC is both a more reliable and meaningful measure to use

An information extraction tool for microbial characters

Automated extraction of phenotypic and metabolic characters from microbial taxonomic descriptions will benefit biology research and study. In this poster, we describe a Microbial Phenomics Information Extractor (MicroPIE) system. MicroPIE takes taxonomic descriptions in XML files as input and can extract 58 types of microbial characters. The main extraction steps are :1) splitting paragraphs into sentences; 2)predicting the characters described in the sentences by using automated classifiers; 3)extracting character values from the sentences by applying a variety of methods, such as Regular Expression Rule, Term Matching, and Unsupervised Semantic Parsing. Parts of the system have been implemented and currently been optimized for better performance. Results on optimizing the sentence classifiers show that the SVMs (Support Vector Machines) achieved better performance over the Naive Bayes classifiers, in addition, resolving the problem of unbalanced training instances helped improve the performance of SVMs

A rule-based parameter aided with object-based classification approach for extraction of building and roads from WorldView-2 images

Roads and buildings constitute a significant proportion of urban areas. Considerable amount of research has been done on the road and building extraction from remotely sensed imagery. However, a few of them have been concentrating on using only spectral information. This study presents a comparison between three object-based models for urban features’ classification, specifically roads and buildings, from WorldView-2 satellite imagery. The three applied algorithms are support vector machines (SVMs), nearest neighbour (NN) and proposed rule-based system. The results indicated that the proposed rules in this study, despite the spectral complexity of land cover types, performed a satisfactory output with an overall accuracy of 92.92%. The advantages offered by the proposed rules were not provided by other two applied algorithms and it revealed the highest accuracy compared to SVM and NN. The overall accuracy for SVM was 76.76%, which is almost similar to the result achieved by NN (77.3%)