Opening the Neural Network Black Box: An Algorithm for Extracting Rules from Function Approximating Artificial Neural Networks

Artificial neural networks have been successfully applied to solve a variety of business applications involving classification and function approximation. In many such applications, it is desirable to extract knowledge from trained neural networks so that the users can gain a better understanding of the solution. Existing research has focused primarily on extracting symbolic rules for classification problems with few methods devised for function approximation problems. In order to fill this gap, we propose an approach to extract rules from neural networks that have been trained to solve function approximation problems. The extracted rules divide the data samples into groups. For all samples within a group, a linear function of the relevant input attributes of the data approximates the network output. Experimental results show that the proposed approach generates rules that are more accurate than the existing methods based on decision trees and regression

Maximal Associated Regression: A nonlinear extension to Least Angle Regression

This paper proposes Maximal Associated Regression (MAR), a novel algorithm that performs forward stage-wise regression by applying nonlinear transformations to fit predictor covariates. For each predictor, MAR selects between a linear or additive fit as determined by the dataset. The proposed algorithm is an adaptation of Least Angle Regression (LARS) and retains its efficiency in building sparse models. Constrained penalized splines are used to generate smooth nonlinear transformations for the additive fits. A monotonically constrained extension of MAR (MARm) is also introduced in this paper to fit isotonic regression problems. The proposed algorithms are validated on both synthetic and real datasets. The performances of MAR and MARm are compared against LARS, Generalized Linear Models (GLM), and Generalized Additive Models (GAM) under the Gaussian assumption with a unity link function. Results indicate that MAR-type algorithms achieve a superior subset selection accuracy, generating sparser models that generalize well to new data. MAR is also able to generate models for sample deficient datasets. Thus, MAR is proposed as a valuable tool for subset selection and data exploration, especially when a priori knowledge of the dataset is unavailable