Incremental learning with respect to new incoming input attributes

Neural networks are generally exposed to a dynamic environment where the training patterns or the input attributes (features) will likely be introduced into the current domain incrementally. This paper considers the situation where a new set of input attributes must be considered and added into the existing neural network. The conventional method is to discard the existing network and redesign one from scratch. This approach wastes the old knowledge and the previous effort. In order to reduce computational time, improve generalization accuracy, and enhance intelligence of the learned models, we present ILIA algorithms (namely ILIA1, ILIA2, ILIA3, ILIA4 and ILIA5) capable of Incremental Learning in terms of Input Attributes. Using the ILIA algorithms, when new input attributes are introduced into the original problem, the existing neural network can be retained and a new sub-network is constructed and trained incrementally. The new sub-network and the old one are merged later to form a new network for the changed problem. In addition, ILIA algorithms have the ability to decide whether the new incoming input attributes are relevant to the output and consistent with the existing input attributes or not and suggest to accept or reject them. Experimental results show that the ILIA algorithms are efficient and effective both for the classification and regression problems

Modular feature selection using relative importance factors

Feature selection plays an important role in finding relevant or irrelevant features in classification. Genetic algorithms (GAs) have been used as conventional methods for classifiers to adaptively evolve solutions for classification problems. In this paper, we explore the use of feature selection in modular GA-based classification. We propose a new feature selection technique, Relative Importance Factor (RIF), to find irrelevant features in the feature space of each module. By removing these features, we aim to improve classification accuracy and reduce the dimensionality of classification problems. Benchmark classification data sets are used to evaluate the proposed approaches. The experiment results show that RIF can be used to determine irrelevant features and help achieve higher classification accuracy with the feature space dimension reduced. The complexity of the resulting rule sets is also reduced which means the modular classifiers with irrelevant features removed will be able to classify data with a higher throughput

Feature selection for modular GA-based classification

Genetic algorithms (GAs) have been used as conventional methods for classifiers to adaptively evolve solutions for classification problems. Feature selection plays an important role in finding relevant features in classification. In this paper, feature selection is explored with modular GA-based classification. A new feature selection technique, Relative Importance Factor (RIF), is proposed to find less relevant features in the input domain of each class module. By removing these features, it is aimed to reduce the classification error and dimensionality of classification problems. Benchmark classification data sets are used to evaluate the proposed approach. The experiment results show that RIF can be used to find less relevant features and help achieve lower classification error with the feature space dimension reduced

A Learning Algorithm based on High School Teaching Wisdom

A learning algorithm based on primary school teaching and learning is presented. The methodology is to continuously evaluate a student and to give them training on the examples for which they repeatedly fail, until, they can correctly answer all types of questions. This incremental learning procedure produces better learning curves by demanding the student to optimally dedicate their learning time on the failed examples. When used in machine learning, the algorithm is found to train a machine on a data with maximum variance in the feature space so that the generalization ability of the network improves. The algorithm has interesting applications in data mining, model evaluations and rare objects discovery

Generating Interpretable Fuzzy Systems for Classification Problems

En este artículo se presenta un nuevo método para generar sistemas difusos interpretables, a partir de datos experimentalesde entrada y salida, para resolver problemas de clasificación. En la partición antecedente se emplean conjuntos triangulares con interpolación de 0.5 lo cual evita la presencia de solapamientos complejos que suceden en otros métodos. Los consecuentes, tipo Singleton, son generados por la proyección de los valores modales de cada función de membresía triangular en el espacio de salida y se emplea el método de mínimos cuadrados para el ajuste de los consecuentes. El método propuesto consigue una mayor precisión que la alcanzada con los métodos actuales existentes, empleando un número reducido de reglas y parámetros y sin sacrificar la interpretabilidad del modelo difuso. El enfoque propuesto es aplicado a dos problemas clásicos de clasificación: el Wisconsin Breast Cancer (WBC) y el Iris Data Classification Problem, para mostrar las ventajas del método y comparar los resultados con los alcanzados por otros investigadores.This paper presents a new method to generate interpretable fuzzy systems from training data to deal with classification problems. The antecedent partition uses triangular sets with 0.5 interpolations avoiding the presence of complex overlapping that happens in another method. Singleton consequents are generated form the projection of the modal values of each triangular membership function into the output space. Least square method is used to adjust the consequents. The proposed method gets a higher average classification accuracy rate than the existing methods with a reduced number of rules andparameters and without sacrificing the fuzzy system interpretability. The proposed approach is applied to two classical classification problems: Iris data and the Wisconsin Breast Cancer classification problem

Usage of Simplified Fuzzy ARTMAP for improvement of classification performances

This study presents a simplified fuzzy ARTMAP (SFAM) for different classification applications. The proposed SFAM model is synergy of fuzzy logic and adaptive resonance theory (ART) neural networks. SFAM is supervised network consisting of two layers (Fuzzy ART and Inter ART) that build constant classification groups in answer to series of input patterns. Fuzzy ART layer takes a series of input patterns and relate them to output classes. Inter ART layer functions in such a way that it raises the vigilance parameter of fuzzy ART layer. By combining this two layers, SFAM is capable to perform classification very efficiently and giving very high performances. Lastly, the SFAM model is applied to different simulations. The simulation results obtained for the three different datasets: Iris, Wisconsin breast cancer and wine dataset prove that SFAM model has better performance results than other models for these classification applications

Improving Neural Network Performance with Feature Selection Using Pearson Correlation Method for Diabetes Disease Detection

Diabetic or silent killer diseases are an alarming scourge for the world and are classed as serious diseases. In Indonesia, the increase in diabetics occurred by 2% in vulnerable times between 2013 to 2018. This affects all sectors, both medical services and the financial sector. The Neural Network method as a data mining algorithm is present to overcome the burden that arises as an early detection analysis of the onset of disease. However, Neural Network has slow training capabilities and can identify important attributes in the data resulting in a decrease in performance. Pearson correlation is good at handling data with mixed-type attributes and is good at measuring information between attributes and attributes with labels. With this, the purpose of this study will be to use the Pearson correlation method as a selection of features to improve neural network performance in diabetes detection and measure the extent of accuracy obtained from the method. The dataset used is diabetes data 130-US hospital UCI with a record number of 101767 and the number of attributes as many as 50 attributes. The results of this study found that Pearson correlation can improve neural network accuracy performance from 94.93% to 96.00%. As for the evaluation results on the AUC value increased from 0.8077 to 0.8246. Thus Pearson's Correlation algorithm can work well for feature selection on neural network methods and can provide solutions to improved diabetes detection accuracy

Bayesian neural network learning for repeat purchase modelling in direct marketing.

We focus on purchase incidence modelling for a European direct mail company. Response models based on statistical and neural network techniques are contrasted. The evidence framework of MacKay is used as an example implementation of Bayesian neural network learning, a method that is fairly robust with respect to problems typically encountered when implementing neural networks. The automatic relevance determination (ARD) method, an integrated feature of this framework, allows to assess the relative importance of the inputs. The basic response models use operationalisations of the traditionally discussed Recency, Frequency and Monetary (RFM) predictor categories. In a second experiment, the RFM response framework is enriched by the inclusion of other (non-RFM) customer profiling predictors. We contribute to the literature by providing experimental evidence that: (1) Bayesian neural networks offer a viable alternative for purchase incidence modelling; (2) a combined use of all three RFM predictor categories is advocated by the ARD method; (3) the inclusion of non-RFM variables allows to significantly augment the predictive power of the constructed RFM classifiers; (4) this rise is mainly attributed to the inclusion of customer\slash company interaction variables and a variable measuring whether a customer uses the credit facilities of the direct mailing company.Marketing; Companies; Models; Model; Problems; Neural networks; Networks; Variables; Credit;