A study of early stopping, ensembling, and patchworking for cascade correlation neural networks

The constructive topology of the cascade correlation algorithm makes it a popular choice for many researchers wishing to utilize neural networks. However, for multimodal problems, the mean squared error of the approximation increases significantly as the number of modes increases. The components of this error will comprise both bias and variance and we provide formulae for estimating these values from mean squared errors alone. We achieve a near threefold reduction in the overall error by using early stopping and ensembling. Also described is a new subdivision technique that we call patchworking. Patchworking, when used in combination with early stopping and ensembling, can achieve an order of magnitude improvement in the error. Also presented is an approach for validating the quality of a neural network’s training, without the explicit use of a testing dataset

Improving the performance of cascade correlation neural networks on multimodal functions

Intrinsic qualities of the cascade correlation algorithm make it a popular choice for many researchers wishing to utilize neural networks. Problems arise when the outputs required are highly multimodal over the input domain. The mean squared error of the approximation increases significantly as the number of modes increases. By applying ensembling and early stopping, we show that this error can be reduced by a factor of three. We also present a new technique based on subdivision that we call patchworking. When used in combination with early stopping and ensembling the mean improvement in error is over 10 in some cases

Feature detection in satellite images using neural network technology

A feasibility study of automated classification of satellite images is described. Satellite images were characterized by the textures they contain. In particular, the detection of cloud textures was investigated. The method of second-order gray level statistics, using co-occurrence matrices, was applied to extract feature vectors from image segments. Neural network technology was employed to classify these feature vectors. The cascade-correlation architecture was successfully used as a classifier. The use of a Kohonen network was also investigated but this architecture could not reliably classify the feature vectors due to the complicated structure of the classification problem. The best results were obtained when data from different spectral bands were fused

Overfitting control inside cascade correlation neural networks applied to electricity contract price prediction

Prediction of electricity prices is considered a difficult task due to the number and complexity of factors that influence their performance, and their relationships. Neural networks cascade correlation - CASCOR allows to do a constructive learning and it captures better the characteristics of the data; however, it has a high tendency to overfitting. To control overfitting in some areas regularization techniques are used. However, in the literature there are no studies that: i) use regularization techniques to control overfitting in CASCOR networks, ii) use CASCOR networks in predicting of electrical series iii) compare the performance with tra­ditional neural networks or statistical models. The aim of this paper is to model and predict the behavior of the price series of electricity contracts in Colombia, using CASCOR networks and controlling the overfitting by regularization techniques.La predicción de precios de electricidad es considerada una tarea difí­cil debido a la cantidad y complejidad de los factores que influyen en su representación, y sus relaciones. Las redes neuronales tipo cascada correlación –CASCOR– permiten, realizar un aprendizaje constructivo, capturando mejor las características de los datos; sin embargo, presentan una alta tendencia al sobreajuste. Para el control del sobreajuste en algunos ámbitos se usan técnicas de regularización. No obstante, en la literatura no existen estudios que: i) Utilicen técnicas de regularización para el control de sobreajuste en redes CASCOR; ii) Usen redes CASCOR en la predicción de series de electricidad; iii) comparen el desempeño con redes neuronales tradicionales o modelos estadísticos. El objetivo de este artículo es modelar y predecir el comportamiento de la serie de precios de contratos de electricidad en Colombia, usando redes CASCOR y con­trolando el sobreajuste con técnicas de regularización