Improving the performance of cascade correlation neural networks on multimodal functions

Intrinsic qualities of the cascade correlation algorithm make it a popular choice for many researchers wishing to utilize neural networks. Problems arise when the outputs required are highly multimodal over the input domain. The mean squared error of the approximation increases significantly as the number of modes increases. By applying ensembling and early stopping, we show that this error can be reduced by a factor of three. We also present a new technique based on subdivision that we call patchworking. When used in combination with early stopping and ensembling the mean improvement in error is over 10 in some cases

A study of early stopping, ensembling, and patchworking for cascade correlation neural networks

The constructive topology of the cascade correlation algorithm makes it a popular choice for many researchers wishing to utilize neural networks. However, for multimodal problems, the mean squared error of the approximation increases significantly as the number of modes increases. The components of this error will comprise both bias and variance and we provide formulae for estimating these values from mean squared errors alone. We achieve a near threefold reduction in the overall error by using early stopping and ensembling. Also described is a new subdivision technique that we call patchworking. Patchworking, when used in combination with early stopping and ensembling, can achieve an order of magnitude improvement in the error. Also presented is an approach for validating the quality of a neural network’s training, without the explicit use of a testing dataset

The Improvement of Neural Network Cascade-Correlation Algorithm and Its Application in Picking Seismic First Break

Neural Network is a kind of widely used seismic wave travel time auto-picking method. Most commercial software such as Promax often uses Back Propagation (BP) neural network. Here we introduce a cascade-correlation algorithm for constructing neural network. The algorithm’s convergence is faster than BP algorithm and can determine its own network architecture according to training samples, in addition, it can be able to expand network topology to learn new samples. The cascaded-correlation algorithm is improved. Different from the standard cascade-correlation algorithm, improved algorithm starts at an appropriate BP network architecture (exits hidden units), but the standard one’s initial network only includes input layer and output layer. In addition, in order to prevent weight-illgrowth, adding regularization term to the objective function when training candidate hidden units can decay weights. The simulation experiment demonstrates that improved cascade-correlation algorithm is faster convergence speed and stronger generalization ability. Analytically study five attributes, including instantaneous intensity ratio, amplitude, frequency, curve length ratio, adjacent seismic channel correlation. Intersection figure shows that these five attributes have distinctiveness of first break and stability. The neural network first break picking method of this paper has achieved good effect in testing actual seismic data.Key words: Neural network; Cascade-correlation algorithm; Picking seismic first brea

Forecasting creditworthiness in retail banking: a comparison of cascade correlation neural networks, CART and logistic regression scoring models

The preoccupation with modelling credit scoring systems including their relevance to forecasting and decision making in the financial sector has been with developed countries whilst developing countries have been largely neglected. The focus of our investigation is the Cameroonian commercial banking sector with implications for fellow members of the Banque des Etats de L’Afrique Centrale (BEAC) family which apply the same system. We investigate their currently used approaches to assessing personal loans and we construct appropriate scoring models. Three statistical modelling scoring techniques are applied, namely Logistic Regression (LR), Classification and Regression Tree (CART) and Cascade Correlation Neural Network (CCNN). To compare various scoring models’ performances we use Average Correct Classification (ACC) rates, error rates, ROC curve and GINI coefficient as evaluation criteria. The results demonstrate that a reduction in terms of forecasting power from 15.69% default cases under the current system, to 3.34% based on the best scoring model, namely CART can be achieved. The predictive capabilities of all three models are rated as at least very good using GINI coefficient; and rated excellent using the ROC curve for both CART and CCNN. It should be emphasised that in terms of prediction rate, CCNN is superior to the other techniques investigated in this paper. Also, a sensitivity analysis of the variables identifies borrower’s account functioning, previous occupation, guarantees, car ownership, and loan purpose as key variables in the forecasting and decision making process which are at the heart of overall credit policy

Probing many-body correlations using quantum-cascade correlation spectroscopy

The radiative quantum cascade, i.e. the consecutive emission of photons from a ladder of energy levels, is of fundamental importance in quantum optics. For example, the two-photon cascaded emission from calcium atoms was used in pioneering experiments to test Bell inequalities. In solid-state quantum optics, the radiative biexciton-exciton cascade has proven useful to generate entangled-photon pairs. More recently, correlations and entanglement of microwave photons emitted from a two-photon cascaded process were measured using superconducting circuits. All these experiments rely on the highly non-linear nature of the underlying energy ladder, enabling direct excitation and probing of specific single-photon transitions. Here, we use exciton polaritons to explore the cascaded emission of photons in the regime where individual transitions of the ladder are not resolved, a regime that has not been addressed so far. We excite a polariton quantum cascade by off-resonant laser excitation and probe the emitted luminescence using a combination of spectral filtering and correlation spectroscopy. Remarkably, the measured photon-photon correlations exhibit a strong dependence on the polariton energy, and therefore on the underlying polaritonic interaction strength, with clear signatures from two- and three-body Feshbach resonances. Our experiment establishes photon-cascade correlation spectroscopy as a highly sensitive tool to provide valuable information about the underlying quantum properties of novel semiconductor materials and we predict its usefulness in view of studying many-body quantum phenomena

Feature detection in satellite images using neural network technology

A feasibility study of automated classification of satellite images is described. Satellite images were characterized by the textures they contain. In particular, the detection of cloud textures was investigated. The method of second-order gray level statistics, using co-occurrence matrices, was applied to extract feature vectors from image segments. Neural network technology was employed to classify these feature vectors. The cascade-correlation architecture was successfully used as a classifier. The use of a Kohonen network was also investigated but this architecture could not reliably classify the feature vectors due to the complicated structure of the classification problem. The best results were obtained when data from different spectral bands were fused

Sample positioning in neutron diffraction experiments using a multi-material fiducial marker

An alternative sample positioning method is reported for use in conjunction with sample positioning and experiment planning software systems deployed on some neutron diffraction strain scanners. In this approach, the spherical fiducial markers and location trackers used with optical metrology hardware are replaced with a specifically designed multi-material fiducial marker that requires one diffraction measurement. In a blind setting, the marker position can be determined within an accuracy of ±164 µm with respect to the instrument gauge volume. The scheme is based on a pre-determined relationship that links the diffracted peak intensity to the absolute positioning of the fiducial marker with respect to the instrument gauge volume. Two methods for establishing the linking relationship are presented, respectively based on fitting multi-dimensional quadratic functions and a cross-correlation artificial neural network

Connectivity and Performance Tradeoffs in the Cascade Correlation Learning Architecture

The Cascade Correlation [1] is a very flexible, efficient and fast algorithm for supervised learning. It incrementally builds the network by adding hidden units one at a time, until the desired input/output mapping is achieved. It connects all the previously installed units to the new unit being added. Consequently, each new unit in effect adds a new layer and the fan–in of the hidden and output units keeps on increasing as more units get added. The resulting structure could be hard to implement in VLSI, because the connections are irregular and the fan-in is unbounded. Moreover, the depth or the propagation delay through the resulting network is directly proportional to the number of units and can be excessive. We have modified the algorithm to generate networks with restricted fan-in and small depth (propagation delay) by controlling the connectivity. Our results reveal that there is a tradeoff between connectivity and other performance attributes like depth, total number of independent parameters, learning time, etc. When the number of inputs or outputs is small relative to the size of the training set, a higher connectivity usually leads to faster learning, and fewer independent parameters, but it also results in unbounded fan-in and depth. Strictly layered architectures with restricted connectivity, on the other hand, need more epochs to learn and use more parameters, but generate more regular structures, with smaller, limited fan-in and significantly smaller depth (propagation delay), and may be better suited for VLSI implementations. When the number of inputs or outputs is not very small compared to the size of the training set, however, a strictly layered topology is seen to yield an overall better performance

Exploiting context when learning to classify

This paper addresses the problem of classifying observations when features are context-sensitive, specifically when the testing set involves a context that is different from the training set. The paper begins with a precise definition of the problem, then general strategies are presented for enhancing the performance of classification algorithms on this type of problem. These strategies are tested on two domains. The first domain is the diagnosis of gas turbine engines. The problem is to diagnose a faulty engine in one context, such as warm weather, when the fault has previously been seen only in another context, such as cold weather. The second domain is speech recognition. The problem is to recognize words spoken by a new speaker, not represented in the training set. For both domains, exploiting context results in substantially more accurate classification