Experimental Analysis of Input Weight Freezing in Constructive Neural Networks

An important research problem in constructive network algorithms is how to train the new network after the addition of a hidden unit. There are two ways to train the resultant network. One calls for freezing the input weights of the original network and training only those of the new unit, whereas the other approach allows complete retraining of the whole network. Some previous empirical analyses performed on the cascade-correlation architecture indicate that the effectiveness of freezing is different for different problem domains and hence is not conclusive. This paper describes a series of experiments with the single-hidden-layer network on a number of artificial pattern classification problems. We compare the performance of the network with and without input weight freezing, and against standard back-propagation. We also identify drawbacks with freezing and some directions for future work are discussed. Keywords: supervised learning, constructive network, cascade-correlation 1. Intr..