Mutual information neuro-evolutionary system (MINES) presents a novel self-governing approach to
determine the optimal quantity and connectivity of the hidden layer of a three layer feed-forward neural
network founded on theoretical and practical basis. The system is a combination of a feed-forward neural
network, back-propagation algorithm, genetic algorithm, mutual information and clustering. Back-propagation
is used for parameter learning to reduce the system’s error; while mutual information aides
back-propagation to follow an effective path in the weight space. A genetic algorithm changes the incoming
synaptic connections of the hidden nodes, based on the fitness provided by the mutual information
from the error space to the hidden layer, to perform structural learning. Mutual information determines
the appropriate synapses, connecting the hidden nodes to the input layer; however, in effect it also links
the back-propagation to the genetic algorithm. Weight clustering is applied to reduce hidden nodes having
similar functionality; i.e. those possessing same connectivity patterns and close Euclidean angle in
the weight space. Finally, the performance of the system is assessed on two theoretical and one empirical
problems. A nonlinear polynomial regression problem and the well known two-spiral classification task
are used to evaluate the theoretical performance of the system. Forecasting daily crude oil prices are
conducted to observe the performance of MINES on a real world application
