Relationship between fault tolerance, generalization and the Vapnik-Chervonenkis (VC) dimension of feedforward ANNs

It is demonstrated that Fault tolerance, generalization and the Vapnik–Chervonenkis (VC) dimension (which is in turn related to the intrinsic capacity/complexity of the ANN) are inter-related attributes. It is well known that the generalization error if plotted as a function of the VC dimension h, exhibits a well defined minimum corresponding to an optimal value of h, say �opt. We show that if the VC dimension � of an ANN satisfies � � �opt (i.e., there is no excess capacity or redundancy), then Fault Tolerance and Generalization are mutually conflicting attributes. On the other hand, if ���opt (i.e., there is excess capacity or redundancy, then fault tolerance and generalization are mutually synergistic attributes. In other words, training methods geared toward improving the fault tolerance can also lead to better generalization and vice versa, only when there is excess capacity or redundancy. This is consistent with our previous results indicating that complete fault tolerance in ANNs requires a significant amount of redundancy. I