Skip to main content
Article thumbnail
Location of Repository

Approximator networks and the principle of optimal information distribution

By Rüdiger W. Brause


It is well known that artificial neural nets can be used as approximators of any continous functions to any desired degree. Nevertheless, for a given application and a given network architecture the non-trivial task rests to determine the necessary number of neurons and the necessary accuracy (number of bits) per weight for a satisfactory operation. In this paper the problem is treated by an information theoretic approach. The values for the weights and thresholds in the approximator network are determined analytically. Furthermore, the accuracy of the weights and the number of neurons are seen as general system parameters which determine the the maximal output information (i.e. the approximation error) by the absolute amount and the relative distribution of information contained in the network. A new principle of optimal information distribution is proposed and the conditions for the optimal system parameters are derived. For the simple, instructive example of a linear approximation of a non-linear, quadratic function, the principle of optimal information distribution gives the the optimal system parameters, i.e. the number of neurons and the different resolutions of the variables

Topics: ddc:004
Year: 1991
OAI identifier:

Suggested articles


  1. (1989). H.White: Multilayer Feedforward Networks are Universal Approximators Neural Networks
  2. (1989). H.White: Universal approximation using feedforward networks with non-sigmoid hidden layer activation functions
  3. (1990). Jerry Gollub: Chaotic dynamics: an introduction;
  4. (1990). Networks and the Best Approximation Property;
  5. (1987). Nonlinear Signal Processing using Neural Networks: Prediction and System Modelling; Los Alamos preprint
  6. (1990). Optimal Information Distribution and Performance in Neighbourhood-conserving Maps for Robot Control;
  7. Performance and Storage Requirements of Topology-Conserving Maps for Robot Manipulator Control;
  8. (1949). W.Weaver: The Mathematical Theory of Information;

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.