Universal Approximation to Nonlinear Operators by Neural Networks with Arbitrary Activation Functions and Its Application to Dynamical Systems

Abstract

The purpose of this paper is to investigate neural network capability systematically. The main results are: (1) Every Tauber-Wiener function is qualified as an activation function in the hidden layer of a three-layered neural network; (2) For a continuous function to be a Tauber-Wiener function, the necessary and sufficient condition is that it is not a polynomial; (3) The capability of approximating nonlinear functionals defined on some Banach space and nonlinear operators has been shown, which implies that (4) we can use neural network computation to approximate the output as a whole (not at a fixed point) of a dynamical system. Key words: Approximation theory, neural networks, dynamical systems, compact set, functional, operator. 1 The author is with the Department of Mathematics, Fudan University, Shanghai, P.R.China. 2 The author was with the Department of Electrical Engineering, University of Notre Dame, Notre Dame, Indiana 46556, USA. He is now with VLSI Libraries, Inc., 183..