Heuristic Learning In Recurrent Neural Fuzzy Networks

A novel recurrent neural fuzzy network is proposed in this paper. The network model is composed by two structures: a fuzzy system and a neural network. The fuzzy system contains fuzzy neurons modeled with the aid of logic and and or operations processed via t-norms and s-norms. The neural network is composed by nonlinear elements placed in series with the previous logical elements. The network model implicitly encodes a set of if-then rules and its recurrent multilayered structure performs fuzzy inference. The topology induces a clear relationship between the network structure and an associated fuzzy rule-based system. In particular we explore this structure with an heuristic learning algorithm based on associative reinforcement learning and gradient search. These learning algorithms are associated to the fuzzy system and neural network, respectively. That is, output layer weights are adjusted via an error gradient method whereas a reward and punishment scheme updates the hidden layer weights. The recurrent fuzzy neural network is particularly suitable to model nonlinear dynamic systems and to learn sequences. Computational experiments with system identification problems show that the fuzzy neural models learned are simpler and that learning is faster than its counterparts.1302/04/156374Narendra, K.S., Parthasarathy, K., Identification and control of dynamical systems using neural networks (1990) IEEE Trans. on Neural Networks, 1 (1), p. 4Atiya, A.F., El-Shoura, S.M., Shaheen, S.I., El-Sherif, M.S., A comparison between neural-network forecasting techniques - Case study: River flow forecasting (1999) IEEE Trans. on Neural Networks, 10 (2), pp. 402-409. , MarchGalicki, M., Leistritz, L., Witte, H., Learning continuous trajectories in recurrent neural networks with time-dependent weights (1999) IEEE Trans. on Neural Networks, 10 (4), pp. 741-756. , JulySantos, E.P., Von Zuben, F.J., Efficient second-order learning algorithms for discrete-time recurrent neural networks (1999) Recurrent Neural Networks: Design and Applications, pp. 47-75. , L.R. Medsker and L.C. Jain, eds, CRC PressFunahashi, K.I., Nakamura, Y., Approximation of dynamical systems by continuous time recurrent neural networks (1993) Neural Networks, 6, pp. 801-806Jin, L., Pikiforuk, P.N., Gupta, M.M., Approximation capability of feedforward and recurrent neural networks (1995) Intelligent Control Systems: Concepts and Applications, , M.M. Gupta and N.K. Sinha, eds, IEEE PressWilliams, R.J., Zipser, D., A learning algorithm for continually running fully recurrent neural networks (1989) Neural Computing, (1), pp. 270-280Williams, R.J., Peng, T.M., An efficient gradient-based algorithm for on-line training of recurrent network trajectories (1990) Neural Computing, (2), pp. 490-501Werbos, P.J., Backpropagation through time: What it does and how to do it (1990) Proc. IEEE, 10 (78), pp. 1550-1560Lee, C.H., Teng, C.C., Identification and control of dynamic systems using recurrente fuzzy neural networks (2000) IEEE Trans. on Fuzzy Systems, 8 (4), pp. 3493-366. , AugustBuckley, A., Hayashi, Y., Fuzzy neural networks: A survey (1994) Fuzzy Sets and Systems, 66, pp. 41-49Lin, C.T., Lee, C.S., (1996) Neural Fuzzy Systems: A Neuro-fuzzy Synergism to Intelligent Systems, , Prentice Hall, Upper Saddle River, N.JLin, F.J., Hwang, W.J., Wai, R.J., A supervisory fuzzy neural network control system for tracking periodic inputs (1999) IEEE Trans. on Fuzzy Systems, 7, pp. 41-52. , FebruaryChen, Y.C., Teng, C.C., A model reference control structure using a fuzzy neural network (1995) Fuzzy Sets and Systems, 73, pp. 291-312Wang, L.X., (1997) A Course in Fuzzy Systems and Control, , Prentice-HallCastro, J., Fuzzy logic controllers are universal approximators (1995) IEEE Trans. on Systems, Man, and Cybernetics, 25 (4), pp. 629-635Wang, L.X., (1994) Adaptive Fuzzy Systems and Control, , Prentice-HallCastro, J., Delgado, M., Fuzzy systems with defuzzification are universal approximators (1996) IEEE Trans. on Systems, Man, and Cybernetics, 26 (1), pp. 149-152Blanco, A., Delgado, M., Pegalajar, M.C., Identification of fuzzy dynamic systems using Max-Min recurrent neural networks (2001) Fuzzy Sets and Systems, 122 (3), pp. 451-467. , SeptemberBallini, R., Scares, S., Gomide, F., A recurrent neurofuzzy network structure and learning procedure (2001) 10th IEEE International Conference on Fuzzy Systems- FUZZ-IEEE 2001, 3Caminhas, W., Tavares, H., Gomide, F., Pedrycz, W., Fuzzy set based neural networks: Structure, learning and application (1999) Journal of Advanced Computational Intelligence, 3 (3), pp. 151-157Pedrycz, W., Rocha, A., Fuzzy-set based models of neuron and knowledge-based networks (1998) IEEE Transactions on Fuzzy Systems, 4 (1), pp. 254-266Pedrycz, W., Gomide, F., (1998) An Introduction to Fuzzy Sets: Analysis and Design, , MIT Press, Cambridge, MABarto, A.G., Jordan, M.I., Gradient following without back-propagation in layered networks (1987) Proceedings of the IEEE First International Conference on Neural Networks, 2, pp. 629-636. , San DiegoNozaki, K., Ishibuchi, H., Tanaka, H., Trainable fuzzy classification systems based on fuzzy if-then rules (1994) Proc. of the Third IEEE International Conference on Fuzzy Systems, pp. 408-502Oliveira, M., Figueiredo, M., Gomide, F., A neurofuzzy approach for autonomous control (1994) Proc. of the Third IEEE International Conference on Fuzzy Systems, Neural Nets and Soft Computing, pp. 597-598Rumelhart, D., Hinton, G.E., Williams, R.J., Learning representations by back-propagating errors (1996) Nature (London), 323, pp. 533-536Barto, A.G., Anadan, P., Pattern recognizing stochastic learning automata (1985) IEEE Trans. on Systems, Man, and Cybernetics, 15, pp. 360-375Lee, C.H., Teng, C.C., A model reference control structure using a fuzzy neural network (1995) Fuzzy Sets and Systems, 73, pp. 291-31