Neurofuzzy approach for nonlinear dynamical systems modeling

Orientadores: Fernando Antonio Campos Gomide, Pyramo Pires Costa JuniorTese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenharia Eletrica e de ComputaçãoResumo: Este trabalho propõe um procedimento sistemático para obtenção de modelos de sistemas dinâmicos não-lineares complexos utilizando redes neurais nebulosas. As redes neurais nebulosas aplicadas em modelagem são capazes de extrair conhecimento de dados entrada/saída e representar este conhecimento na forma de regras nebulosas do tipo se-então, gerando modelos lingüísticos convenientes para compreensão humana. Duas novas classes de redes neurais nebulosas são propostas a partir de generalizações dos neurônios lógicos AND e OR. Estas generalizações, denominadas unineurons e nullneurons, implementam, além da plasticidade sináptica, outra importante característica dos neurônios biológicos, a plasticidade neuronal. Desta forma, os neurônios propostos neste trabalho são capazes de modificar parâmetros internos em resposta à alterações, permitindo que unineurons e nullneurons variem individualmente de um neurônio AND para um neurônio OR (e vice-e-versa), dependendo da necessidade do problema. Conseqüentemente, uma rede neural nebulosa composta por unineurons e nullneurons é mais geral do que as redes neurais nebulosas similares sugeridas na literatura. Além da introdução de redes neurais com unineurons e nullneurons, um novo algoritmo de treinamento para obtenção de modelos nebulosos de sistemas dinâmicos é proposto utilizando aprendizado participativo. Neste algoritmo, uma nova informação fornecida à rede por meio de um dado entrada/saída é comparada com o conhecimento que já se tem a respeito do sistema. A nova informação só tem influência na atualização do conhecimento se não entrar em conflito com o conhecimento adquirido anteriormente. Como conseqüência, redes neurais nebulosas que utilizam este novo algoritmo de treinamento são mais robustas a dados de treinamento com valores que correspondem a comportamentos anômalos ou mesmo a erros durante a obtenção destes dados. As abordagens propostas foram utilizadas para desenvolver modelos para previsão de séries temporais e modelagem térmica de transformadores de potência. Os resultados experimentais mostram que os modelos aqui propostos são mais robustos e apresentam os melhores desempenhos, tanto em termos de precisão quanto em termos de custos computacionais, quando comparados com abordagens alternativas sugeridas na literaturaAbstract: This work suggests a systematic procedure to develop models of complex nonlinear dynamical systems using neural fuzzy networks. The neural fuzzy networks are able to extract knowledge from input/output data and to encode it explicitly in the form of if-then rules. Therefore, linguistic models are obtained in a form suitable for human understanding. Two new classes of fuzzy neurons are introduced to generalize AND and OR logic neurons. These generalized login neurons, called unineurons and nullneurons, provide a mechanism to implement synaptic plasticity and an important characteristic of biological neurons, the neuronal plasticity. Unineurons and nullneurons achieve synaptic and neuronal plasticity modifying their internal parameters in response to external changes. Thus, unineurons and nullneurons may individually vary from a AND neuron to a OR neuron (and vice-versa), depending upon the necessity of the modeling task. Neural fuzzy networks constructed with unineurons and nullneurons are more general than similar fuzzy neural approaches suggested in literature. Training algorithms for neural fuzzy networks with unineurons and nullneurons are also studied. In particular, a new training algorithm based on the participatory learning is introduced to develop fuzzy models of dynamical systems. In the participatory learning algorithm, a new information brought to the network through an input/output data is first compared with the knowledge that the network already has about the model. The new information influences the update of the knowledge only if it does not conflict with the current knowledge. As a result, neural fuzzy networks trained with participatory learning show greater robustness to training data with anomalous values than their counterparts. The neural fuzzy network and training algorithms suggested herein are used to develop time series forecast models and thermal models of power transformers. Experimental results show that the models proposed here are more robust and perform best in terms of accuracy and computational costs when compared against alternative approaches suggested in the literatureDoutoradoAutomaçãoDoutor em Engenharia Elétric

Recurrent Neural Approaches For Power Transformers Thermal Modeling

This paper introduces approaches for power transformer thermal modeling based on two conceptually different recurrent neural networks. The first is the Elman recurrent neural network model whereas the second is a recurrent neural fuzzy network constructed with fuzzy neurons based on triangular norms. These two models are used to model the thermal behavior of power transformers using data reported in literature. The paper details the neural modeling approaches and discusses their main capabilities and properties. Comparisons with the classic deterministic model and static neural modeling approaches are also reported. Computational experiments suggest that the recurrent neural fuzzy-based modeling approach outperforms the remaining models from both, computational processing speed and robustness point of view. © Springer-Verlag Berlin Heidelberg 2005.3776 LNCS287293Swift, G.W., Adaptative transformer thermal overload protection (2001) IEEE Transactions on Power Delivery, 16 (4), pp. 516-521Galdi, V., Ippolito, L., Piccolo, A., Vaccaro, A., Neural diagnostic system for transformer thermal overload protection (2000) IEE Proceedings of Electric Power Applications, 147 (5), pp. 415-421Narendra, K.S., Parthasarathy, K., Identification and control of dynamic systems using neural networks (1990) IEEE Transactions on Neural Networks, 1, pp. 4-27Haykin, S., (1998) Neural Networks: A Comprehensive Foundation, , Prentice Hall, NJ-USA, ed. 2Jang, J.-S.R., ANFIS: Adaptative-network-based fuzzy inference system (1993) IEEE Transactions on System, Man, and Cybernetics, 23 (3), pp. 665-685Elman, J., Finding structure in time (1990) Cognitive Science, 14, pp. 179-211Ballini, R., Gomide, F., Learning in recurrent, hybrid neurofuzzy networks (2002) IEEE International Conference on Fuzzy Systems, pp. 785-79

A Fast Learning Algorithm For Uninorm-based Fuzzy Neural Networks

This paper suggests a fast learning algorithm for weighted uninorm-based neural networks. Fuzzy neural networks are models capable to approximate functions with high accuracy and to generate transparent models through extraction of linguistic information from the resulting topology. A fuzzy neural network model based on weighted uninorms has been developed recently. It was shown that this model approximates any continuous real function on a compact subset. In this paper we introduce a fast learning algorithm for this class of fuzzy neural networks based on ideas from extreme learning machine. The algorithm is detailed and computational experiments reported to illustrate the accuracy and time efficiency of the learning approach. The results show that neural fuzzy model is accurate and learning speed is as good as or faster than alternative neural network models. © 2012 IEEE.Minist. Commun. Inf. Technol. Republic AzerbaijanPedrycz, W., Fuzzy Neural Networks and Neurocomputations (1993) Fuzzy Sets and Systems, 56 (1), pp. 1-28. , MAY 25Caminhas, W., Tavares, H., Gomide, F., Pedrycz, W., Fuzzy sets based neural networks: Structure, learning and applications (1999) Journal of Advanced Computational Intelligence, 3 (3), pp. 151-157Gomide, F., Pedrycz, W., (2007) Fuzzy Systems Engineering: Toward Human-Centric Computing, , NJ, USA: Wiley InterscienceBallini, R., Gomide, F., Learning in recurrent, hybrid neurofuzzy networks (2002) IEEE International Conference on Fuzzy Systems, pp. 785-791Hell, M., Costa, P., Gomide, F., Participatory learning in power transformers thermal modeling (2008) IEEE Transactions on Power Delivery, 23 (4), pp. 2058-2067. , OctGobi, A.E., Pedrycz, D., Logic minimization as an efficient means of fuzzy structure discovery (2008) IEEE Transactions on Fuzzy Systems, 16 (3), pp. 553-566. , JUNPedrycz, W., Logic-based fuzzy neurocomputing with unineurons (2006) IEEE Transactions on Fuzzy Systems, 14 (6), pp. 860-873. , DECPedrycz, W., Hirota, K., Uninorm-based logic neurons as adaptive and interpretable processing constructs (2007) Soft Computing, 11 (1), pp. 41-52. , JANHell, M., Gomide, F., Ballini, R., Costa, P., Uninetworks in time series forecasting (2009) NAFIPS 2009. Annual Meeting of the North American Fuzzy Information Processing Society, 2009, pp. 1-6. , juneLemos, A., Caminhas, W., Gomide, F., New uninorm-based neuron model and fuzzy neural networks (2010) 2010 Annual Meeting of the North American Fuzzy Information Processing Society (NAFIPS), pp. 1-6Lemos, A., Kreinovich, V., Caminhas, W., Gomide, F., Universal approximation with uninorm-based fuzzy neural networks (2011) 2011 Annual Meeting of the North American Fuzzy Information Processing Society (NAFIPS), pp. 1-6. , marchYager, R., Rybalov, A., Uninorm aggregation operators (1996) Fuzzy Sets and Systems, 80 (1), pp. 111-120. , MAY 27Calvo, T., Baets, B.D., Fodor, J., The functional equations of frank and alsina for uninorms and nullnorms (2001) Fuzzy Sets and Systems, 120 (3), pp. 385-394Herrera, F., Lozano, M., Verdegay, J.L., Tackling real-coded genetic algorithms: Operators and tools for behavioural analysis (1998) Artif. Intell. Rev., 12 (4), pp. 265-319Huang, G.-B., Zhu, Q.-Y., Siew, C.-K., Extreme learning machine: A new learning scheme of feedforward neural networks (2004) 2004 IEEE International Joint Conference on Neural Networks (IJCNN), 2, pp. 985-990. , july vol.2Huang, G.-B., Chen, L., Siew, C.-K., Universal approximation using incremental constructive feedforward networks with random hidden nodes (2006) Neural Networks, IEEE Transactions on, 17 (4), pp. 879-892. , julyYager, R., Uninorms in fuzzy systems modeling (2001) Fuzzy Sets and Systems, 122 (1), pp. 167-175. , AUG 16Huang, G.-B., Siew, C.-K., Extreme learning machine with randomly assigned rbf kernels (2005) International Journal of Information Technology, 11 (1), pp. 16-24Montesino-Pouzols, F., Lendasse, A., Evolving fuzzy optimally pruned extreme learning machine for regression problems (2010) Evolving Systems, 1 (1), pp. 43-58. , AugustBartlett, P., The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the network (1998) IEEE Transactions on Information Theory, 44 (2), pp. 525-536. , marSerre, D., (2002) Matrices: Theory and Applications, , New York, US: Springer- VerlagBox, G.E.P., Jenkins, G., (1990) Time Series Analysis, Forecasting and Control, , Holden-Day, IncorporatedRiedmiller, M., Braun, H., A direct adaptive method for faster backpropagation learning: The rprop algorithm (1993) IEEE International Conference on Neural Networks, 1993, 1, pp. 586-591Jang, J., ANFIS - Adaptive-Network-Based Fuzzy Inference System (1993) IEEE Transactions on Systems Man and Cybernetics, 23 (3

Focusing On Interpretability And Accuracy Of A Genetic Fuzzy System

This research work presents a new approach for fuzzy system building taking into account the accuracy and interpretability of the system. One difficulty in the handling of high-dimensional problems by fuzzy rule-based systems is the exponential increase in the number of rules and in the number of conditions in the antecedent part of the rule. Thus, as first step of the proposed approach we apply a feature selection process in order to exclude irrelevant variables. Besides that, dimensionality reduction generally promotes the accuracy and comprehensibility of the system. After that, a genetic algorithm is used for deriving short and comprehensible fuzzy rules. Finally another genetic algorithm is used for optimizing the rule base obtained in the last step, excluding unnecessary and redundant rules. The fitness function of the algorithms consider both accuracy and interpretability of the fuzzy model and the use of "don't care" condition allows to generate more comprehensible with high generalization capacity. The application domain is multidimensional fuzzy pattern classification. By computational simulation in some well-know datasets, we can see that the proposed approach is able to generate compact fuzzy rule bases with high classification ability. When compared to other fuzzy building method reported in the literature, our proposed method presented a good performance. © 2005 IEEE.696701Cordón, O., Herrera, F., Gomide, F., Hoffmann, F., Magdalena, L., Ten years of genetic-fuzzy systems: A current framework and new trends (2001) Proceedings of Joint 9th IFSA World Congress and 20th NAFIPS Internation Conference, pp. 1241-1246. , Vancouver - CanadaCordón, O., Herrera, F., Hybridizing genetic algorithms with sharing scheme and evolution strategies for designing approximate fuzzy rule-based systems (2001) Fuzzy Sets and Systems, 118, pp. 235-255Nauck, D., Cruse, R., A neuro-fuzzy method to learn fuzzy classification rules from data (1997) Fuzzy Sets and Systems, 89, pp. 277-288Liao, T.W., Celmins, A.K., Hammell, R.J., A fuzzy c-means variant for the generation of fuzzy term sets (1997) Fuzzy Sets and Systems, 135, pp. 241-257Nomura, H., Hayashi, L., Wakami, N., A learning method of fuzzy inference rules by descent method (1992) Proceedings of the 1st IEEE International Conference on Fuzzy Systems, pp. 203-210. , San Diego - USACordón, O., Herrera, F., Hoffmann, F., Magdalena, L., Recent advances in genetic fuzzy systems (2001) Journal of Information Sciences, 136, pp. 1-5Ishibuchi, H., Murata, T., Turksen, I.B., Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems (1997) Fuzzy Sets and Systems, 89, pp. 134-150Yuan, Y., Zhuang, H., A genetic algorithm for generating fuzzy classification rules (1996) Fuzzy Sets and Systems, 84 (4), pp. 1-19Ishibuchi, H., Nakashima, T., Murata, T., Performance evaluation of fuzzy classifier systems of multidimensional Pattern Classification Problems (1999) IEEE Transactions on Fuzzy Systems, 29, pp. 601-618Hoffman, F., Baesens, B., Martens, J., Put, F., Vanthienen, J., Comparing a genetic fuzzy and a neurofuzzy classifier for credit scoring (2002) International Journal of Intelligent Systems, 17 (11), pp. 1067-1083Castro, P.A.D., Pires, M.G., Camargo, H.A., Aprendizado e seleção de regras nebulosas usando algoritmos genéticos (2003) VI Simpósio Brasileiro de Automação Inteligente, pp. 970-975Castro, P.A.D., Camargo, H.A., Learning and optimization of fuzzy rule base by means of self adaptive genetic algorithms (2004) IEEE International Conference on Fuzzy Systems, , Budapest, HungaryCamargo, H.A., Pires, M.G., Castro, P.A.D., Genetic design of fuzzy knowledge bases - A study of different approaches (2004) 23rd IEEE International Conference of NAFIPS, 2, pp. 954-959. , Alberta,CanadCastro, P.A.D., Camargo, H.A., A study of the impact of reasoning methods on the genetic learning and optimization of fuzzy rule bases (2004) Proc. of XVII Brazilian Symposium on Artificial Intelligence - (SBIA), pp. 414-423. , So Luis, Maranho, BrazilGuillaume, S., Designing fuzzy inference systems from data: An interpretability oriented review (2001) IEEE Transactions on Fuzzy Systems, 9 (3), pp. 426-443Castellano, G., Fanelli, A.M., Gentile, E., Roselli, T., A GA-based approach to optimization of fuzzy models learned from data (2002) GECCO-2002 Program, pp. 5-8. , NewYorkJimnez, F., Gmez-Skarmeta, A.F., Roubos, H., Babuska, R., Accurate, transparent, and compact fuzzy models for function approximation and dynamic modeling through multi-objective evolutionary optimization (2001) First Internat. Conf. on Evolutionary Multi-criterion Optimization, pp. 653-667Jin, Y., Fuzzy modeling of high-dimensional systems: Complexity reduction and interpretability improvement (2000) IEEE Trans. Fuzzy Systems, 8 (2), pp. 212-221Jin, Y., Sendhoff, B., Extracting interpretable fuzzy rules from RBF networks (2003) Neural Process. Lett., 17 (2), pp. 149-164Rojas, I., Pomares, H., Ortega, J., Prieto, A., Self-organized fuzzy system generation from training examples (2000) IEEE Trans. on Fuzzy Systems, 8 (1), pp. 23-26Roubos, H., Setnes, M., GA-fuzzy modeling and classication: Complexity and performance (2000) IEEE Trans. on Fuzzy Systems, 8 (5), pp. 509-522Wang, L., Yen, J., Exacting fuzzy rules for system modeling using a hybrid of genetic algorithms and Kalman filter (1999) Fuzzy Sets and Systems, 101, pp. 353-362Castro, P.A.D., Santoro, D.M., Camargo, H.A., Nicoletti, M.C., Improving a Pittsburgh learnt fuzzy rule base using feature subset selection (2004) 4th International Conference on Hybrid Intelligent Systems, , Kitakyushu, JapanKononenko, I., Estimating attributes: Analysis and extension of Relief (1994) Proceedings of European Conference on Machine Learning, pp. 171-182Smith, S.F., (1980) A Learning System Based on Genetic Adaptive Algorithms, , Ph.D. thesis, University of PittsburghGonzalez, A., Herrera, F., Multi-stage genetic fuzzy systems based on the iterative rule learning approach (1997) Mathware and Soft Computing, 4 (3), pp. 233-249Ballini, R., Gomide, F., Learning in recurrent, hybrid neurofuzzy networks (2002) Proceedings of the 2002 IEEE International Conference on Fuzzy Systems, 1, pp. 785-790. , Honolulu, HI, USABezdek, J.C., (1981) Pattern Recognition with Fuzzy Objective Function Algorithms, , Plenum Press, New YorkVan Berg, J.D., Kaymak, U., Van Den Bergh, W.-M., Fuzzy classification using probability-based rule weighting (2002) Proceedings of the 11th IEEE International Conference on Fuzzy Systems, , Hawaii - USACordón, O., Del Jesus, M.J., Herrera, F., A proposal on reasoning methods in fuzzy rule-based classification systems (2001) International Journal of Approximate Reasoning, 20, pp. 21-45Klir, G., Yuan, B., (1995) Fuzzy Sets and Fuzzy Logic - Theory and Applications, , Prentice-HallBlake, C.L., Merz, C.J., (1998) UCI Repository of Machine Learning Databases, , http://www.ics.uci.edu/mlearn/MLRepository.html, Irvine, CA: University of California, Department of Information and Computer ScienceCastillo, L., Gonzalez, A., Perez, P., Including a simplicity criterion in the selection of the best rule in a genetic fuzzy learning algorithm (2001) Fuzzy Sets and Systems, 120 (2), pp. 309-32