Exponential fuzzy associative memories with application in classification

Orientador: Marcos Eduardo Ribeiro do Valle MesquitaTese (doutorado) - Universidade Estadual de Campinas, Instituto de Matemática Estatística e Computação CientíficaResumo: Memórias associativas são modelos matemáticos cujo principal objetivo é armazenar e recuperar informação por associação. Tais modelos são projetados para armazenar um conjunto finito de pares, chamado conjunto das memórias fundamentais, e devem apresentar certa tolerância a ruído, isto é, serem capazes de recuperar uma certa informação armazenada mesmo a partir de uma versão incompleta ou corrompida de um item memorizado. As memórias associativas recorrentes por correlação (RCAMs, do inglês Recurrent Correlation Associative Memories), introduzidas por Chiueh e Goodman, apresentam grande capacidade de armazenamento e excelente tolerância a ruído. Todavia, as RCAMs são projetadas para armazenar e recuperar padrões bipolares. As memórias associativas recorrentes exponenciais fuzzy generalizadas (GRE-FAMs, do inglês Generalized Recurrent Exponential Fuzzy Associative Memories) podem ser vistas como uma versão generalizada das RCAMs capazes de armazenar e recuperar conjuntos fuzzy. Nesta tese, introduzimos as memórias associativas bidirecionais exponenciais fuzzy generalizadas (GEB-FAMs, do inglês Generalized Exponential Bidirectional Fuzzy Associative Memories), uma extensão das GRE-FAMs para o caso heteroassociativo. Uma vez que as GEB-FAMs são baseadas em uma medida de similaridade, realizamos um estudo de diversas medidas de similaridade da literatura, dentre elas as medidas de similaridade baseadas em cardinalidade e a medida de similaridade estrutural (SSIM). Além disso, mostramos que as GEB-FAMs exibem ótima capacidade de armazenamento e apresentamos uma caracterização da saída de um passo das GEB-FAMs quando um dos seus parâmetros tende a infinito. No entanto, em experimentos computacionais, bons resultados foram obtidos por um único passo da GEB-FAM com valores do parâmetro no intervalo [1,10]. Como a dinâmica das GEB-FAMs ainda não está totalmente compreendida, este fato motivou um estudo mais aprofundado das GEB-FAMs de passo único, modelos denominados memórias associativas fuzzy com núcleo (fuzzy-KAM, do inglês fuzzy Kernel Associative Memories). Interpretamos este modelo utilizando um núcleo fuzzy e propomos ajustar seu parâmetro utilizando o conceito de entropia. Apresentamos também duas abordagens para classificação de padrões usando as fuzzy-KAMs. Finalmente, descrevemos os experimentos computacionais realizados para avaliar o desempenho de tais abordagens em problemas de classificação e reconhecimento de faces. Na maioria dos experimentos realizados, em ambos os tipos de problemas, os classificadores definidos com base nas abordagens propostas obtiveram desempenho satisfatório e competitivo com os obtidos por outros modelos da literatura, o que mostra a versatilidade de tais abordagensAbstract: Associative memories are mathematical models whose main objective is to store and recall information by association. Such models are designed for the storage a finite set of pairs, called fundamental memory set, and they must present certain noise tolerance, that is, they should be able to retrieve a stored information even from an incomplete or corrupted version of a memorized item. The recurrent correlation associative memories (RCAMs), introduced by Chiueh and Goodman, present large storage capacity and excellent noise tolerance. However, RCAMs are designed to store and retrieve bipolar patterns. The generalized recurrent exponential fuzzy associative memories (GRE-FAMs) can be seen as a generalized version of RCAMs capable of storing and retrieving fuzzy sets. In this thesis, we introduce the generalized exponential bidirectional fuzzy associative memories (GEB-FAMs), an extension of GRE-FAMs to the heteroassociative case. Since GEB-FAMs are based on a similarity measure, we conducted a study of several measures from the literature, including the cardinality based similarity measure and the structural similarity index (SSIM). Furthermore, we show that GEB-FAMs exhibit optimal storage capacity and we present a characterization of the output of a single-step GEB-FAM when one of its parameters tends to infinity. However, in computational experiments, good results were obtained by a single-step GEB-FAM with parameter values in the interval [1,10]. As the dynamics of the GEB-FAMs is still not fully understood, this fact led to a more detailed study of the single-step GEB-FAMs, refered to as fuzzy kernel associative memories (fuzzy-KAMs). We interpret this model by using a fuzzy kernel and we propose to adjust its parameter by using the concept of entropy. Also, we present two approaches to pattern classification using the fuzzy-KAMs. Finally, we describe computational experiments used to evaluate the performance of such approaches in classification and face recognition problems. In most of the experiments performed, in both types of problems, the classifiers defined based on the proposed approaches obtained satisfactory and competitive performance with those obtained by other models from the literature, which shows the versatility of such approachesDoutoradoMatematica AplicadaDoutora em Matemática Aplicada2015/00745-1CAPESFAPES

Community detection with spiking neural networks for neuromorphic hardware

We present results related to the performance of an algorithm for community detection which incorporates event-driven computation. We define a mapping which takes a graph G to a system of spiking neurons. Using a fully connected spiking neuron system, with both inhibitory and excitatory synaptic connections, the firing patterns of neurons within the same community can be distinguished from firing patterns of neurons in different communities. On a random graph with 128 vertices and known community structure we show that by using binary decoding and a Hamming-distance based metric, individual communities can be identified from spike train similarities. Using bipolar decoding and finite rate thresholding, we verify that inhibitory connections prevent the spread of spiking patterns.Comment: Conference paper presented at ORNL Neuromorphic Workshop 2017, 7 pages, 6 figure

Proceedings of the Third International Workshop on Neural Networks and Fuzzy Logic, volume 1

Documented here are papers presented at the Neural Networks and Fuzzy Logic Workshop sponsored by the National Aeronautics and Space Administration and cosponsored by the University of Houston, Clear Lake. The workshop was held June 1-3, 1992 at the Lyndon B. Johnson Space Center in Houston, Texas. During the three days approximately 50 papers were presented. Technical topics addressed included adaptive systems; learning algorithms; network architectures; vision; robotics; neurobiological connections; speech recognition and synthesis; fuzzy set theory and application, control, and dynamics processing; space applications; fuzzy logic and neural network computers; approximate reasoning; and multiobject decision making

The 1991 3rd NASA Symposium on VLSI Design

Papers from the symposium are presented from the following sessions: (1) featured presentations 1; (2) very large scale integration (VLSI) circuit design; (3) VLSI architecture 1; (4) featured presentations 2; (5) neural networks; (6) VLSI architectures 2; (7) featured presentations 3; (8) verification 1; (9) analog design; (10) verification 2; (11) design innovations 1; (12) asynchronous design; and (13) design innovations 2

A study in the use of fuzzy logic in the management of an automotive heat engine / electric hybrid vehicle powertrain

This thesis addresses the problem of the instant-by-instant control of the powertrain of a hybrid heat engine/electric vehicle. In the absence of a prototype vehicle on which the work could be carried out the work has taken the form of computer simulation experiments. In order to develop the powertrain control strategies, a computer model of a conceptual hybrid vehicle is then developed, containing components from real, production and prototype vehicles. The use of this component based modelling approach allows the models to be validated by comparing their predictions with the performance of the real vehicles in which the components are used. The previous work conducted in the field of hybrid vehicle powertrain control is then reviewed. It is found that fuzzy logic could potentially provide a means of controlling the hybrid powertrain in a realistic manner, in which some of the disadvantages of previous hybrid powertrain control strategies could be overcome. The results of initial simulation experiments are then reported, finding that whilst the basic method appears to have the potential to successfully control the powertrain, there is a need for an adaptive fuzzy powertrain controller. A review is then presented of previous work conducted in the field of adaptive fuzzy control, finding that none of the reported adaptive fuzzy control methods are capable of being easily applied in the case of the hybrid powertrain. An adaptive fuzzy controller is then developed, whose rule modification strategy is specifically designed to work in the hybrid powertrain control problem. This initial adaptive powertrain controller is then modified to improve its ability to control the overall performance of a hybrid vehicle, whilst maintaining vehicle driveability. It is found that this controller is able to adapt to the different driving styles of individual vehicle users within the space of a few simulated urban journeys. Experiments are then performed in which improvements in the overall efficiency of the vehicle powertrain are investigated. It is found that significant improvements in the operation of the powertrain are impossible, due to some of the features of the vehicle model and constraints placed upon the control strategy. Conclusions are then drawn, for the work done in the field of hybrid vehicle powertrain control and, also, for the work done in adaptive methods of fuzzy control. The most significant contribution in the field of hybrid powertrain control is the development of a controller that can adapt to the habits of different users. The most significant contribution in the field of fuzzy control is the form of the basic hybrid powertrain controller and the use of small fuzzy controllers in the powertrain controller adaptation strategy

Fuzzy control and its application to a pH process

In the chemical industry, the control of pH is a well-known problem that presents difficulties due to the large variations in its process dynamics and the static nonlinearity between pH and concentration. pH control requires the application of advanced control techniques such as linear or nonlinear adaptive control methods. Unfortunately, adaptive controllers rely on a mathematical model of the process being controlled, the parameters being determined or modified in real time. Because of its characteristics, the pH control process is extremely difficult to model accurately. Fuzzy logic, which is derived from Zadeh's theory of fuzzy sets and algorithms, provides an effective means of capturing the approximate, inexact nature of the physical world. It can be used to convert a linguistic control strategy based on expert knowledge, into an automatic control strategy to control a system in the absence of an exact mathematical model. The work described in this thesis sets out to investigate the suitability of fuzzy techniques for the control of pH within a continuous flow titration process. Initially, a simple fuzzy development system was designed and used to produce an experimental fuzzy control program. A detailed study was then performed on the relationship between fuzzy decision table scaling factors and the control constants of a digital PI controller. Equation derived from this study were then confirmed experimentally using an analogue simulation of a first order plant. As a result of this work a novel method of tuning a fuzzy controller by adjusting its scaling factors, was derived. This technique was then used for the remainder of the work described in this thesis. The findings of the simulation studies were confirmed by an extensive series of experiments using a pH process pilot plant. The performance of the tunable fuzzy controller was compared with that of a conventional PI controller in response to step change in the set-point, at a number of pH levels. The results showed not only that the fuzzy controller could be easily adjusted to provided a wide range of operating characteristics, but also that the fuzzy controller was much better at controlling the highly non-linear pH process, than a conventional digital PI controller. The fuzzy controller achieved a shorter settling time, produced less over-shoot, and was less affected by contamination than the digital PI controller. One of the most important characteristics of the tunable fuzzy controller is its ability to implement a wide variety of control mechanisms simply by modifying one or two control variables. Thus the controller can be made to behave in a manner similar to that of a conventional PI controller, or with different parameter values, can imitate other forms of controller. One such mode of operation uses sliding mode control, with the fuzzy decision table main diagonal being used as the variable structure system (VSS) switching line. A theoretical explanation of this behavior, and its boundary conditions, are given within the text. While the work described within this thesis has concentrated on the use of fuzzy techniques in the control of continuous flow pH plants, the flexibility of the fuzzy control strategy described here, make it of interest in other areas. It is likely to be particularly useful in situations where high degrees of non-linearity make more conventional control methods ineffective

Using features for automated problem solving

We motivate and present an architecture for problem solving where an abstraction layer of "features" plays the key role in determining methods to apply. The system is presented in the context of theorem proving with Isabelle, and we demonstrate how this approach to encoding control knowledge is expressively different to other common techniques. We look closely at two areas where the feature layer may offer benefits to theorem proving — semi-automation and learning — and find strong evidence that in these particular domains, the approach shows compelling promise. The system includes a graphical theorem-proving user interface for Eclipse ProofGeneral and is available from the project web page, http://feasch.heneveld.org

Neuroengineering of Clustering Algorithms

Cluster analysis can be broadly divided into multivariate data visualization, clustering algorithms, and cluster validation. This dissertation contributes neural network-based techniques to perform all three unsupervised learning tasks. Particularly, the first paper provides a comprehensive review on adaptive resonance theory (ART) models for engineering applications and provides context for the four subsequent papers. These papers are devoted to enhancements of ART-based clustering algorithms from (a) a practical perspective by exploiting the visual assessment of cluster tendency (VAT) sorting algorithm as a preprocessor for ART offline training, thus mitigating ordering effects; and (b) an engineering perspective by designing a family of multi-criteria ART models: dual vigilance fuzzy ART and distributed dual vigilance fuzzy ART (both of which are capable of detecting complex cluster structures), merge ART (aggregates partitions and lessens ordering effects in online learning), and cluster validity index vigilance in fuzzy ART (features a robust vigilance parameter selection and alleviates ordering effects in offline learning). The sixth paper consists of enhancements to data visualization using self-organizing maps (SOMs) by depicting in the reduced dimension and topology-preserving SOM grid information-theoretic similarity measures between neighboring neurons. This visualization\u27s parameters are estimated using samples selected via a single-linkage procedure, thereby generating heatmaps that portray more homogeneous within-cluster similarities and crisper between-cluster boundaries. The seventh paper presents incremental cluster validity indices (iCVIs) realized by (a) incorporating existing formulations of online computations for clusters\u27 descriptors, or (b) modifying an existing ART-based model and incrementally updating local density counts between prototypes. Moreover, this last paper provides the first comprehensive comparison of iCVIs in the computational intelligence literature --Abstract, page iv

A study on identification of anomalies in fuzzy rule bases applied to the problem of estimation of risk of endometriosis

Orientador: Estevão Esmi LaureanoDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de Matemática Estatística e Computação CientíficaResumo: O trabalho propõe um sistema baseado em regras fuzzy para o problema de estimação de risco de endometriose. A endometriose é caracterizada pela presença de tecido endometrial fora da cavidade uterina. Isso acarreta em um processo inflamatório crônico, de forma que a paciente portadora da enfermidade sofre de diversos sintomas, dentre eles dores que chegam a ser incapacitantes e infertilidade. Destaca-se o tempo de diagnóstico da enfermidade que é em média maior que cinco anos. Esse atraso justifica o desenvolvimento de um modelo matemático que auxilie o especialista a encaminhar as pacientes para o procedimento padrão-ouro de diagnóstico: a laparoscopia com verificação histológica. Um sistema de base de regras fuzzy utilizando o método de Mamdani é proposto com a colaboração do Dr. Kleber Cursino de Andrade, um especialista que atua no CAISM (Centro de Atenção Integral à Saúde da Mulher), Campinas, Brazil. Após testes preliminares, é constatada a necessidade de se identificarem potenciais inconsistências na base de regras. São apresentados dois métodos para detecção de anomalias em uma base de regras fuzzy: um estático, baseado em uma medida de similaridade, e outro dinâmico, baseado em um método denominado "refletindo sobre as entradas", adequado a regras implicativas. É aplicado o método baseado em similaridade, por se tratar de uma base de regras conjuntiva. Dessa forma, são apontadas as regras potencialmente contraditórias do sistemaAbstract: This work purposes a fuzzy rule-based system for the problem of estimation of endometriosis risk. Endometriosis is defined as the presence of endometrial tissue outside the uterine cavity. This implies in a chronic inflamatory reaction, so that the disease carrier suffers from several symptoms, among them potencially disabling pains and infertility. The diagnosis time of the disease is highlighted and it is on average greater than five years. This delay justifies the development of a mathematical model able to help the expert to refer the pacients to the gold standard for the diagnosis: laparoscopy with histological verification. A fuzzy rule-based system using the Mamdani¿s method is purposed with the contribution of Dr. Kleber Cursino de Andrade, a specialist who works at CAISM (Integral Attention to Women¿s Health Center, in portuguese), Campinas, Brazil. After preliminaries tests, it is verified the necessity of identify potencial inconsistencies on the rule base. Two methods for anomalies detection in a fuzzy rule base are presented: one of them classified as static, based on a similarity measure, and the other one classified as dinamic, based on a method called "reflecting on the inputs" that is proper for implicative rules. The method based in similarity is applied, due to the rule base is conjunctive. Therefore, potencially contradictory rules of the system are detectedMestradoMatematica AplicadaMestre em Matemática Aplicada170715/2017-5CAPESCNP