Approximation properties of the neuro-fuzzy minimum function

The integration of fuzzy logic systems and neural networks in data driven nonlinear modeling applications has generally been limited to functions based upon the multiplicative fuzzy implication rule for theoretical and computational reasons. We derive a universal approximation result for the minimum fuzzy implication rule as well as a differentiable substitute function that allows fast optimization and function approximation with neuro-fuzzy networks. --Fuzzy Logic,Neural Networks,Nonlinear Modeling,Optimization

Bandler-Kohout Subproduct with Yager’s Families of Fuzzy Implications: A Comprehensive Study

Approximate reasoning schemes involving fuzzy sets are one of the best known applications of fuzzy logic in the wider sense. Fuzzy Inference Systems (FIS) or Fuzzy Inference Mechanisms (FIM) have many degrees of freedom, viz., the underlying fuzzy partition of the input and output spaces, the fuzzy logic operations employed, the fuzzification and defuzzification mechanism used, etc. This freedom gives rise to a variety of FIS with differing capabilities

Noncommutative Logic Systems with Applications in Management and Engineering

Zadeh's (min-max, standard) fuzzy logic and various other logics are commutative, but natural language has nuances suggesting the premises are not equal, with premises contributing to the conclusion according to their prominency. Therefore, we suggest variants of salience-based, noncommutative and non-associative fuzzy logic (prominence logic) that may better model natural language and reasoning when using linguistic variables. Noncommutative fuzzy logics have several theoretical and applicative motivations to be used as models for human inference and decision making processes. Among others, asymmetric relations in economy and management, such as buyer-seller, provider-user, and employer-employee are noncommutative relations and induce noncommutative logic operations between premises or conclusions. A class of noncommutative fuzzy logic operators is introduced and fuzzy logic systems based on the corresponding noncommutative logics are described and analyzed. The prominence of the operators in the noncommutative operations is conventionally assumed to be determined by their precedence. Specific versions of noncommutative logics in the class of the salience-based, noncommutative logics are discussed. We show how fuzzy logic systems may be built based on these types of logics. Compared with classic fuzzy systems, the noncommutative fuzzy logic systems have improved performances in modeling problems, including the modeling of economic and social processes, and offer more flexibility in approximation and control. Applications discussed include management and engineering problems and issues in the field of firms’ ethics or ethics of AI algorithms

Mathematical Neural Networks

Financial support from the Spanish Ministry of Universities. "Disruptive group decision making systems in fuzzy context: Applications in smart energy and people analytics" (PID2019-103880RB-I00). Main Investigator: Enrique Herrera Viedma, and Junta de Andalucia. "Excellence Groups" (P12.SEJ.2463) and Junta de Andalucia (SEJ340) are gratefully acknowledged. Research partially supported by the "Maria de Maeztu" Excellence Unit IMAG, reference CEX2020001105-M, funded by MCIN/AEI/10.13039/501100011033/.ANNs succeed in several tasks for real scenarios due to their high learning abilities. This paper focuses on theoretical aspects of ANNs to enhance the capacity of implementing those modifications that make ANNs absorb the defining features of each scenario. This work may be also encompassed within the trend devoted to providing mathematical explanations of ANN performance, with special attention to activation functions. The base algorithm has been mathematically decoded to analyse the required features of activation functions regarding their impact on the training process and on the applicability of the Universal Approximation Theorem. Particularly, significant new results to identify those activation functions which undergo some usual failings (gradient preserving) are presented here. This is the first paper—to the best of the author’s knowledge—that stresses the role of injectivity for activation functions, which has received scant attention in literature but has great incidence on the ANN performance. In this line, a characterization of injective activation functions has been provided related to monotonic functions which satisfy the classical contractive condition as a particular case of Lipschitz functions. A summary table on these is also provided, targeted at documenting how to select the best activation function for each situation.Spanish Government PID2019-103880RB-I00Junta de Andalucía P12.SEJ.2463 SEJ340"Maria de Maeztu" Excellence Unit IMAG - MCIN/AEI CEX2020001105-

A New Optimization Algorithm for Single Hidden Layer Feedforward Neural Networks

Feedforward neural networks are the most commonly used function approximation techniques in neural networks. By the universal approximation theorem, it is clear that a single-hidden layer feedforward neural network (FNN) is sufficient to approximate the corresponding desired outputs arbitrarily close. Some researchers use genetic algorithms (GAs) to explore the global optimal solution of the FNN structure. However, it is rather time consuming to use GA for the training of FNN. In this paper, we propose a new optimization algorithm for a single-hidden layer FNN. The method is based on the convex combination algorithm for massaging information in the hidden layer. In fact, this technique explores a continuum idea which combines the classic mutation and crossover strategies in GA together. The proposed method has the advantage over GA which requires a lot of preprocessing works in breaking down the data into a sequence of binary codes before learning or mutation can apply. Also, we set up a new error function to measure the performance of the FNN and obtain the optimal choice of the connection weights and thus the nonlinear optimization problem can be solved directly. Several computational experiments are used to illustrate the proposed algorithm, which has good exploration and exploitation capabilities in search of the optimal weight for single hidden layer FNNs

Value Function Estimation in Optimal Control via Takagi-Sugeno Models and Linear Programming

[ES] La presente Tesis emplea técnicas de programación dinámica y aprendizaje por refuerzo para el control de sistemas no lineales en espacios discretos y continuos. Inicialmente se realiza una revisión de los conceptos básicos de programación dinámica y aprendizaje por refuerzo para sistemas con un número finito de estados. Se analiza la extensión de estas técnicas mediante el uso de funciones de aproximación que permiten ampliar su aplicabilidad a sistemas con un gran número de estados o sistemas continuos. Las contribuciones de la Tesis son: -Se presenta una metodología que combina identificación y ajuste de la función Q, que incluye la identificación de un modelo Takagi-Sugeno, el cálculo de controladores subóptimos a partir de desigualdades matriciales lineales y el consiguiente ajuste basado en datos de la función Q a través de una optimización monotónica. -Se propone una metodología para el aprendizaje de controladores utilizando programación dinámica aproximada a través de programación lineal. La metodología hace que ADP-LP funcione en aplicaciones prácticas de control con estados y acciones continuos. La metodología propuesta estima una cota inferior y superior de la función de valor óptima a través de aproximadores funcionales. Se establecen pautas para los datos y la regularización de regresores con el fin de obtener resultados satisfactorios evitando soluciones no acotadas o mal condicionadas. -Se plantea una metodología bajo el enfoque de programación lineal aplicada a programación dinámica aproximada para obtener una mejor aproximación de la función de valor óptima en una determinada región del espacio de estados. La metodología propone aprender gradualmente una política utilizando datos disponibles sólo en la región de exploración. La exploración incrementa progresivamente la región de aprendizaje hasta obtener una política convergida.[CA] La present Tesi empra tècniques de programació dinàmica i aprenentatge per reforç per al control de sistemes no lineals en espais discrets i continus. Inicialment es realitza una revisió dels conceptes bàsics de programació dinàmica i aprenentatge per reforç per a sistemes amb un nombre finit d'estats. S'analitza l'extensió d'aquestes tècniques mitjançant l'ús de funcions d'aproximació que permeten ampliar la seua aplicabilitat a sistemes amb un gran nombre d'estats o sistemes continus. Les contribucions de la Tesi són: -Es presenta una metodologia que combina identificació i ajust de la funció Q, que inclou la identificació d'un model Takagi-Sugeno, el càlcul de controladors subòptims a partir de desigualtats matricials lineals i el consegüent ajust basat en dades de la funció Q a través d'una optimització monotónica. -Es proposa una metodologia per a l'aprenentatge de controladors utilitzant programació dinàmica aproximada a través de programació lineal. La metodologia fa que ADP-LP funcione en aplicacions pràctiques de control amb estats i accions continus. La metodologia proposada estima una cota inferior i superior de la funció de valor òptima a través de aproximadores funcionals. S'estableixen pautes per a les dades i la regularització de regresores amb la finalitat d'obtenir resultats satisfactoris evitant solucions no fitades o mal condicionades. -Es planteja una metodologia sota l'enfocament de programació lineal aplicada a programació dinàmica aproximada per a obtenir una millor aproximació de la funció de valor òptima en una determinada regió de l'espai d'estats. La metodologia proposa aprendre gradualment una política utilitzant dades disponibles només a la regió d'exploració. L'exploració incrementa progressivament la regió d'aprenentatge fins a obtenir una política convergida.[EN] The present Thesis employs dynamic programming and reinforcement learning techniques in order to obtain optimal policies for controlling nonlinear systems with discrete and continuous states and actions. Initially, a review of the basic concepts of dynamic programming and reinforcement learning is carried out for systems with a finite number of states. After that, the extension of these techniques to systems with a large number of states or continuous state systems is analysed using approximation functions. The contributions of the Thesis are: -A combined identification/Q-function fitting methodology, which involves identification of a Takagi-Sugeno model, computation of (sub)optimal controllers from Linear Matrix Inequalities, and the subsequent data-based fitting of Q-function via monotonic optimisation. -A methodology for learning controllers using approximate dynamic programming via linear programming is presented. The methodology makes that ADP-LP approach can work in practical control applications with continuous state and input spaces. The proposed methodology estimates a lower bound and upper bound of the optimal value function through functional approximators. Guidelines are provided for data and regressor regularisation in order to obtain satisfactory results avoiding unbounded or ill-conditioned solutions. -A methodology of approximate dynamic programming via linear programming in order to obtain a better approximation of the optimal value function in a specific region of state space. The methodology proposes to gradually learn a policy using data available only in the exploration region. The exploration progressively increases the learning region until a converged policy is obtained.This work was supported by the National Department of Higher Education, Science, Technology and Innovation of Ecuador (SENESCYT), and the Spanish ministry of Economy and European Union, grant DPI2016-81002-R (AEI/FEDER,UE). The author also received the grant for a predoctoral stay, Programa de Becas Iberoamérica- Santander Investigación 2018, of the Santander Bank.Díaz Iza, HP. (2020). Value Function Estimation in Optimal Control via Takagi-Sugeno Models and Linear Programming [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/139135TESI

Semantics of fuzzy quantifiers

The aim of this thesis is to discuss the semantics of FQs (fuzzy quantifiers), formal semantics in particular. The approach used is fuzzy semantic based on fuzzy set theory (Zadeh 1965, 1975), i.e. we explore primarily the denotational meaning of FQs represented by membership functions. Some empirical data from both Chinese and English is used for illustration. A distinguishing characteristic of the semantics of FQs like about 200 students and many students as opposed to other sorts of quantifiers like every student and no students, is that they have fuzzy meaning boundaries. There is considerable evidence to suggest that the doctrine that a proposition is either true or false has a limited application in natural languages, which raises a serious question towards any linguistic theories that are based on a binary assumption. In other words, the number of elements in a domain that must satisfy a predicate is not precisety given by an FQ and so a proposition con¬ taining one may be more or less true depending on how closely numbers of elements approximate to a given norm. The most significant conclusion drawn here is that FQs are compositional in that FQs of the same type function in the same way to generate a constant semantic pattern. It is argued that although basic membership functions are subject to modification depending on context, they vary only with certain limits (i.e. FQs are motivated—neither completely predicated nor completely arbitrary), which does not deny compositionality in any way. A distinctive combination of compositionality and motivation of FQs makes my formal semantic framework of FQs unique in the way that although some specific values, such as a norm, have to be determined pragmatically, semantic and inferential patterns are systematic and predictable. A number of interdisciplinary implications, such as semantic, general linguistic, logic and psychological, are discussed. The study here seems to be a somewhat troublesome but potentially important area for developing theories (and machines) capable of dealing with, and accounting for, natural languages