Synergies between machine learning and reasoning - An introduction by the Kay R. Amel group

This paper proposes a tentative and original survey of meeting points between Knowledge Representation and Reasoning (KRR) and Machine Learning (ML), two areas which have been developed quite separately in the last four decades. First, some common concerns are identified and discussed such as the types of representation used, the roles of knowledge and data, the lack or the excess of information, or the need for explanations and causal understanding. Then, the survey is organised in seven sections covering most of the territory where KRR and ML meet. We start with a section dealing with prototypical approaches from the literature on learning and reasoning: Inductive Logic Programming, Statistical Relational Learning, and Neurosymbolic AI, where ideas from rule-based reasoning are combined with ML. Then we focus on the use of various forms of background knowledge in learning, ranging from additional regularisation terms in loss functions, to the problem of aligning symbolic and vector space representations, or the use of knowledge graphs for learning. Then, the next section describes how KRR notions may benefit to learning tasks. For instance, constraints can be used as in declarative data mining for influencing the learned patterns; or semantic features are exploited in low-shot learning to compensate for the lack of data; or yet we can take advantage of analogies for learning purposes. Conversely, another section investigates how ML methods may serve KRR goals. For instance, one may learn special kinds of rules such as default rules, fuzzy rules or threshold rules, or special types of information such as constraints, or preferences. The section also covers formal concept analysis and rough sets-based methods. Yet another section reviews various interactions between Automated Reasoning and ML, such as the use of ML methods in SAT solving to make reasoning faster. Then a section deals with works related to model accountability, including explainability and interpretability, fairness and robustness. Finally, a section covers works on handling imperfect or incomplete data, including the problem of learning from uncertain or coarse data, the use of belief functions for regression, a revision-based view of the EM algorithm, the use of possibility theory in statistics, or the learning of imprecise models. This paper thus aims at a better mutual understanding of research in KRR and ML, and how they can cooperate. The paper is completed by an abundant bibliography

Proceedings of the Third International Workshop on Neural Networks and Fuzzy Logic, volume 2

Papers presented at the Neural Networks and Fuzzy Logic Workshop sponsored by the National Aeronautics and Space Administration and cosponsored by the University of Houston, Clear Lake, held 1-3 Jun. 1992 at the Lyndon B. Johnson Space Center in Houston, Texas are included. During the three days approximately 50 papers were presented. Technical topics addressed included adaptive systems; learning algorithms; network architectures; vision; robotics; neurobiological connections; speech recognition and synthesis; fuzzy set theory and application, control and dynamics processing; space applications; fuzzy logic and neural network computers; approximate reasoning; and multiobject decision making

Building environmentally-aware classifiers on streaming data

The three biggest challenges currently faced in machine learning, in our estimation, are the staggering quantity of data we wish to analyze, the incredibly small proportion of these data that are labeled, and the apparent lack of interest in creating algorithms that continually learn during inference. An unsupervised streaming approach addresses all three of these challenges, storing only a finite amount of information to model an unbounded dataset and adapting to new structures as they arise. Specifically, we are motivated by automated target recognition (ATR) in synthetic aperture sonar (SAS) imagery, the problem of finding explosive hazards on the sea oor. It has been shown that the performance of ATR can be improved by, instead of using a single classifier for the entire ATR task, creating several specialized classifers and fusing their predictions [44]. The prevailing opinion seems be that one should have different classifiers for varying complexity of sea oor [74], but we hypothesize that fusing classifiers based on sea bottom type will yield higher accuracy and better lend itself to making explainable classification decisions. The first step of building such a system is developing a robust framework for online texture classification, the topic of this research. xi In this work, we improve upon StreamSoNG [85], an existing algorithm for streaming data analysis (SDA) that models each structure in the data with a neural gas [69] and detects new structures by clustering an outlier list with the possibilistic 1-means [62] (P1M) algorithm. We call the modified algorithm StreamSoNGv2, denoting that it is the second version, or verse, if you will, of StreamSoNG. Notable improvements include detection of arbitrarily-shaped clusters by using DBSCAN [37] instead of P1M, using growing neural gas [43] to model each structure with an adaptive number of prototypes, and an automated approach to estimate the n parameters. Furthermore, we propose a novel algorithm called single-pass possibilistic clustering (SPC) for solving the same task. SPC maintains a fixed number of structures to model the data stream. These structures can be updated and merged based only on their "footprints", that is, summary statistics that contain all of the information from the stream needed by the algorithm without directly maintaining the entire stream. SPC is built on a damped window framework, allowing the user to balance the weight between old and new points in the stream with a decay factor parameter. We evaluate the two algorithms under consideration against four state of the art SDA algorithms from the literature on several synthetic datasets and two texture datasets: one real (KTH-TIPS2b [68]) and xii one simulated. The simulated dataset, a significant research effort in itself, is of our own construction in Unreal Engine and contains on the order of 6,000 images at 720 x 720 resolution from six different texture types. Our hope is that the methodology developed here will be effective texture classifiers for use not only in underwater scene understanding, but also in improving performance of ATR algorithms by providing a context in which the potential target is embedded.Includes bibliographical references

A Hybrid intelligent system for diagnosing and solving financial problems

Tese (doutorado) - Universidade Federal de Santa Catarina, Centro Tecnologico. Programa de Pós-Graduação em Engenharia de Produção2012-10-16T09:55:39

Fuzzy Sets, Fuzzy Logic and Their Applications

The present book contains 20 articles collected from amongst the 53 total submitted manuscripts for the Special Issue “Fuzzy Sets, Fuzzy Loigic and Their Applications” of the MDPI journal Mathematics. The articles, which appear in the book in the series in which they were accepted, published in Volumes 7 (2019) and 8 (2020) of the journal, cover a wide range of topics connected to the theory and applications of fuzzy systems and their extensions and generalizations. This range includes, among others, management of the uncertainty in a fuzzy environment; fuzzy assessment methods of human-machine performance; fuzzy graphs; fuzzy topological and convergence spaces; bipolar fuzzy relations; type-2 fuzzy; and intuitionistic, interval-valued, complex, picture, and Pythagorean fuzzy sets, soft sets and algebras, etc. The applications presented are oriented to finance, fuzzy analytic hierarchy, green supply chain industries, smart health practice, and hotel selection. This wide range of topics makes the book interesting for all those working in the wider area of Fuzzy sets and systems and of fuzzy logic and for those who have the proper mathematical background who wish to become familiar with recent advances in fuzzy mathematics, which has entered to almost all sectors of human life and activity

The design and implementation of fuzzy query processing on sensor networks

Sensor nodes and Wireless Sensor Networks (WSN) enable observation of the physical world in unprecedented levels of granularity. A growing number of environmental monitoring applications are being designed to leverage data collection features of WSN, increasing the need for efficient data management techniques and for comparative analysis of various data management techniques. My research leverages aspects of fuzzy database, specifically fuzzy data representation and fuzzy or flexible queries to improve upon the efficiency of existing data management techniques by exploiting the inherent uncertainty of the data collected by WSN. Herein I present my research contributions. I provide classification of WSN middleware to illustrate varying approaches to data management for WSN and identify a need to better handle the uncertainty inherent in data collected from physical environments and to take advantage of the imprecision of the data to increase the efficiency of WSN by requiring less information be transmitted to adequately answer queries posed by WSN monitoring applications. In this dissertation, I present a novel approach to querying WSN, in which semantic knowledge about sensor attributes is represented as fuzzy terms. I present an enhanced simulation environment that supports more flexible and realistic analysis by using cellular automata models to separately model the deployed WSN and the underlying physical environment. Simulation experiments are used to evaluate my fuzzy query approach for environmental monitoring applications. My analysis shows that using fuzzy queries improves upon other data management techniques by reducing the amount of data that needs to be collected to accurately satisfy application requests. This reduction in data transmission results in increased battery life within sensors, an important measure of cost and performance for WSN applications

Fusing actigraphy signals for outpatient monitoring

[EN] Actigraphy devices have been successfully used as effective tools in the treatment of diseases such as sleep disorders or major depression. Although several efforts have been made in recent years to develop smaller and more portable devices, the features necessary for the continuous monitoring of outpatients require a less intrusive, obstructive and stigmatizing acquisition system. A useful strategy to overcome these limitations is based on adapting the monitoring system to the patient lifestyle and behavior by providing sets of different sensors that can be worn simultaneously or alternatively. This strategy offers to the patient the option of using one device or other according to his/her particular preferences. However this strategy requires a robust multi-sensor fusion methodology capable of taking maximum profit from all of the recorded information. With this aim, this study proposes two actigraphy fusion models including centralized and distributed architectures based on artificial neural networks. These novel fusion methods were tested both on synthetic datasets and real datasets, providing a parametric characterization of the models' behavior, and yielding results based on real case applications. The results obtained using both proposed fusion models exhibit good performance in terms of robustness to signal degradation, as well as a good behavior in terms of the dependence of signal quality on the number of signals fused. The distributed and centralized fusion methods reduce the mean averaged error of the original signals to 44% and 46% respectively when using simulated datasets. The proposed methods may therefore facilitate a less intrusive and more dependable way of acquiring valuable monitoring information from outpatients.This work was partially funded by the European Commission: Help4Mood (Contract No. FP7-ICT-2009-4: 248765). E. FusterGarcia acknowledges Programa Torres Quevedo from Ministerio de Educacion y Ciencia, co-founded by the European Social Fund (PTQ-12-05693).Fuster García, E.; Bresó Guardado, A.; Martínez Miranda, JC.; Rosell-Ferrer, J.; Matheson, C.; García Gómez, JM. (2015). Fusing actigraphy signals for outpatient monitoring. Information Fusion. 23:69-80. https://doi.org/10.1016/j.inffus.2014.08.003S69802

Advances in Possibilistic Clustering with Application to Hyperspectral Image Processing

Η ομαδοποίηση δεδομένων είναι μια εδραιωμένη μεθοδολογία ανάλυσης δεδομένων που έχει χρησιμοποιηθεί εκτενώς σε διάφορα πεδία εφαρμογών κατά τη διάρκεια των τελευταίων δεκαετιών. Η παρούσα διατριβή εστιάζει κυρίως στην ευρύτερη οικογένεια των αλγορίθμων βελτιστοποίησης κόστους και πιο συγκεκριμένα στους αλγόριθμους ομαδοποίησης με βάση τα ενδεχόμενα (Possibilistic c-Means, PCM). Συγκεκριμένα, αφού εκτίθενται τα αδύνατα σημεία τους, προτείνονται νέοι (batch και online) PCM αλγόριθμοι που αποτελούν επεκτάσεις των προηγουμένων και αντιμετωπίζουν τα αδύνατα σημεία των πρώτων. Οι προτεινόμενοι αλγόριθμοι ομαδοποίησης βασίζονται κυρίως στην υιοθέτηση των εννοιών (α) της προσαρμοστικότητας παραμέτρων (parameter adaptivity), οι οποίες στους κλασσικούς PCM αλγορίθμους παραμένουν σταθερές κατά την εκτέλεσή τους και (β) της αραιότητας (sparsity). Αυτά τα χαρακτηριστικά προσδίδουν νέα δυναμική στους προτεινόμενους αλγορίθμους οι οποίοι πλέον: (α) είναι (κατ' αρχήν) σε θέση να προσδιορίσουν τον πραγματικό αριθμό των φυσικών ομάδων που σχηματίζονται από τα δεδομένα, (β) είναι ικανοί να αποκαλύψουν την υποκείμενη δομή ομαδοποίησης, ακόμη και σε δύσκολες περιπτώσεις, όπου οι φυσικές ομάδες βρίσκονται κοντά η μία στην άλλη ή/και έχουν σημαντικές διαφορές στις διακυμάνσεις ή/και στις πυκνότητές τους και (γ) είναι εύρωστοι στην παρουσία θορύβου και ακραίων σημείων. Επίσης, δίνονται θεωρητικά αποτελέσματα σχετικά με τη σύγκλιση των προτεινόμενων αλγορίθμων, τα οποία βρίσκουν επίσης εφαρμογή και στους κλασσικούς PCM αλγορίθμους. Η δυναμική των προτεινόμενων αλγορίθμων αναδεικνύεται μέσω εκτεταμένων πειραμάτων, τόσο σε συνθετικά όσο και σε πραγματικά δεδομένα. Επιπλέον, οι αλγόριθμοι αυτοί έχουν εφαρμοστεί με επιτυχία στο ιδιαίτερα απαιτητικό πρόβλημα της ομαδοποίησης σε υπερφασματικές εικόνες. Τέλος, αναπτύχθηκε και μια μέθοδος επιλογής χαρακτηριστικών κατάλληλη για υπερφασματικές εικόνες.Clustering is a well established data analysis methodology that has been extensively used in various fields of applications during the last decades. The main focus of the present thesis is on a well-known cost-function optimization-based family of clustering algorithms, called Possibilistic C-Means (PCM) algorithms. Specifically, the shortcomings of PCM algorithms are exposed and novel batch and online PCM schemes are proposed to cope with them. These schemes rely on (i) the adaptation of certain parameters which remain fixed during the execution of the original PCMs and (ii) the adoption of sparsity. The incorporation of these two characteristics renders the proposed schemes: (a) capable, in principle, to reveal the true number of physical clusters formed by the data, (b) capable to uncover the underlying clustering structure even in demanding cases, where the physical clusters are closely located to each other and/or have significant differences in their variances and/or densities, and (c) immune to the presence of noise and outliers. Moreover, theoretical results concerning the convergence of the proposed algorithms, also applicable to the classical PCMs, are provided. The potential of the proposed methods is demonstrated via extensive experimentation on both synthetic and real data sets. In addition, they have been successfully applied on the challenging problem of clustering in HyperSpectral Images (HSIs). Finally, a feature selection technique suitable for HSIs has also been developed