Fuzzy set covering as a new paradigm for the induction of fuzzy classification rules

In 1965 Lofti A. Zadeh proposed fuzzy sets as a generalization of crisp (or classic) sets to address the incapability of crisp sets to model uncertainty and vagueness inherent in the real world. Initially, fuzzy sets did not receive a very warm welcome as many academics stood skeptical towards a theory of imprecise'' mathematics. In the middle to late 1980's the success of fuzzy controllers brought fuzzy sets into the limelight, and many applications using fuzzy sets started appearing. In the early 1970's the first machine learning algorithms started appearing. The AQ family of algorithms pioneered by Ryszard S. Michalski is a good example of the family of set covering algorithms. This class of learning algorithm induces concept descriptions by a greedy construction of rules that describe (or cover) positive training examples but not negative training examples. The learning process is iterative, and in each iteration one rule is induced and the positive examples covered by the rule removed from the set of positive training examples. Because positive instances are separated from negative instances, the term separate-and-conquer has been used to contrast the learning strategy against decision tree induction that use a divide-and-conquer learning strategy. This dissertation proposes fuzzy set covering as a powerful rule induction strategy. We survey existing fuzzy learning algorithms, and conclude that very few fuzzy learning algorithms follow a greedy rule construction strategy and no publications to date made the link between fuzzy sets and set covering explicit. We first develop the theoretical aspects of fuzzy set covering, and then apply these in proposing the first fuzzy learning algorithm that apply set covering and make explicit use of a partial order for fuzzy classification rule induction. We also investigate several strategies to improve upon the basic algorithm, such as better search heuristics and different rule evaluation metrics. We then continue by proposing a general unifying framework for fuzzy set covering algorithms. We demonstrate the benefits of the framework and propose several further fuzzy set covering algorithms that fit within the framework. We compare fuzzy and crisp rule induction, and provide arguments in favour of fuzzy set covering as a rule induction strategy. We also show that our learning algorithms outperform other fuzzy rule learners on real world data. We further explore the idea of simultaneous concept learning in the fuzzy case, and continue to propose the first fuzzy decision list induction algorithm. Finally, we propose a first strategy for encoding the rule sets generated by our fuzzy set covering algorithms inside an equivalent neural network

Investigation of hidden multipolar spin order in frustrated magnets using interpretable machine learning techniques

Frustration gives rise to a plethora of intricate phenomena, the most salient of which are spin liquids, both classical ones—such as the spin-ice phase which has been realized experimentally in rare-earth oxide pyrochlore materials—and their more elusive quantum counterparts. At low temperatures, classical frustrated spin systems may still order, despite their extensive ground-state degeneracy, due to the order-by-disorder phenomenon. The resulting orders are often of a multipolar type which defies conventional probes. Identifying and characterizing such “hidden” orders is thus a challenging endeavor. This thesis introduces a machine-learning framework for studying the phase diagram of classical frustrated spin models in an unbiased and automated way. The interpretability of the resulting classification was of paramount importance in the design of the method. It allows for the inference of both the order parameter tensors of the phases with broken symmetries as well as the constraints which are characteristic of classical spin liquids and signal their emergent gauge structure. On top of that, it establishes a hierarchical relationship among the various phases according to their degree of disorder. The framework is applied to three different models and spin configurations are harvested from classical Monte Carlo simulations of those. A gauge model is used to mimic the interactions between the mesogens of generalized nematics. These may possess arbitrary point group symmetry, resulting in benchmark models with a low-temperature phase that breaks the O(3) spin symmetry accordingly. In addition, two frustrated spin models are considered. The historically important case of the Heisenberg model on the kagome lattice gives rise to hidden triatic order which requires a description in terms of two tensors of different ranks; the machine is capable of finding both. Meanwhile, for the XXZ model on the pyrochlore lattice, the machine reconstructs the complex phase diagram which was only recently obtained and correctly identifies the spin nematic phase as well as three distinct types of classical spin liquids, including their crossovers. The method has the potential to accelerate the characterization of model Hamiltonians of frustrated magnets. It can scrutinize the whole parameter space at once and may thus help to identify interesting regimes, paving the way for the search of new orders and spin liquids.Frustration führt zu einer Fülle komplexer Phänomene, von denen die herausragendsten Spinflüssigkeiten sind, sowohl klassische – wie beispielsweise die Spin-Eis-Phase, die experimentell in den Oxiden seltener Erden auf dem Pyrochlor-Gitter realisiert wurde – und ihre schwerer fassbaren quantenmechanischen Gegenstücke. Bei niedrigen Temperaturen können klassische frustrierte Spinsysteme obgleich der extensiven Entartung des Grundzustandes aufgrund des Phänomens der „Ordnung durch Unordnung“ dennoch Ordnungen ausbilden. Diese sind oft multipolarer Natur und entziehen sich herkömmlichen Messgrößen. Die Identifikation und Charakterisierung solcher „verborgener“ Ordnungen ist daher eine herausfordernde Aufgabe. In dieser Arbeit wird ein Verfahren für das unvoreingenommene und automatisierte maschinelle Lernen der Phasendiagramme klassischer frustrierter Spinmodelle eingeführt. Die Interpretierbarkeit der resultierenden Klassifikatoren war für das Design der Methode ausschlaggebend. Sie erlaubt den Rückschluss sowohl auf die Ordnungsparametertensoren der symmetriebrechenden Phasen als auch auf die Nebenbedingungen, die für klassische Spinflüssigkeiten charakteristisch sind und auf deren emergente Eichstruktur hindeuten. Darüber hinaus wird eine hierarchische Beziehung zwischen den verschiedenen Phasen gemäß dem Grade ihrer jeweiligen Unordnung hergestellt. Das Verfahren wird auf drei verschiedene Modelle angewendet und Spin-Konfigurationen werden jeweils aus klassischen Monte-Carlo-Simulationen dieser gewonnen. Ein Eichmodell dient dazu, die Wechselwirkungen zwischen den Mesogenen verallgemeinerter nematischer Flüssigkristalle nachzuahmen. Diese können beliebige Punktgruppensymmetrien besitzen, was zu Benchmark-Modellen mit einer Niedertemperaturphase führt, die die O(3)-Spinsymmetrie entsprechend herunterbricht. Darüber hinaus werden zwei frustrierte Spinmodelle betrachtet. Der historisch wichtige Fall des Heisenberg-Modells auf dem Kagome-Gitter führt zu einer verborgenen trigonalen Ordnung, die eine Beschreibung in Form von zwei Tensoren unterschiedlichen Ranges erforderlich macht; die Maschine ist in der Lage, beide zu finden. Währenddessen rekonstruiert die Maschine für das XXZ-Modell auf dem Pyrochlor-Gitter das komplexe Phasendiagramm, das erst vor Kurzem ausgearbeitet wurde, und identifiziert die spin-nematische Phase sowie drei verschiedene Arten klassischer Spinflüssigkeiten, einschließlich ihrer Übergänge, korrekt. Die Methode hat das Potenzial, die Charakterisierung von Spinmodellen frustrierter Magnete zu beschleunigen. Sie kann den gesamten Parameterraum auf einmal untersuchen und somit dazu beitragen, interessante Bereiche zu identifizieren. Dies bereitet den Weg für die Suche nach neuen Ordnungen und Spinflüssigkeiten

Investigation of hidden multipolar spin order in frustrated magnets using interpretable machine learning techniques

Frustration gives rise to a plethora of intricate phenomena, the most salient of which are spin liquids, both classical ones—such as the spin-ice phase which has been realized experimentally in rare-earth oxide pyrochlore materials—and their more elusive quantum counterparts. At low temperatures, classical frustrated spin systems may still order, despite their extensive ground-state degeneracy, due to the order-by-disorder phenomenon. The resulting orders are often of a multipolar type which defies conventional probes. Identifying and characterizing such “hidden” orders is thus a challenging endeavor. This thesis introduces a machine-learning framework for studying the phase diagram of classical frustrated spin models in an unbiased and automated way. The interpretability of the resulting classification was of paramount importance in the design of the method. It allows for the inference of both the order parameter tensors of the phases with broken symmetries as well as the constraints which are characteristic of classical spin liquids and signal their emergent gauge structure. On top of that, it establishes a hierarchical relationship among the various phases according to their degree of disorder. The framework is applied to three different models and spin configurations are harvested from classical Monte Carlo simulations of those. A gauge model is used to mimic the interactions between the mesogens of generalized nematics. These may possess arbitrary point group symmetry, resulting in benchmark models with a low-temperature phase that breaks the O(3) spin symmetry accordingly. In addition, two frustrated spin models are considered. The historically important case of the Heisenberg model on the kagome lattice gives rise to hidden triatic order which requires a description in terms of two tensors of different ranks; the machine is capable of finding both. Meanwhile, for the XXZ model on the pyrochlore lattice, the machine reconstructs the complex phase diagram which was only recently obtained and correctly identifies the spin nematic phase as well as three distinct types of classical spin liquids, including their crossovers. The method has the potential to accelerate the characterization of model Hamiltonians of frustrated magnets. It can scrutinize the whole parameter space at once and may thus help to identify interesting regimes, paving the way for the search of new orders and spin liquids.Frustration führt zu einer Fülle komplexer Phänomene, von denen die herausragendsten Spinflüssigkeiten sind, sowohl klassische – wie beispielsweise die Spin-Eis-Phase, die experimentell in den Oxiden seltener Erden auf dem Pyrochlor-Gitter realisiert wurde – und ihre schwerer fassbaren quantenmechanischen Gegenstücke. Bei niedrigen Temperaturen können klassische frustrierte Spinsysteme obgleich der extensiven Entartung des Grundzustandes aufgrund des Phänomens der „Ordnung durch Unordnung“ dennoch Ordnungen ausbilden. Diese sind oft multipolarer Natur und entziehen sich herkömmlichen Messgrößen. Die Identifikation und Charakterisierung solcher „verborgener“ Ordnungen ist daher eine herausfordernde Aufgabe. In dieser Arbeit wird ein Verfahren für das unvoreingenommene und automatisierte maschinelle Lernen der Phasendiagramme klassischer frustrierter Spinmodelle eingeführt. Die Interpretierbarkeit der resultierenden Klassifikatoren war für das Design der Methode ausschlaggebend. Sie erlaubt den Rückschluss sowohl auf die Ordnungsparametertensoren der symmetriebrechenden Phasen als auch auf die Nebenbedingungen, die für klassische Spinflüssigkeiten charakteristisch sind und auf deren emergente Eichstruktur hindeuten. Darüber hinaus wird eine hierarchische Beziehung zwischen den verschiedenen Phasen gemäß dem Grade ihrer jeweiligen Unordnung hergestellt. Das Verfahren wird auf drei verschiedene Modelle angewendet und Spin-Konfigurationen werden jeweils aus klassischen Monte-Carlo-Simulationen dieser gewonnen. Ein Eichmodell dient dazu, die Wechselwirkungen zwischen den Mesogenen verallgemeinerter nematischer Flüssigkristalle nachzuahmen. Diese können beliebige Punktgruppensymmetrien besitzen, was zu Benchmark-Modellen mit einer Niedertemperaturphase führt, die die O(3)-Spinsymmetrie entsprechend herunterbricht. Darüber hinaus werden zwei frustrierte Spinmodelle betrachtet. Der historisch wichtige Fall des Heisenberg-Modells auf dem Kagome-Gitter führt zu einer verborgenen trigonalen Ordnung, die eine Beschreibung in Form von zwei Tensoren unterschiedlichen Ranges erforderlich macht; die Maschine ist in der Lage, beide zu finden. Währenddessen rekonstruiert die Maschine für das XXZ-Modell auf dem Pyrochlor-Gitter das komplexe Phasendiagramm, das erst vor Kurzem ausgearbeitet wurde, und identifiziert die spin-nematische Phase sowie drei verschiedene Arten klassischer Spinflüssigkeiten, einschließlich ihrer Übergänge, korrekt. Die Methode hat das Potenzial, die Charakterisierung von Spinmodellen frustrierter Magnete zu beschleunigen. Sie kann den gesamten Parameterraum auf einmal untersuchen und somit dazu beitragen, interessante Bereiche zu identifizieren. Dies bereitet den Weg für die Suche nach neuen Ordnungen und Spinflüssigkeiten

Bioinspired metaheuristic algorithms for global optimization

This paper presents concise comparison study of newly developed bioinspired algorithms for global optimization problems. Three different metaheuristic techniques, namely Accelerated Particle Swarm Optimization (APSO), Firefly Algorithm (FA), and Grey Wolf Optimizer (GWO) are investigated and implemented in Matlab environment. These methods are compared on four unimodal and multimodal nonlinear functions in order to find global optimum values. Computational results indicate that GWO outperforms other intelligent techniques, and that all aforementioned algorithms can be successfully used for optimization of continuous functions

Tune your brown clustering, please

Brown clustering, an unsupervised hierarchical clustering technique based on ngram mutual information, has proven useful in many NLP applications. However, most uses of Brown clustering employ the same default configuration; the appropriateness of this configuration has gone predominantly unexplored. Accordingly, we present information for practitioners on the behaviour of Brown clustering in order to assist hyper-parametre tuning, in the form of a theoretical model of Brown clustering utility. This model is then evaluated empirically in two sequence labelling tasks over two text types. We explore the dynamic between the input corpus size, chosen number of classes, and quality of the resulting clusters, which has an impact for any approach using Brown clustering. In every scenario that we examine, our results reveal that the values most commonly used for the clustering are sub-optimal

Experimental Evaluation of Growing and Pruning Hyper Basis Function Neural Networks Trained with Extended Information Filter

In this paper we test Extended Information Filter (EIF) for sequential training of Hyper Basis Function Neural Networks with growing and pruning ability (HBF-GP). The HBF neuron allows different scaling of input dimensions to provide better generalization property when dealing with complex nonlinear problems in engineering practice. The main intuition behind HBF is in generalization of Gaussian type of neuron that applies Mahalanobis-like distance as a distance metrics between input training sample and prototype vector. We exploit concept of neuron’s significance and allow growing and pruning of HBF neurons during sequential learning process. From engineer’s perspective, EIF is attractive for training of neural networks because it allows a designer to have scarce initial knowledge of the system/problem. Extensive experimental study shows that HBF neural network trained with EIF achieves same prediction error and compactness of network topology when compared to EKF, but without the need to know initial state uncertainty, which is its main advantage over EKF