Explainable AI using expressive Boolean formulas

We propose and implement an interpretable machine learning classification model for Explainable AI (XAI) based on expressive Boolean formulas. Potential applications include credit scoring and diagnosis of medical conditions. The Boolean formula defines a rule with tunable complexity (or interpretability), according to which input data are classified. Such a formula can include any operator that can be applied to one or more Boolean variables, thus providing higher expressivity compared to more rigid rule-based and tree-based approaches. The classifier is trained using native local optimization techniques, efficiently searching the space of feasible formulas. Shallow rules can be determined by fast Integer Linear Programming (ILP) or Quadratic Unconstrained Binary Optimization (QUBO) solvers, potentially powered by special purpose hardware or quantum devices. We combine the expressivity and efficiency of the native local optimizer with the fast operation of these devices by executing non-local moves that optimize over subtrees of the full Boolean formula. We provide extensive numerical benchmarking results featuring several baselines on well-known public datasets. Based on the results, we find that the native local rule classifier is generally competitive with the other classifiers. The addition of non-local moves achieves similar results with fewer iterations, and therefore using specialized or quantum hardware could lead to a speedup by fast proposal of non-local moves.Comment: 28 pages, 16 figures, 4 table

Active learning of link specifications using decision tree learning

In this work we presented an implementation that uses decision trees to learn highly accurate link specifications. We compared our approach with three state-of-the-art classifiers on nine datasets and showed, that our approach gives comparable results in a reasonable amount of time. It was also shown, that we outperform the state-of-the-art on four datasets by up to 30%, but are still behind slightly on average. The effect of user feedback on the active learning variant was inspected pertaining to the number of iterations needed to deliver good results. It was shown that we can get FScores above 0.8 with most datasets after 14 iterations

Advanced analysis of branch and bound algorithms

Als de code van een cijferslot zoek is, kan het alleen geopend worden door alle cijfercom­binaties langs te gaan. In het slechtste geval is de laatste combinatie de juiste. Echter, als de code uit tien cijfers bestaat, moeten tien miljard mogelijkheden bekeken worden. De zogenaamde 'NP-lastige' problemen in het proefschrift van Marcel Turkensteen zijn vergelijkbaar met het 'cijferslotprobleem'. Ook bij deze problemen is het aantal mogelijkheden buitensporig groot. De kunst is derhalve om de zoekruimte op een slimme manier af te tasten. Bij de Branch and Bound (BnB) methode wordt dit gedaan door de zoekruimte op te splitsen in kleinere deelgebieden. Turkensteen past de BnB methode onder andere toe bij het handelsreizigersprobleem, waarbij een kortste route door een verzameling plaatsen bepaald moet worden. Dit probleem is in algemene vorm nog steeds niet opgelost. De economische gevolgen kunnen groot zijn: zo staat nog steeds niet vast of bijvoorbeeld een routeplanner vrachtwagens optimaal laat rondrijden. De huidige BnB-methoden worden in dit proefschrift met name verbeterd door niet naar de kosten van een verbinding te kijken, maar naar de kostentoename als een verbinding niet gebruikt wordt: de boventolerantie.

Nested Sampling Methods

Nested sampling (NS) computes parameter posterior distributions and makes Bayesian model comparison computationally feasible. Its strengths are the unsupervised navigation of complex, potentially multi-modal posteriors until a well-defined termination point. A systematic literature review of nested sampling algorithms and variants is presented. We focus on complete algorithms, including solutions to likelihood-restricted prior sampling, parallelisation, termination and diagnostics. The relation between number of live points, dimensionality and computational cost is studied for two complete algorithms. A new formulation of NS is presented, which casts the parameter space exploration as a search on a tree. Previously published ways of obtaining robust error estimates and dynamic variations of the number of live points are presented as special cases of this formulation. A new on-line diagnostic test is presented based on previous insertion rank order work. The survey of nested sampling methods concludes with outlooks for future research.Comment: Updated version incorporating constructive input from four(!) positive reports (two referees, assistant editor and editor). The open-source UltraNest package and astrostatistics tutorials can be found at https://johannesbuchner.github.io/UltraNest

Deformation Correlations and Machine Learning: Microstructural inference and crystal plasticity predictions

The present thesis makes a connection between spatially resolved strain correlations and material processing history. Such correlations can be used to infer and classify prior deformation history of a sample at various strain levels with the use of Machine Learning approaches. A simple and concrete example of uniaxially compressed crystalline thin films of various sizes, generated by two-dimensional discrete dislocation plasticity simulations is examined. At the nanoscale, thin films exhibit yield-strength size effects with noisy mechanical responses which create an interesting challenge for the application of Machine Learning techniques. Moreover, this thesis demonstrates the prediction of the average mechanical responses of thin films based on the classified prior deformation history and discusses the possible ramifications for modelling crystal plasticity behavior in extreme settings

Machine Learning for Disease Outbreak Detection Using Probabilistic Models

RÉSUMÉ L’expansion de maladies connues et l’émergence de nouvelles maladies ont affecté la vie de nombreuses personnes et ont eu des conséquences économiques importantes. L’Ébola n’est que le dernier des exemples récents. La détection précoce d’infections épidémiologiques s’avère donc un enjeu de taille. Dans le secteur de la surveillance syndromique, nous avons assisté récemment à une prolifération d’algorithmes de détection d’épidémies. Leur performance peut varier entre eux et selon différents paramètres de configuration, de sorte que l’efficacité d’un système de surveillance épidémiologique s’en trouve d’autant affecté. Pourtant, on ne possède que peu d’évaluations fiables de la performance de ces algorithmes sous différentes conditions et pour différents types d’épidémie. Les évaluations existantes sont basées sur des cas uniques et les données ne sont pas du domaine public. Il est donc difficile de comparer ces algorithmes entre eux et difficile de juger de la généralisation des résultats. Par conséquent, nous ne sommes pas en mesure de déterminer quel d’algorithme devrait être appliqué dans quelles circonstances. Cette thèse poursuit trois objectifs généraux : (1) établir la relation entre la performance des algorithmes de détection d’épidémies et le type et la sévérité de ces épidémies, (2) améliorer les prédictions d’épidémies par la combinaison d’algorithmes et (3) fournir une méthode d’analyse des épidémies qui englobe une perspective de coûts afin de minimiser l’impact économique des erreurs du type faux positifs et faux négatifs. L’approche générale de notre étude repose sur l’utilisation de données de simulation d’épidémies dont le vecteur de transmission est un réseau d’aqueducs. Les données sont obtenues de la plateforme de simulation SnAP du Department of Epidemiology and Biostatistics Surveillance Lab de l’université McGill. Cette approche nous permet de créer les différentes conditions de types et d’intensités d’épidémiologie nécessaires à l’analyse de la performance des algorithmes de détection. Le premier objectif porte sur l’influence des différents types et différentes intensités d’épidémiologie sur la performance des algorithmes. Elle est modélisée à l’aide d’un modèle basé sur un réseau bayésien. Ce modèle prédit avec succès la variation de performance observée dans les données. De plus, l’utilisation d’un réseau bayésien permet de quantifier l’influence de chaque variable et relève aussi le rôle que jouent d’autres paramètres qui étaient jusqu’ici ignorés dans les travaux antérieurs, à savoir le seuil de détection et l’importance de tenir compte de récurrences hebdomadaires. Le second objectif vise à exploiter les résultats autour du premier objectif et de combiner les algorithmes pour optimiser la performance en fonction des facteurs d’influence. Les résultats des algorithmes sont combinés à l’aide de la méthode de Mixture hiérarchique d’expert (Hierarchical Mixture of Experts—HME). Le modèle HME est entraîné à pondérer la contribution de chaque algorithme en fonction des données. Les résultats de cette combinaison des résultats d’algorithmes sont comparables avec les meilleurs résultats des algorithmes individuels, et s’avèrent plus robustes à travers différentes variations. Le niveau de contamination n’influence pas la performance relative du modèle HME. Finalement, nous avons tenté d’optimiser des méthodes de détection d’épidémies en fonction des coûts et bénéfices escomptés des prédictions correctes et incorrects. Les résultats des algorithms de détection sont évalués en fonction des décisions possibles qui en découlent et en tenant compte de données réelles sur les coûts totaux d’utilisation des ressources du système de santé. Dans un premier temps, une régression polynomiale permet d’estimer le coût d’une épidémie selon le délai de détection. Puis, nous avons développé un modèle d’apprentissage d’arbre de décision qui tient compte du coût et qui prédit les détections à partir des algorithmes connus. Les résultats expérimentaux démontrent que ce modèle permet de réduire le coût total des épidémies, tout en maintenant le niveau de détection des épidémies comparables à ceux d’autres méthodes.----------ABSTRACT The past decade has seen the emergence of new diseases or expansion of old ones (such as Ebola) causing high human and financial costs. Hence, early detection of disease outbreaks is crucial. In the field of syndromic surveillance, there has recently been a proliferation of outbreak detection algorithms. The choice of outbreak detection algorithm and its configuration can result in important variations in the performance of public health surveillance systems. But performance evaluations have not kept pace with algorithm development. These evaluations are usually based on a single data set which is not publicly available, so the evaluations are difficult to generalize or replicate. Furthermore, the performance of different algorithms is influenced by the nature of the disease outbreak. As a result of the lack of thorough performance evaluations, one cannot determine which algorithm should be applied under what circumstances. Briefly, this research has three general objectives: (1) characterize the dependence of the performance of detection algorithms on the type and severity of outbreak, (2) aggregate the predictions of several outbreak detection algorithms, (3) analyze outbreak detection methods from a cost-benefit point of view and develop a detection method which minimizes the total cost of missing outbreaks and false alarms. To achieve the first objective, we propose a Bayesian network model learned from simulated outbreak data overlaid on real healthcare utilization data which predicts detection performance as a function of outbreak characteristics and surveillance system parameters. This model predicts the performance of outbreak detection methods with high accuracy. The model can also quantify the influence of different outbreak characteristics and detection methods on detection performance in a variety of practically relevant surveillance scenarios. In addition to identifying outbreak characteristics expected to have a strong influence on detection performance, the learned model suggests a role for other algorithm features, such as alerting threshold and taking weekly patterns into account, which was previously not the focus of attention in the literature. To achieve the second objective, we use Hierarchical Mixture of Experts (HME) to combine the responses of multiple experts (i.e., predictors) which are outbreak detection methods. The contribution of each predictor in forming the final output is learned and depends on the input data. The developed HME algorithm is competitive with the best detection algorithm in the experimental evaluation, and is more robust under different circumstances. The level of contamination of the surveillance time series does not influence the relative performance of the HME. The optimization of outbreak detection methods also relies on the estimation of future benefits of true alarms and the cost of false alarms. In the third part of the thesis, we analyze some commonly used outbreak detection methods in terms of the cost of missing outbreaks and false alarms, using simulated outbreak data overlaid on real healthcare utilization data. We estimate the total cost of missing outbreaks and false alarms, in addition to the accuracy of outbreak detection and we fit a polynomial regression function to estimate the cost of an outbreak based on the delay until it is detected. Then, we develop a cost-sensitive decision tree learner, which predicts outbreaks by looking at the prediction of commonly used detection methods. Experimental results show that using the developed cost-sensitive decision tree decreases the total cost of the outbreak, while the accuracy of outbreak detection remains competitive with commonly used methods

Approximate Bayesian conditional copulas

Copula models are flexible tools to represent complex structures of dependence for multivariate random variables. According to Sklar's theorem, any multidimensional absolutely continuous distribution function can be uniquely represented as a copula, i.e. a joint cumulative distribution function on the unit hypercube with uniform marginals, which captures the dependence structure among the vector components. In real data applications, the interest of the analyses often lies on specific functionals of the dependence, which quantify aspects of it in a few numerical values. A broad literature exists on such functionals, however extensions to include covariates are still limited. This is mainly due to the lack of unbiased estimators of the conditional copula, especially when one does not have enough information to select the copula model. Several Bayesian methods to approximate the posterior distribution of functionals of the dependence varying according covariates are presented and compared; the main advantage of the investigated methods is that they use nonparametric models, avoiding the selection of the copula, which is usually a delicate aspect of copula modelling. These methods are compared in simulation studies and in two realistic applications, from civil engineering and astrophysics. (C) 2022 Elsevier B.V. All rights reserved