Efficient Maximum A-Posteriori Inference in Markov Logic and Application in Description Logics

Maximum a-posteriori (MAP) query in statistical relational models computes the most probable world given evidence and further knowledge about the domain. It is arguably one of the most important types of computational problems, since it is also used as a subroutine in weight learning algorithms. In this thesis, we discuss an improved inference algorithm and an application for MAP queries. We focus on Markov logic (ML) as statistical relational formalism. Markov logic combines Markov networks with first-order logic by attaching weights to first-order formulas. For inference, we improve existing work which translates MAP queries to integer linear programs (ILP). The motivation is that existing ILP solvers are very stable and fast and are able to precisely estimate the quality of an intermediate solution. In our work, we focus on improving the translation process such that we result in ILPs having fewer variables and fewer constraints. Our main contribution is the Cutting Plane Aggregation (CPA) approach which leverages symmetries in ML networks and parallelizes MAP inference. Additionally, we integrate the cutting plane inference (Riedel 2008) algorithm which significantly reduces the number of groundings by solving multiple smaller ILPs instead of one large ILP. We present the new Markov logic engine RockIt which outperforms state-of-the-art engines in standard Markov logic benchmarks. Afterwards, we apply the MAP query to description logics. Description logics (DL) are knowledge representation formalisms whose expressivity is higher than propositional logic but lower than first-order logic. The most popular DLs have been standardized in the ontology language OWL and are an elementary component in the Semantic Web. We combine Markov logic, which essentially follows the semantic of a log-linear model, with description logics to log-linear description logics. In log-linear description logic weights can be attached to any description logic axiom. Furthermore, we introduce a new query type which computes the most-probable 'coherent' world. Possible applications of log-linear description logics are mainly located in the area of ontology learning and data integration. With our novel log-linear description logic reasoner ELog, we experimentally show that more expressivity increases quality and that the solutions of optimal solving strategies have higher quality than the solutions of approximate solving strategies

Graphical preference representation under a possibilistic framework

La modélisation structurée de préférences, fondée sur les notions d'indépendance préférentielle, a un potentiel énorme pour fournir des approches efficaces pour la représentation et le raisonnement sur les préférences des décideurs dans les applications de la vie réelle. Cette thèse soulève la question de la représentation des préférences par une structure graphique. Nous proposons une nouvelle lecture de réseaux possibilistes, que nous appelons p-pref nets, où les degrés de possibilité représentent des degrés de satisfaction. L'approche utilise des poids de possibilité non instanciés (appelés poids symboliques), pour définir les tables de préférences conditionnelles. Ces tables donnent naissance à des vecteurs de poids symboliques qui codent les préférences qui sont satisfaites et celles qui sont violées dans un contexte donné. Nous nous concentrons ensuite sur les aspects théoriques de la manipulation de ces vecteurs. En effet, la comparaison de ces vecteurs peut s'appuyer sur différentes méthodes: celles induites par la règle de chaînage basée sur le produit ou celle basée sur le minimum que sous-tend le réseau possibiliste, les raffinements du minimum le discrimin, ou leximin, ainsi que l'ordre Pareto, et le Pareto symétrique qui le raffine. Nous prouvons que la comparaison par produit correspond exactement au celle du Pareto symétrique et nous nous concentrons sur les avantages de ce dernier par rapport aux autres méthodes. En outre, nous montrons que l'ordre du produit est consistant avec celui obtenu en comparant des ensembles de préférences satisfaites des tables. L'image est complétée par la proposition des algorithmes d'optimisation et de dominance pour les p-pref nets. Dans ce travail, nous discutons divers outils graphiques pour la représentation des préférences. Nous nous focalisons en particulier sur les CP-nets car ils partagent la même structure graphique que les p-pref nets et sont basés sur la même nature de préférences. Nous prouvons que les ordres induits par les CP-nets ne peuvent pas contredire ceux des p-pref nets et nous avons fixé les contraintes nécessaires pour raffiner les ordres des p-pref nets afin de capturer les contraintes Ceteris Paribus des CP-nets. Cela indique que les CP-nets représentent potentiellement une sous-classe des p-pref nets avec des contraintes. Ensuite, nous fournissons une comparaison approfondie entre les différents modèles graphiques qualitatifs et quantitatifs, et les p-pref nets. Nous en déduisons que ces derniers peuvent être placés à mi- chemin entre les modèles qualitatifs et les modèles quantitatifs puisqu'ils ne nécessitent pas une instanciation complète des poids symboliques alors que des informations supplémentaires sur l'importance des poids peuvent être prises en compte. La dernière partie de ce travail est consacrée à l'extension du modèle proposé pour représenter les préférences de plusieurs agents. Dans un premier temps, nous proposons l'utilisation de réseaux possibilistes où les préférences sont de type tout ou rien et nous définissons le conditionnement dans le cas de distributions booléennes. Nous montrons par ailleurs que ces réseaux multi-agents ont une contrepartie logique utile pour vérifier la cohérence des agents. Nous expliquons les étapes principales pour transformer ces réseaux en format logique. Enfin, nous décrivons une extension pour représenter des préférences nuancées et fournissons des algorithmes pour les requêtes d'optimisation et de dominance.Structured modeling of preference statements, grounded in the notions of preferential independence, has tremendous potential to provide efficient approaches for modeling and reasoning about decision maker preferences in real-life applications. This thesis raises the question of representing preferences through a graphical structure. We propose a new reading of possibilistic networks, that we call p-pref nets, where possibility weights represent satisfaction degrees. The approach uses non-instantiated possibility weights, which we call symbolic weights, to define conditional preference tables. These conditional preference tables give birth to vectors of symbolic weights that reflect the preferences that are satisfied and those that are violated in a considered situation. We then focus on the theoretical aspects of handling of these vectors. Indeed, the comparison of such vectors may rely on different orderings: the ones induced by the product-based, or the minimum based chain rule underlying the possibilistic network, the discrimin, or leximin refinements of the minimum- based ordering, as well as Pareto ordering, and the symmetric Pareto ordering that refines it. We prove that the product-based comparison corresponds exactly to symmetric Pareto and we focus on its assets compared to the other ordering methods. Besides, we show that productbased ordering is consistent with the ordering obtained by comparing sets of satisfied preference tables. The picture is then completed by the proposition of algorithms for handling optimization and dominance queries. In this work we discuss various graphical tools for preference representation. We shed light particularly on CP-nets since they share the same graphical structure as p-pref nets and are based on the same preference statements. We prove that the CP-net orderings cannot contradict those of the p-pref nets and we found suitable additional constraints to refine p-pref net orderings in order to capture Ceteris Paribus constraints of CP-nets. This indicates that CP-nets potentially represent a subclass of p-pref nets with constraints. Finally, we provide an thorough comparison between the different qualitative and quantitative graphical models and p-pref nets. We deduce that the latter can be positioned halfway between qualitative and quantitative models since they do not need a full instantiation of the symbolic weights while additional information about the relative strengths of these weights can be taken into account. The last part of this work is dedicated to extent the proposed model to represent multiple agents preferences. As a first step, we propose the use of possibilistic networks for representing all or nothing multiple agents preferences and define conditioning in the case of Boolean possibilities. These multiple agents networks have a logical counterpart helpful for checking agents consistency. We explain the main steps for transforming multiple agents networks into logical format. Finally, we outline an extension with priority levels of these networks and provide algorithms for handling optimization and dominance queries

Fuzzy decision making: A bibliometric-based review

Fuzzy decision-making consists in making decisions under complex and uncertain environments where the information can be assessed with fuzzy sets and systems. The aim of this study is to review the main contributions in this field by using a bibliometric approach. For doing so, the article uses a wide range of bibliometric indicators including the citations and the h-index. Moreover, it also uses the VOS viewer software in order to map the main trends in this area. The work considers the leading journals, articles, authors and institutions. The results indicate that the USA was the traditional leader in this field with the most significant researcher. However, during the last years, this field is receiving more attention by Asian authors that are starting to lead the field. This discipline has a strong potential and the expectations for the future is that it will continue to grow

Generalized belief change with imprecise probabilities and graphical models

We provide a theoretical investigation of probabilistic belief revision in complex frameworks, under extended conditions of uncertainty, inconsistency and imprecision. We motivate our kinematical approach by specializing our discussion to probabilistic reasoning with graphical models, whose modular representation allows for efficient inference. Most results in this direction are derived from the relevant work of Chan and Darwiche (2005), that first proved the inter-reducibility of virtual and probabilistic evidence. Such forms of information, deeply distinct in their meaning, are extended to the conditional and imprecise frameworks, allowing further generalizations, e.g. to experts' qualitative assessments. Belief aggregation and iterated revision of a rational agent's belief are also explored

ISIPTA'07: Proceedings of the Fifth International Symposium on Imprecise Probability: Theories and Applications

B

Regularized model learning in EDAs for continuous and multi-objective optimization

Probabilistic modeling is the de�ning characteristic of estimation of distribution algorithms (EDAs) which determines their behavior and performance in optimization. Regularization is a well-known statistical technique used for obtaining an improved model by reducing the generalization error of estimation, especially in high-dimensional problems. `1-regularization is a type of this technique with the appealing variable selection property which results in sparse model estimations. In this thesis, we study the use of regularization techniques for model learning in EDAs. Several methods for regularized model estimation in continuous domains based on a Gaussian distribution assumption are presented, and analyzed from di�erent aspects when used for optimization in a high-dimensional setting, where the population size of EDA has a logarithmic scale with respect to the number of variables. The optimization results obtained for a number of continuous problems with an increasing number of variables show that the proposed EDA based on regularized model estimation performs a more robust optimization, and is able to achieve signi�cantly better results for larger dimensions than other Gaussian-based EDAs. We also propose a method for learning a marginally factorized Gaussian Markov random �eld model using regularization techniques and a clustering algorithm. The experimental results show notable optimization performance on continuous additively decomposable problems when using this model estimation method. Our study also covers multi-objective optimization and we propose joint probabilistic modeling of variables and objectives in EDAs based on Bayesian networks, speci�cally models inspired from multi-dimensional Bayesian network classi�ers. It is shown that with this approach to modeling, two new types of relationships are encoded in the estimated models in addition to the variable relationships captured in other EDAs: objectivevariable and objective-objective relationships. An extensive experimental study shows the e�ectiveness of this approach for multi- and many-objective optimization. With the proposed joint variable-objective modeling, in addition to the Pareto set approximation, the algorithm is also able to obtain an estimation of the multi-objective problem structure. Finally, the study of multi-objective optimization based on joint probabilistic modeling is extended to noisy domains, where the noise in objective values is represented by intervals. A new version of the Pareto dominance relation for ordering the solutions in these problems, namely �-degree Pareto dominance, is introduced and its properties are analyzed. We show that the ranking methods based on this dominance relation can result in competitive performance of EDAs with respect to the quality of the approximated Pareto sets. This dominance relation is then used together with a method for joint probabilistic modeling based on `1-regularization for multi-objective feature subset selection in classi�cation, where six di�erent measures of accuracy are considered as objectives with interval values. The individual assessment of the proposed joint probabilistic modeling and solution ranking methods on datasets with small-medium dimensionality, when using two di�erent Bayesian classi�ers, shows that comparable or better Pareto sets of feature subsets are approximated in comparison to standard methods

Improving Data Quality, Model Functionalities and Optimizing User Interfaces in Decision Support Systems

This dissertation contributes to the research on three core elements of decision support systems for managers and consumers: data management, model management and user interface. With respect to data management this dissertation proposes an approach for reducing unobserved product heterogeneity in online transaction data sets. The example of an online auction data set is used to investigate the approach’s ability to improve data quality. In the area of model management this dissertation contributes an approach to elicit consumer product preferences for exponential (beside linear) utility functions aiming at predicting consumers’ utilities and willingness-to-pay for individual products. The question which utility function (linear or exponential) is better suited for predicting product utilities and the willingness to pay is evaluated using a laboratory experiment. Further, in the area of user interfaces this dissertation deals with information visualization. Focusing on coordinate systems, a laboratory experiment is used to investigate which visualization format (two or three dimensional) is better suited for supporting simple vs. complex decision making scenarios and which criteria matter when choosing a visualization format for a particular level of decision making complexity