Using Constraint Satisfaction Techniques and Variational Methods for Probabilistic Reasoning

RÉSUMÉ Cette thèse présente un certain nombre de contributions à la recherche pour la création de systèmes efficaces de raisonnement probabiliste sur les modèles graphiques de problèmes issus d'une variété d'applications scientifiques et d'ingénierie. Ce thème touche plusieurs sous-disciplines de l'intelligence artificielle. Généralement, la plupart de ces problèmes ont des modèles graphiques expressifs qui se traduisent par de grands réseaux impliquant déterminisme et des cycles, ce qui représente souvent un goulot d'étranglement pour tout système d'inférence probabiliste et affaiblit son exactitude ainsi que son évolutivité. Conceptuellement, notre recherche confirme les hypothèses suivantes. D'abord, les techniques de satisfaction de contraintes et méthodes variationnelles peuvent être exploitées pour obtenir des algorithmes précis et évolutifs pour l'inférence probabiliste en présence de cycles et de déterminisme. Deuxièmement, certaines parties intrinsèques de la structure du modèle graphique peuvent se révéler bénéfiques pour l'inférence probabiliste sur les grands modèles graphiques, au lieu de poser un défi important pour elle. Troisièmement, le re-paramétrage du modèle graphique permet d'ajouter à sa structure des caractéristiques puissantes qu'on peut utiliser pour améliorer l'inférence probabiliste. La première contribution majeure de cette thèse est la formulation d'une nouvelle approche de passage de messages (message-passing) pour inférer dans un graphe de facteurs étendu qui combine des techniques de satisfaction de contraintes et des méthodes variationnelles. Contrairement au message-passing standard, il formule sa structure sous forme d'étapes de maximisation de l'espérance variationnelle. Ainsi, on a de nouvelles règles de mise à jour des marginaux qui augmentent une borne inférieure à chaque mise à jour de manière à éviter le dépassement d'un point fixe. De plus, lors de l'étape d'espérance, nous mettons à profit les structures locales dans le graphe de facteurs en utilisant la cohérence d'arc généralisée pour effectuer une approximation de champ moyen variationnel. La deuxième contribution majeure est la formulation d'une stratégie en deux étapes qui utilise le déterminisme présent dans la structure du modèle graphique pour améliorer l'évolutivité du problème d'inférence probabiliste. Dans cette stratégie, nous prenons en compte le fait que si le modèle sous-jacent implique des contraintes inviolables en plus des préférences, alors c'est potentiellement un gaspillage d'allouer de la mémoire pour toutes les contraintes à l'avance lors de l'exécution de l'inférence. Pour éviter cela, nous commençons par la relaxation des préférences et effectuons l'inférence uniquement avec les contraintes inviolables. Cela permet d'éviter les calculs inutiles impliquant les préférences et de réduire la taille effective du réseau graphique. Enfin, nous développons une nouvelle famille d'algorithmes d'inférence par le passage de messages dans un graphe de facteurs étendus, paramétrées par un facteur de lissage (smoothing parameter). Cette famille permet d'identifier les épines dorsales (backbones) d'une grappe qui contient des solutions potentiellement optimales. Ces épines dorsales ne sont pas seulement des parties des solutions optimales, mais elles peuvent également être exploitées pour intensifier l'inférence MAP en les fixant de manière itérative afin de réduire les parties complexes jusqu'à ce que le réseau se réduise à un seul qui peut être résolu avec précision en utilisant une méthode MAP d'inférence classique. Nous décrivons ensuite des variantes paresseuses de cette famille d'algorithmes. Expérimentalement, une évaluation empirique approfondie utilisant des applications du monde réel démontre la précision, la convergence et l'évolutivité de l'ensemble de nos algorithmes et stratégies par rapport aux algorithmes d'inférence existants de l'état de l'art.----------ABSTRACT This thesis presents a number of research contributions pertaining to the theme of creating efficient probabilistic reasoning systems based on graphical models of real-world problems from relational domains. These models arise in a variety of scientific and engineering applications. Thus, the theme impacts several sub-disciplines of Artificial Intelligence. Commonly, most of these problems have expressive graphical models that translate into large probabilistic networks involving determinism and cycles. Such graphical models frequently represent a bottleneck for any probabilistic inference system and weaken its accuracy and scalability. Conceptually, our research here hypothesizes and confirms that: First, constraint satisfaction techniques and variational methods can be exploited to yield accurate and scalable algorithms for probabilistic inference in the presence of cycles and determinism. Second, some intrinsic parts of the structure of the graphical model can turn out to be beneficial to probabilistic inference on large networks, instead of posing a significant challenge to it. Third, the proper re-parameterization of the graphical model can provide its structure with characteristics that we can use to improve probabilistic inference. The first major contribution of this thesis is the formulation of a novel message-passing approach to inference in an extended factor graph that combines constraint satisfaction techniques with variational methods. In contrast to standard message-passing, it formulates the Message-Passing structure as steps of variational expectation maximization. Thus it has new marginal update rules that increase a lower bound at each marginal update in a way that avoids overshooting a fixed point. Moreover, in its expectation step, we leverage the local structures in the factor graph by using generalized arc consistency to perform a variational mean-field approximation. The second major contribution is the formulation of a novel two-stage strategy that uses the determinism present in the graphical model's structure to improve the scalability of probabilistic inference. In this strategy, we take into account the fact that if the underlying model involves mandatory constraints as well as preferences then it is potentially wasteful to allocate memory for all constraints in advance when performing inference. To avoid this, we start by relaxing preferences and performing inference with hard constraints only. This helps avoid irrelevant computations involving preferences, and reduces the effective size of the graphical network. Finally, we develop a novel family of message-passing algorithms for inference in an extended factor graph, parameterized by a smoothing parameter. This family allows one to find the ”backbones” of a cluster that involves potentially optimal solutions. The cluster's backbones are not only portions of the optimal solutions, but they also can be exploited for scaling MAP inference by iteratively fixing them to reduce the complex parts until the network is simplified into one that can be solved accurately using any conventional MAP inference method. We then describe lazy variants of this family of algorithms. One limiting case of our approach corresponds to lazy survey propagation, which in itself is novel method which can yield state of the art performance. We provide a thorough empirical evaluation using real-world applications. Our experiments demonstrate improvements to the accuracy, convergence and scalability of all our proposed algorithms and strategies over existing state-of-the-art inference algorithms

Incremental inference on higher-order probabilistic graphical models applied to constraint satisfaction problems

Thesis (PhD)--Stellenbosch University, 2022.ENGLISH ABSTRACT: Probabilistic graphical models (PGMs) are used extensively in the probabilistic reasoning domain. They are powerful tools for solving systems of complex relationships over a variety of probability distributions, such as medical and fault diagnosis, predictive modelling, object recognition, localisation and mapping, speech recognition, and language processing [5, 6, 7, 8, 9, 10, 11]. Furthermore, constraint satisfaction problems (CSPs) can be formulated as PGMs and solved with PGM inference techniques. However, the prevalent literature on PGMs shows that suboptimal PGM structures are primarily used in practice and a suboptimal formulation for constraint satisfaction PGMs. This dissertation aimed to improve the PGM literature through accessible algorithms and tools for improved PGM structures and inference procedures, specifically focusing on constraint satisfaction. To this end, this dissertation presents three published contributions to the current literature: a comparative study to compare cluster graph topologies to the prevalent factor graphs [1], an application of cluster graphs in land cover classification in the field of cartography [2], and a comprehensive integration of various aspects required to formulate CSPs as PGMs and an algorithm to solve this formulation for problems too complex for traditional PGM tools [3]. First, we present a means of formulating and solving graph colouring problems with probabilistic graphical models. In contrast to the prevailing literature that mostly uses factor graph configurations, we approach it from a cluster graph perspective, using the general-purpose cluster graph construction algorithm, LTRIP. Our experiments indicate a significant advantage for preferring cluster graphs over factor graphs, both in terms of accuracy as well as computational efficiency. Secondly, we use these tools to solve a practical problem: land cover classification. This process is complex due to measuring errors, inefficient algorithms, and low-quality data. We proposed a PGM approach to boost geospatial classifications from different sources and consider the effects of spatial distribution and inter-class dependencies (similarly to graph colouring). Our PGM tools were shown to be robust and were able to produce a diverse, feasible, and spatially-consistent land cover classification even in areas of incomplete and conflicting evidence. Lastly, in our third publication, we investigated and improved the PGM structures used for constraint satisfaction. It is known that tree-structured PGMs always result in an exact solution [12, p355], but is usually impractical for interesting problems due to exponential blow-up. We, therefore, developed the “purge-and merge” algorithm to incrementally approximate a tree-structured PGM. This algorithm iteratively nudges a malleable graph structure towards a tree structure by selectively merging factors. The merging process is designed to avoid exponential blow-up through sparse data structures from which redundancy is purged as the algorithm progresses. This algorithm is tested on constraint satisfaction puzzles such as Sudoku, Fill-a-pix, and Kakuro and manages to outperform other PGM-based approaches reported in the literature [13, 14, 15]. Overall, the research reported in this dissertation contributed to developing a more optimised approach for higher order probabilistic graphical models. Further studies should concentrate on applying purge-and-merge on problems closer to probabilistic reasoning than constraint satisfaction and report its effectiveness in that domain.AFRIKAANSE OPSOMMING: Grafiese waarskynlikheidsmodelle (PGM) word wyd gebruik vir komplekse waarskynlikheidsprobleme. Dit is kragtige gereedskap om sisteme van komplekse verhoudings oor ‘n versameling waarskynlikheidsverspreidings op te los, soos die mediese en foutdiagnoses, voorspellingsmodelle, objekherkenning, lokalisering en kartering, spraakherkenning en taalprosessering [5, 6, 7, 8, 9, 10, 11]. Voorts kan beperkingvoldoeningsprobleme (CSP) as PGM’s geformuleer word en met PGM gevolgtrekkingtegnieke opgelos word. Die heersende literatuur oor PGM’s toon egter dat sub-optimale PGM-strukture hoofsaaklik in die praktyk gebruik word en ‘n sub-optimale PGM-formulering vir CSP’s. Die doel met die verhandeling is om die PGM-literatuur deur toeganklike algoritmes en gereedskap vir verbeterde PGM-strukture en gevolgtrekking-prosedures te verbeter deur op CSP toepassings te fokus. Na aanleiding hiervan voeg die verhandeling drie gepubliseerde bydraes by die huidige literatuur: ‘n vergelykende studie om bundelgrafieke tot die heersende faktorgrafieke te vergelyk [1], ‘n praktiese toepassing vir die gebruik van bundelgrafieke in “land-cover”- klassifikasie in die kartografieveld [2] en ‘n omvattende integrasie van verskeie aspekte om CSP’s as PGM’s te formuleer en ‘n algoritme vir die formulering van probleme te kompleks vir tradisionele PGM-gereedskap [3] Eerstens bied ons ‘n wyse van formulering en die oplos van grafiekkleurprobleme met PGM’s. In teenstelling met die huidige literatuur wat meestal faktorgrafieke gebruik, benader ons dit van ‘n bundelgrafiek-perspektief deur die gebruik van die automatiese bundelgrafiekkonstruksie-algoritme, LTRIP. Ons eksperimente toon ‘n beduidende voorkeur vir bundelgrafieke teenoor faktorgrafieke, wat akku raatheid asook berekende doeltreffendheid betref. Tweedens gebruik ons die gereedskap om ‘n praktiese probleem op te los: “landcover”-klassifikasie. Die proses is kompleks weens metingsfoute, ondoeltreffende algoritmes en lae-gehalte data. Ons stel ‘n PGM-benadering voor om die georuimtelike klassifikasies van verskillende bronne te versterk, asook die uitwerking van ruimtelike verspreiding en interklas-afhanklikhede (soortgelyk aan grafiekkleurprobleme). Ons PGM-gereedskap is robuus en kon ‘n diverse, uitvoerbare en ruimtelik-konsekwente “land-cover”-klassifikasie selfs in gebiede van onvoltooide en konflikterende inligting bewys. Ten slotte het ons in ons derde publikasie die PGM-strukture vir CSP’s ondersoek en verbeter. Dit is bekend dat boomstrukture altyd tot ‘n eksakte oplossing lei [12, p355], maar is weens eksponensiële uitbreiding gewoonlik onprakties vir interessante probleme. Ons het gevolglik die algoritme, purge-and-merge, ontwikkel om inkrementeel ‘n boomstruktuur na te doen. Die algoritme hervorm ‘n bundelgrafiek stapsgewys in ‘n boomstruktuur deur faktore selektief te “merge”. Die saamsmeltproses is ontwerp om eksponensiële uitbreiding te vermy deur van yl datastrukture gebruik te maak waarvan die waarskeinlikheidsruimte ge-“purge” word namate die algoritme vorder. Die algoritme is getoets op CSP-speletjies soos Sudoku, Fill-a-pix en Kakuro en oortref ander PGM-gegronde benaderings waaroor in die literatuur verslag gedoen word [13, 14, 15]. In die geheel gesien, het die navorsing bygedra tot die ontwikkeling van ‘n meer geoptimaliseerde benadering vir hoër-orde PGM’s. Verdere studies behoort te fokus op die toepassing van purge-and-merge op probleme nader aan waarskynlikheidsredenasie-probleme as aan CSP’s en moet sy effektiwiteit in daar die domein rapporteer.Doctora

Analyzing Structured Scenarios by Tracking People and Their Limbs

The analysis of human activities is a fundamental problem in computer vision. Though complex, interactions between people and their environment often exhibit a spatio-temporal structure that can be exploited during analysis. This structure can be leveraged to mitigate the effects of missing or noisy visual observations caused, for example, by sensor noise, inaccurate models, or occlusion. Trajectories of people and their hands and feet, often sufficient for recognition of human activities, lead to a natural qualitative spatio-temporal description of these interactions. This work introduces the following contributions to the task of human activity understanding: 1) a framework that efficiently detects and tracks multiple interacting people and their limbs, 2) an event recognition approach that integrates both logical and probabilistic reasoning in analyzing the spatio-temporal structure of multi-agent scenarios, and 3) an effective computational model of the visibility constraints imposed on humans as they navigate through their environment. The tracking framework mixes probabilistic models with deterministic constraints and uses AND/OR search and lazy evaluation to efficiently obtain the globally optimal solution in each frame. Our high-level reasoning framework efficiently and robustly interprets noisy visual observations to deduce the events comprising structured scenarios. This is accomplished by combining First-Order Logic, Allen's Interval Logic, and Markov Logic Networks with an event hypothesis generation process that reduces the size of the ground Markov network. When applied to outdoor one-on-one basketball videos, our framework tracks the players and, guided by the game rules, analyzes their interactions with each other and the ball, annotating the videos with the relevant basketball events that occurred. Finally, motivated by studies of spatial behavior, we use a set of features from visibility analysis to represent spatial context in the interpretation of human spatial activities. We demonstrate the effectiveness of our representation on trajectories generated by humans in a virtual environment

IBIA: An Incremental Build-Infer-Approximate Framework for Approximate Inference of Partition Function

Exact computation of the partition function is known to be intractable, necessitating approximate inference techniques. Existing methods for approximate inference are slow to converge for many benchmarks. The control of accuracy-complexity trade-off is also non-trivial in many of these methods. We propose a novel incremental build-infer-approximate (IBIA) framework for approximate inference that addresses these issues. In this framework, the probabilistic graphical model is converted into a sequence of clique tree forests (SCTF) with bounded clique sizes. We show that the SCTF can be used to efficiently compute the partition function. We propose two new algorithms which are used to construct the SCTF and prove the correctness of both. The first is an algorithm for incremental construction of CTFs that is guaranteed to give a valid CTF with bounded clique sizes and the second is an approximation algorithm that takes a calibrated CTF as input and yields a valid and calibrated CTF with reduced clique sizes as the output. We have evaluated our method using several benchmark sets from recent UAI competitions and our results show good accuracies with competitive runtimes