Brief survey on computational solutions for Bayesian inference

In this paper, we present a brief review of research work attempting to tackle the issue of tractability in Bayesian inference, including an analysis of the applicability and trade-offs of each proposed solution. In recent years, the Bayesian approach has become increasingly popular, endowing autonomous systems with the ability to deal with uncertainty and incompleteness. However, these systems are also expected to be efficient, while Bayesian inference in general is known to be an NP-hard problem, making it paramount to develop approaches dealing with this complexity in order to allow the implementation of usable Bayesian solutions. Novel computational paradigms and also major developments in massively parallel computation technologies, such as multi-core processors, GPUs and FPGAs, provide us with an inkling of the roadmap in Bayesian computation for upcoming years

Testing and improving local adaptive importance sampling in LJF local-JT in multiply sectioned Bayesian networks

Multiply Sectioned Bayesian Network (MSBN) provides a model for probabilistic reasoning in multi-agent systems. The exact inference is costly and difficult to be applied in the context of MSBNs. So the approximate inference is used as an alternative. Recently, for reasoning in MSBNs, LJF-based Local Adaptive Importance Sampler (LLAIS) has been developed for approximate reasoning in MSBNs. However, the prototype of LLAIS is tested on Alarm Network (37 nodes). But further testing on larger networks has not been reported. In this thesis, LLAIS algorithm is tested on three large networks namely Hailfinder (56 nodes), Win95pts (76 nodes) and PathFinder (109 nodes), to measure for its reliability and scalability. The experiments done show that LLAIS without parameters tuned shows good convergence for Hailfinder and Win95pts but not for Pathfinder network. However, when the parameters are tuned the algorithm shows considerable improvement in its accuracy for all the three networks tested

Learning Bayesian network equivalence classes using ant colony optimisation

Bayesian networks have become an indispensable tool in the modelling of uncertain knowledge. Conceptually, they consist of two parts: a directed acyclic graph called the structure, and conditional probability distributions attached to each node known as the parameters. As a result of their expressiveness, understandability and rigorous mathematical basis, Bayesian networks have become one of the first methods investigated, when faced with an uncertain problem domain. However, a recurring problem persists in specifying a Bayesian network. Both the structure and parameters can be difficult for experts to conceive, especially if their knowledge is tacit.To counteract these problems, research has been ongoing, on learning both the structure and parameters of Bayesian networks from data. Whilst there are simple methods for learning the parameters, learning the structure has proved harder. Part ofthis stems from the NP-hardness of the problem and the super-exponential space of possible structures. To help solve this task, this thesis seeks to employ a relatively new technique, that has had much success in tackling NP-hard problems. This technique is called ant colony optimisation. Ant colony optimisation is a metaheuristic based on the behaviour of ants acting together in a colony. It uses the stochastic activity of artificial ants to find good solutions to combinatorial optimisation problems. In the current work, this method is applied to the problem of searching through the space of equivalence classes of Bayesian networks, in order to find a good match against a set of data. The system uses operators that evaluate potential modifications to a current state. Each of the modifications is scored and the results used to inform the search. In order to facilitate these steps, other techniques are also devised, to speed up the learning process. The techniques includeThe techniques are tested by sampling data from gold standard networks and learning structures from this sampled data. These structures are analysed using various goodnessof-fit measures to see how well the algorithms perform. The measures include structural similarity metrics and Bayesian scoring metrics. The results are compared in depth against systems that also use ant colony optimisation and other methods, including evolutionary programming and greedy heuristics. Also, comparisons are made to well known state-of-the-art algorithms and a study performed on a real-life data set. The results show favourable performance compared to the other methods and on modelling the real-life data

Байєсівські мережі в системах підтримки прийняття рішень

Пропонується докладне висвітлення сучасних підходів до моделювання процесів довільної природи за допомогою байєсівських мереж (БМ) і дерев рішень. Байєсівська мережа – ймовірнісна модель, преставлена у формі спрямованого ациклічного графа, вершинами якого є змінні досліджуваного процесу. БМ – потужний сучасний інструмент моделювання процесів та об’єктів, які функціонують в умовах наявності невизначеностей довільної природи. Їх успішно використовують для розв’язання задач прогнозування, передбачення, медичної і технічної діагностики, прийняття управлінських рішень, автоматичного керування і т. ін. Розглянуто теорію побудови байєсівських мереж, яка включає задачі навчання структури мережі та формування ймовірнісного висновку на її основі. Наведено практичні методики побудови (оцінювання) структури мережі на основі статистичних даних і експертних оцінок. Докладно описано відповідні алгоритмічні процедури. Окремо розглянуто варіанти використання дискретних і неперервних змінних, а також можливості створення гібридної мережі. Наведено кілька методів обчислення ймовірнісного висновку за допомогою побудованої мережі, у тому числі методи формування точного і наближеного висновків. Докладно розглянуто приклади розв’язання практичних задач за допомогою мереж Байєса. Зокрема, задачі моделювання, прогнозування і розпізнавання образів. Наведено перелік відомих програмних продуктів та їх виробників для побудови та застосування байєсівських мереж, частина з яких є повністю доступними для використання у мережі Інтернет. Деякі системи можна доповнювати новими програмними модулями. Книга рекомендується як навчальний посібник для студентів, аспірантів та викладачів, а також для інженерів, які спеціалізуються у галузі розв’язання задач ймовірнісного математичного моделювання, прогнозування, передбачення і розпізнавання образів процесів довільної природи, інформація стосовно який представлена статистичними даними та експертними оцінками

Importance Sampling for Bayesian Networks: Principles, Algorithms, and Performance

Bayesian networks (BNs) offer a compact, intuitive, and efficient graphical representation of uncertain relationships among the variables in a domain and have proven their value in many disciplines over the last two decades. However, two challenges become increasingly critical in practical applications of Bayesian networks. First, real models are reaching the size of hundreds or even thousands of nodes. Second, some decision problems are more naturally represented by hybrid models which contain mixtures ofdiscrete and continuous variables and may represent linear or nonlinear equations and arbitrary probability distributions. Both challenges make building Bayesian network models and reasoning withthem more and more difficult.In this dissertation, I address the challenges by developing representational and computational solutions based on importance sampling. I First develop a more solid understanding of the properties of importance sampling in the context of Bayesian networks. Then, I address a fundamental question of importance sampling in Bayesian networks, the representation of the importance function. I derive an exact representation for the optimal importance function and propose an approximation strategy for therepresentation when it is too complex. Based on these theoretical analysis, I propose a suite of importance sampling-based algorithms for (hybrid) Bayesian networks. I believe the new algorithmssignificantly extend the efficiency, applicability, and scalability of approximate inference methods for Bayesian networks. The ultimate goal of this research is to help users to solve difficult reasoning problems emerging from complex decision problems in the most general settings