A Review of Inference Algorithms for Hybrid Bayesian Networks

Hybrid Bayesian networks have received an increasing attention during the last years. The difference with respect to standard Bayesian networks is that they can host discrete and continuous variables simultaneously, which extends the applicability of the Bayesian network framework in general. However, this extra feature also comes at a cost: inference in these types of models is computationally more challenging and the underlying models and updating procedures may not even support closed-form solutions. In this paper we provide an overview of the main trends and principled approaches for performing inference in hybrid Bayesian networks. The methods covered in the paper are organized and discussed according to their methodological basis. We consider how the methods have been extended and adapted to also include (hybrid) dynamic Bayesian networks, and we end with an overview of established software systems supporting inference in these types of models

Compatible and incompatible abstractions in Bayesian networks

The graphical structure of a Bayesian network (BN) makes it a technology well-suited for developing decision support models from a combination of domain knowledge and data. The domain knowledge of experts is used to determine the graphical structure of the BN, corresponding to the relationships and between variables, and data is used for learning the strength of these relationships. However, the available data seldom match the variables in the structure that is elicited from experts, whose models may be quite detailed; consequently, the structure needs to be abstracted to match the data. Up to now, this abstraction has been informal, loosening the link between the final model and the experts' knowledge. In this paper, we propose a method for abstracting the BN structure by using four 'abstraction' operations: node removal, node merging, state-space collapsing and edge removal. Some of these steps introduce approximations, which can be identified from changes in the set of conditional independence (CI) assertions of a network

Bounding probabilistic relationships in Bayesian networks using qualitative influences: methods and applications

AbstractWe present conditions under which one can bound the probabilistic relationships between random variables in a Bayesian network by exploiting known or induced qualitative relationships. Generic strengthening and weakening operations produce bounds on cumulative distributions, and the directions of these bounds are maintained through qualitative influences. We show how to incorporate these operations in a state-space abstraction method, so that bounds provably tighten as an approximate network is refined. We apply these techniques to qualitative tradeoff resolution demonstrating an ability to identify qualitative relationships among random variables without exhaustively using the probabilistic information encoded in the given network. In an application to path planning, we present an anytime algorithm with run-time computable error bounds

Inference in Bayesian Networks

Tato diplomová práce se zabývá demonstrací různých přístupů k inferencím v Bayesovských sítích. V teoretické části jsou rozebrány základy pravděpodobnosti, základy teorie Bayesovkých sítí, inferenční metody a oblasti aplikací Bayesovských sítí. Inferenční metody jsou krátce představeny a doplněny jejich algoritmem. Princip každé metody je uveden na příkladu. Praktická část obsahuje popis implementace, experimenty s demonstračními aplikacemi a shrnutí dosažených výsledků.This master's thesis deals with demonstration of various approaches to probabilistic inference in Bayesian networks. Basics of probability theory, introduction to Bayesian networks, methods for Bayesian inference and applications of Bayesian networks are described in theoretical part. Inference techniques are explained and complemented by their algorithm. Techniques are also illustrated on example. Practical part contains implementation description, experiments with demonstration applications and conclusion of the results.

THEORETICAL AND PRACTICAL ASPECTS OF DECISION SUPPORT SYSTEMS BASED ON THE PRINCIPLES OF QUERY-BASED DIAGNOSTICS

Diagnosis has been traditionally one of the most successful applications of Bayesian networks. The main bottleneck in applying Bayesian networks to diagnostic problems seems to be model building, which is typically a complex and time consuming task. Query-based diagnostics offers passive, incremental construction of diagnostic models that rest on the interaction between a diagnostician and a computer-based diagnostic system. Every case, passively observed by the system, adds information and, in the long run, leads to construction of a usable model. This approach minimizes knowledge engineering in model building. This dissertation focuses on theoretical and practical aspects of building systems based on the idea of query-based diagnostics. Its main contributions are an investigation of the optimal approach to learning parameters of Bayesian networks from continuous data streams, dealing with structural complexity in building Bayesian networks through removal of the weakest arcs, and a practical evaluation of the idea of query-based diagnostics. One of the main problems of query-based diagnostic systems is dealing with complexity. As data comes in, the models constructed may become too large and too densely connected. I address this problem in two ways. First, I present an empirical comparison of Bayesian network parameter learning algorithms. This study provides the optimal solutions for the system when dealing with continuous data streams. Second, I conduct a series of experiments testing control of the growth of a model by means of removing its weakest arcs. The results show that removing up to 20 percent of the weakest arcs in a network has minimal effect on its classification accuracy, and reduces the amount of memory taken by the clique tree and by this the amount of computation needed to perform inference. An empirical evaluation of query-based diagnostic systems shows that the diagnostic accuracy reaches reasonable levels after merely tens of cases and continues to increase with the number of cases, comparing favorably to state of the art approaches based on learning

Online Spectral Clustering on Network Streams

Graph is an extremely useful representation of a wide variety of practical systems in data analysis. Recently, with the fast accumulation of stream data from various type of networks, significant research interests have arisen on spectral clustering for network streams (or evolving networks). Compared with the general spectral clustering problem, the data analysis of this new type of problems may have additional requirements, such as short processing time, scalability in distributed computing environments, and temporal variation tracking. However, to design a spectral clustering method to satisfy these requirements certainly presents non-trivial efforts. There are three major challenges for the new algorithm design. The first challenge is online clustering computation. Most of the existing spectral methods on evolving networks are off-line methods, using standard eigensystem solvers such as the Lanczos method. It needs to recompute solutions from scratch at each time point. The second challenge is the parallelization of algorithms. To parallelize such algorithms is non-trivial since standard eigen solvers are iterative algorithms and the number of iterations can not be predetermined. The third challenge is the very limited existing work. In addition, there exists multiple limitations in the existing method, such as computational inefficiency on large similarity changes, the lack of sound theoretical basis, and the lack of effective way to handle accumulated approximate errors and large data variations over time. In this thesis, we proposed a new online spectral graph clustering approach with a family of three novel spectrum approximation algorithms. Our algorithms incrementally update the eigenpairs in an online manner to improve the computational performance. Our approaches outperformed the existing method in computational efficiency and scalability while retaining competitive or even better clustering accuracy. We derived our spectrum approximation techniques GEPT and EEPT through formal theoretical analysis. The well established matrix perturbation theory forms a solid theoretic foundation for our online clustering method. We facilitated our clustering method with a new metric to track accumulated approximation errors and measure the short-term temporal variation. The metric not only provides a balance between computational efficiency and clustering accuracy, but also offers a useful tool to adapt the online algorithm to the condition of unexpected drastic noise. In addition, we discussed our preliminary work on approximate graph mining with evolutionary process, non-stationary Bayesian Network structure learning from non-stationary time series data, and Bayesian Network structure learning with text priors imposed by non-parametric hierarchical topic modeling

Байєсівські мережі в системах підтримки прийняття рішень

Пропонується докладне висвітлення сучасних підходів до моделювання процесів довільної природи за допомогою байєсівських мереж (БМ) і дерев рішень. Байєсівська мережа – ймовірнісна модель, преставлена у формі спрямованого ациклічного графа, вершинами якого є змінні досліджуваного процесу. БМ – потужний сучасний інструмент моделювання процесів та об’єктів, які функціонують в умовах наявності невизначеностей довільної природи. Їх успішно використовують для розв’язання задач прогнозування, передбачення, медичної і технічної діагностики, прийняття управлінських рішень, автоматичного керування і т. ін. Розглянуто теорію побудови байєсівських мереж, яка включає задачі навчання структури мережі та формування ймовірнісного висновку на її основі. Наведено практичні методики побудови (оцінювання) структури мережі на основі статистичних даних і експертних оцінок. Докладно описано відповідні алгоритмічні процедури. Окремо розглянуто варіанти використання дискретних і неперервних змінних, а також можливості створення гібридної мережі. Наведено кілька методів обчислення ймовірнісного висновку за допомогою побудованої мережі, у тому числі методи формування точного і наближеного висновків. Докладно розглянуто приклади розв’язання практичних задач за допомогою мереж Байєса. Зокрема, задачі моделювання, прогнозування і розпізнавання образів. Наведено перелік відомих програмних продуктів та їх виробників для побудови та застосування байєсівських мереж, частина з яких є повністю доступними для використання у мережі Інтернет. Деякі системи можна доповнювати новими програмними модулями. Книга рекомендується як навчальний посібник для студентів, аспірантів та викладачів, а також для інженерів, які спеціалізуються у галузі розв’язання задач ймовірнісного математичного моделювання, прогнозування, передбачення і розпізнавання образів процесів довільної природи, інформація стосовно який представлена статистичними даними та експертними оцінками