On the construction of the inclusion boundary neighbourhood for markov equivalence classes of bayesian network structures

peer reviewedThe problem of learning Markov equivalence classes of Bayesian network structures may be solved by searching for the maximum of a scoring metric in a space of these classes. This paper deals with the definition and analysis of one such search space. We use a theoretically motivated neighbourhood, the inclusion boundary, and represent equivalence classes by essential graphs. We show that this search space is connected and that the score of the neighbours can be evaluated incrementally. We devise a practical way of building this neighbourhood for an essential graph that is purely graphical and does not explicitely refer to the underlying independences. We find that its size can be intractable, depending on the complexity of the essential graph of the equivalence class. The emphasis is put on the potential use of this space with greedy hillclimbing search

Towards an inclusion driven learning of Bayesian Networks

Two or more Bayesian Networks are Markov equivalent when their corresponding acyclic digraphs encode the same set of conditional independence (= CI) restrictions. Therefore, the search space of Bayesian Networks may be organized in classes of equivalence, where each of them consists of a particular set of CI restrictions. The collection of sets of CI restrictions obeys a partial order, the graphical Markov model inclusion partial order, or inclusion order for short. This paper discusses in depth the role that inclusion order plays in learning the structure of Bayesian networks. We prove that under very special conditions the traditional hill-climber always recovers the right structure. Moreover, we extend the recent experimental results presented in (Kocka and Castelo, 2001). We show how learning algorithms for Bayesian Networks, that take the inclusion order into account, perform better than those that do not, and we introduce two new ones in the context of heuristic search and the MCMC method

Hyper and structural Markov laws for graphical models

My thesis focuses on the parameterisation and estimation of graphical models, based on the concept of hyper and meta Markov properties. These state that the parameters should exhibit conditional independencies, similar to those on the sample space. When these properties are satisfied, parameter estimation may be performed locally, i.e. the estimators for certain subsets of the graph are determined entirely by the data corresponding to the subset. Firstly, I discuss the applications of these properties to the analysis of case-control studies. It has long been established that the maximum likelihood estimates for the odds-ratio may be found by logistic regression, in other words, the "incorrect" prospective model is equivalent to the correct retrospective model. I use a generalisation of the hyper Markov properties to identify necessary and sufficient conditions for the corresponding result in a Bayesian analysis, that is, the posterior distribution for the odds-ratio is the same under both the prospective and retrospective likelihoods. These conditions can be used to derive a parametric family of prior laws that may be used for such an analysis. The second part focuses on the problem of inferring the structure of the underlying graph. I propose an extension of the meta and hyper Markov properties, which I term structural Markov properties, for both undirected decomposable graphs and directed acyclic graphs. Roughly speaking, it requires that the structure of distinct components of the graph are conditionally independent given the existence of a separating component. This allows the analysis and comparison of multiple graphical structures, while being able to take advantage of the common conditional independence constraints. Moreover, I show that these properties characterise exponential families, which form conjugate priors under sampling from compatible Markov distributions

Complexity analysis of Bayesian learning of high-dimensional DAG models and their equivalence classes

We consider MCMC methods for learning equivalence classes of sparse Gaussian DAG models when $p = e^{o(n)}$. The main contribution of this work is a rapid mixing result for a random walk Metropolis-Hastings algorithm, which we prove using a canonical path method. It reveals that the complexity of Bayesian learning of sparse equivalence classes grows only polynomially in $n$ and $p$, under some common high-dimensional assumptions. Further, a series of high-dimensional consistency results is obtained by the path method, including the strong selection consistency of an empirical Bayes model for structure learning and the consistency of a greedy local search on the restricted search space. Rapid mixing and slow mixing results for other structure-learning MCMC methods are also derived. Our path method and mixing time results yield crucial insights into the computational aspects of high-dimensional structure learning, which may be used to develop more efficient MCMC algorithms

VPRS-based regional decision fusion of CNN and MRF classifications for very fine resolution remotely sensed images

Recent advances in computer vision and pattern recognition have demonstrated the superiority of deep neural networks using spatial feature representation, such as convolutional neural networks (CNN), for image classification. However, any classifier, regardless of its model structure (deep or shallow), involves prediction uncertainty when classifying spatially and spectrally complicated very fine spatial resolution (VFSR) imagery. We propose here to characterise the uncertainty distribution of CNN classification and integrate it into a regional decision fusion to increase classification accuracy. Specifically, a variable precision rough set (VPRS) model is proposed to quantify the uncertainty within CNN classifications of VFSR imagery, and partition this uncertainty into positive regions (correct classifications) and non-positive regions (uncertain or incorrect classifications). Those “more correct” areas were trusted by the CNN, whereas the uncertain areas were rectified by a Multi-Layer Perceptron (MLP)-based Markov random field (MLP-MRF) classifier to provide crisp and accurate boundary delineation. The proposed MRF-CNN fusion decision strategy exploited the complementary characteristics of the two classifiers based on VPRS uncertainty description and classification integration. The effectiveness of the MRF-CNN method was tested in both urban and rural areas of southern England as well as Semantic Labelling datasets. The MRF-CNN consistently outperformed the benchmark MLP, SVM, MLP-MRF and CNN and the baseline methods. This research provides a regional decision fusion framework within which to gain the advantages of model-based CNN, while overcoming the problem of losing effective resolution and uncertain prediction at object boundaries, which is especially pertinent for complex VFSR image classification