Composing Deep Learning and Bayesian Nonparametric Methods

Recent progress in Bayesian methods largely focus on non-conjugate models featured with extensive use of black-box functions: continuous functions implemented with neural networks. Using deep neural networks, Bayesian models can reasonably fit big data while at the same time capturing model uncertainty. This thesis targets at a more challenging problem: how do we model general random objects, including discrete ones, using random functions? Our conclusion is: many (discrete) random objects are in nature a composition of Poisson processes and random functions}. Thus, all discreteness is handled through the Poisson process while random functions captures the rest complexities of the object. Thus the title: composing deep learning and Bayesian nonparametric methods. This conclusion is not a conjecture. In spacial cases such as latent feature models , we can prove this claim by working on infinite dimensional spaces, and that is how Bayesian nonparametric kicks in. Moreover, we will assume some regularity assumptions on random objects such as exchangeability. Then the representations will show up magically using representation theorems. We will see this two times throughout this thesis. One may ask: when a random object is too simple, such as a non-negative random vector in the case of latent feature models, how can we exploit exchangeability? The answer is to aggregate infinite random objects and map them altogether onto an infinite dimensional space. And then assume exchangeability on the infinite dimensional space. We demonstrate two examples of latent feature models by (1) concatenating them as an infinite sequence (Section 2,3) and (2) stacking them as a 2d array (Section 4). Besides, we will see that Bayesian nonparametric methods are useful to model discrete patterns in time series data. We will showcase two examples: (1) using variance Gamma processes to model change points (Section 5), and (2) using Chinese restaurant processes to model speech with switching speakers (Section 6). We also aware that the inference problem can be non-trivial in popular Bayesian nonparametric models. In Section 7, we find a novel solution of online inference for the popular HDP-HMM model

Clustering Hidden Markov Models With Variational Bayesian Hierarchical EM.

The hidden Markov model (HMM) is a broadly applied generative model for representing time-series data, and clustering HMMs attract increased interest from machine learning researchers. However, the number of clusters ( K ) and the number of hidden states ( S ) for cluster centers are still difficult to determine. In this article, we propose a novel HMM-based clustering algorithm, the variational Bayesian hierarchical EM algorithm, which clusters HMMs through their densities and priors and simultaneously learns posteriors for the novel HMM cluster centers that compactly represent the structure of each cluster. The numbers K and S are automatically determined in two ways. First, we place a prior on the pair (K,S) and approximate their posterior probabilities, from which the values with the maximum posterior are selected. Second, some clusters and states are pruned out implicitly when no data samples are assigned to them, thereby leading to automatic selection of the model complexity. Experiments on synthetic and real data demonstrate that our algorithm performs better than using model selection techniques with maximum likelihood estimation

A bayesian allocation model based approach to mixed membership stochastic blockmodels

Although detecting communities in networks has attracted considerable recent attention, estimating the number of communities is still an open problem. In this paper, we propose a model, which replicates the generative process of the mixed-membership stochastic block model (MMSB) within the generic allocation framework of Bayesian allocation model (BAM) and BAM-MMSB. In contrast to traditional blockmodels, BAM-MMSB considers the observations as Poisson counts generated by a base Poisson process and marks according to the generative process of MMSB. Moreover, the optimal number of communities for BAM-MMSB is estimated by computing the variational approximations of the marginal likelihood for each model order. Experiments on synthetic and real data sets show that the proposed approach promises a generalized model selection solution that can choose not only the model size but also the most appropriate decomposition.WOS:000750893600001Scopus - Affiliation ID: 60105072Science Citation Index ExpandedQ3-Q4Article; Early AccessUluslararası işbirliği ile yapılmayan - HAYIRŞubat2022YÖK - 2021-22YÖK - 2021-2

Robust and Accurate Point Set Registration with Generalized Bayesian Coherent Point Drift

Point set registration (PSR) is an essential problem in surgical navigation and image-guided surgery (IGS). It can help align the pre-operative volumetric images with the intra-operative surgical space. The performances of PSR are susceptible to noise and outliers, which are the cases in real-world surgical scenarios. In this paper, we provide a novel point set registration method that utilizes the features extracted from the PSs and can guarantee the convergence of the algorithm simultaneously. More specifically, we formulate the PSR with normal vectors by generalizing the bayesian coherent point drift (BCPD) into the six-dimension scenario. Our contributions can be summarized as follows. (1) The PSR problem with normal vectors is formulated by generalizing the Bayesian coherent point drift (BCPD) approach; (2) The updated parameters during the algorithm's iterations are given in closed-forms; (3) Extensive experiments have been done to verify the proposed approach and its significant improvements over the BCPD has been validated. We have validated our proposed registration approach on both the human femur model. Results demonstrate that our proposed method outperforms the state-of-the-art registration methods and the convergence is guaranteed at the same time