Variational Elliptical Processes

We present elliptical processes, a family of non-parametric probabilistic models that subsume Gaussian processes and Student's t processes. This generalization includes a range of new heavy-tailed behaviors while retaining computational tractability. Elliptical processes are based on a representation of elliptical distributions as a continuous mixture of Gaussian distributions. We parameterize this mixture distribution as a spline normalizing flow, which we train using variational inference. The proposed form of the variational posterior enables a sparse variational elliptical process applicable to large-scale problems. We highlight advantages compared to Gaussian processes through regression and classification experiments. Elliptical processes can supersede Gaussian processes in several settings, including cases where the likelihood is non-Gaussian or when accurate tail modeling is essential.Comment: 14 pages, 15 figures, appendix 9 page

Bayesian Modeling of Directional Data with Acoustic and Other Applications

A direction is defined here as a multi-dimensional unit vector. Such unitvectors form directional data. Closely related to directional data are axialdata for which each direction is equivalent to the opposite direction.Directional data and axial data arise in various fields of science. In probabilisticmodeling of such data, probability distributions are needed whichcount for the structure of the space from which data samples are collected.Such distributions are known as directional distributions and axial distributions.This thesis studies the von Mises-Fisher (vMF) distribution and the(complex) Watson distribution as representatives of directional and axialdistributions.Probabilistic models of the data are defined through a set of parameters.In the Bayesian view to uncertainty, these parameters are regarded as randomvariables in the learning inference. The primary goal of this thesis is todevelop Bayesian inference for directional and axial models, more precisely,vMF and (complex) Watson distributions, and parametric mixture modelsof such distributions. The Bayesian inference is realized using a family ofoptimization methods known as variational inference. With the proposedvariational methods, the intractable Bayesian inference problem is cast asan optimization problem.The variational inference for vMF andWatson models shall open up newapplications and advance existing application domains by reducing restrictiveassumptions made by current modelling techniques. This is the centraltheme of the thesis in all studied applications. Unsupervised clustering ofgene-expression and gene-microarray data is an existing application domain,which has been further advanced in this thesis. This thesis also advancesapplication of the complex Watson models in the problem of blind sourceseparation (BSS) with acoustic applications. Specifically, it is shown thatthe restrictive assumption of prior knowledge on the true number of sourcescan be relaxed by the desirable pruning property in Bayesian learning, resultingin BSS methods which can estimate the number of sources.Furthermore, this thesis introduces a fully Bayesian recursive frameworkfor the BSS task. This is an attempt toward realization of an online BSSmethod. In order to reduce the well-known problem of permutation ambiguityin the frequency domain, the complete BSS problem is solved in one unified modeling step, combining the frequency bin-wise source estimationwith the permutation problem. To realize this, all time frames and frequencybins are connected using a first order Markov chain. The model cancapture dependencies across both time frames and frequency bins, simultaneously,using a feed-forward two-dimensional hidden Markov model (2-DHMM).QC 20141009</p

Bayesian Modeling of Directional Data with Acoustic and Other Applications

A direction is defined here as a multi-dimensional unit vector. Such unitvectors form directional data. Closely related to directional data are axialdata for which each direction is equivalent to the opposite direction.Directional data and axial data arise in various fields of science. In probabilisticmodeling of such data, probability distributions are needed whichcount for the structure of the space from which data samples are collected.Such distributions are known as directional distributions and axial distributions.This thesis studies the von Mises-Fisher (vMF) distribution and the(complex) Watson distribution as representatives of directional and axialdistributions.Probabilistic models of the data are defined through a set of parameters.In the Bayesian view to uncertainty, these parameters are regarded as randomvariables in the learning inference. The primary goal of this thesis is todevelop Bayesian inference for directional and axial models, more precisely,vMF and (complex) Watson distributions, and parametric mixture modelsof such distributions. The Bayesian inference is realized using a family ofoptimization methods known as variational inference. With the proposedvariational methods, the intractable Bayesian inference problem is cast asan optimization problem.The variational inference for vMF andWatson models shall open up newapplications and advance existing application domains by reducing restrictiveassumptions made by current modelling techniques. This is the centraltheme of the thesis in all studied applications. Unsupervised clustering ofgene-expression and gene-microarray data is an existing application domain,which has been further advanced in this thesis. This thesis also advancesapplication of the complex Watson models in the problem of blind sourceseparation (BSS) with acoustic applications. Specifically, it is shown thatthe restrictive assumption of prior knowledge on the true number of sourcescan be relaxed by the desirable pruning property in Bayesian learning, resultingin BSS methods which can estimate the number of sources.Furthermore, this thesis introduces a fully Bayesian recursive frameworkfor the BSS task. This is an attempt toward realization of an online BSSmethod. In order to reduce the well-known problem of permutation ambiguityin the frequency domain, the complete BSS problem is solved in one unified modeling step, combining the frequency bin-wise source estimationwith the permutation problem. To realize this, all time frames and frequencybins are connected using a first order Markov chain. The model cancapture dependencies across both time frames and frequency bins, simultaneously,using a feed-forward two-dimensional hidden Markov model (2-DHMM).QC 20141009</p

Bayesian Modeling of Directional Data with Acoustic and Other Applications

A direction is defined here as a multi-dimensional unit vector. Such unitvectors form directional data. Closely related to directional data are axialdata for which each direction is equivalent to the opposite direction.Directional data and axial data arise in various fields of science. In probabilisticmodeling of such data, probability distributions are needed whichcount for the structure of the space from which data samples are collected.Such distributions are known as directional distributions and axial distributions.This thesis studies the von Mises-Fisher (vMF) distribution and the(complex) Watson distribution as representatives of directional and axialdistributions.Probabilistic models of the data are defined through a set of parameters.In the Bayesian view to uncertainty, these parameters are regarded as randomvariables in the learning inference. The primary goal of this thesis is todevelop Bayesian inference for directional and axial models, more precisely,vMF and (complex) Watson distributions, and parametric mixture modelsof such distributions. The Bayesian inference is realized using a family ofoptimization methods known as variational inference. With the proposedvariational methods, the intractable Bayesian inference problem is cast asan optimization problem.The variational inference for vMF andWatson models shall open up newapplications and advance existing application domains by reducing restrictiveassumptions made by current modelling techniques. This is the centraltheme of the thesis in all studied applications. Unsupervised clustering ofgene-expression and gene-microarray data is an existing application domain,which has been further advanced in this thesis. This thesis also advancesapplication of the complex Watson models in the problem of blind sourceseparation (BSS) with acoustic applications. Specifically, it is shown thatthe restrictive assumption of prior knowledge on the true number of sourcescan be relaxed by the desirable pruning property in Bayesian learning, resultingin BSS methods which can estimate the number of sources.Furthermore, this thesis introduces a fully Bayesian recursive frameworkfor the BSS task. This is an attempt toward realization of an online BSSmethod. In order to reduce the well-known problem of permutation ambiguityin the frequency domain, the complete BSS problem is solved in one unified modeling step, combining the frequency bin-wise source estimationwith the permutation problem. To realize this, all time frames and frequencybins are connected using a first order Markov chain. The model cancapture dependencies across both time frames and frequency bins, simultaneously,using a feed-forward two-dimensional hidden Markov model (2-DHMM).QC 20141009</p

Subband-based Single-channel Source Separation of Instantaneous Audio Mixtures

In this paper, a new algorithm is developed to separate the audio sources from a single instantaneous mixture. The algorithm is based on subband decomposition and uses a hybrid system of Empirical Mode Decomposition (EMD) and Principle Component Analysis (PCA) to construct artificial observations from the single mixture. In the separation stage of algorithm, we use Independent Component Analysis (ICA) to find independent components. At first the observed mixture is divided into a finite number of subbands through filtering with a parallel bank of FIR band-pass filters. Then EMD is employed to extract Intrinsic Mode Functions (IMFs) in each subband. By applying PCA to the extracted components, we find uncorrelated components which are the artificial observations. Then we obtain independent components by applying Independent Component Analysis (ICA) to the uncorrelated components. Finally, we carry out subband synthesis process to reconstruct fullband separated signals. The experimental results substantiate that the proposed method truly performs the task of source separation from a single instantaneous mixture.QC 2011122