In All Likelihood, Deep Belief Is Not Enough

Statistical models of natural stimuli provide an important tool for researchers in the fields of machine learning and computational neuroscience. A canonical way to quantitatively assess and compare the performance of statistical models is given by the likelihood. One class of statistical models which has recently gained increasing popularity and has been applied to a variety of complex data are deep belief networks. Analyses of these models, however, have been typically limited to qualitative analyses based on samples due to the computationally intractable nature of the model likelihood. Motivated by these circumstances, the present article provides a consistent estimator for the likelihood that is both computationally tractable and simple to apply in practice. Using this estimator, a deep belief network which has been suggested for the modeling of natural image patches is quantitatively investigated and compared to other models of natural image patches. Contrary to earlier claims based on qualitative results, the results presented in this article provide evidence that the model under investigation is not a particularly good model for natural image

Estimating Dependency Structure as a Hidden Variable

This paper introduces a probability model, the mixture of trees that can account for sparse, dynamically changing dependence relationships. We present a family of efficient algorithms that use EM and the Minimum Spanning Tree algorithm to find the ML and MAP mixture of trees for a variety of priors, including the Dirichlet and the MDL priors. We also show that the single tree classifier acts like an implicit feature selector, thus making the classification performance insensitive to irrelevant attributes. Experimental results demonstrate the excellent performance of the new model both in density estimation and in classification

Independent component analysis applications in CDMA systems

Thesis (Master)--Izmir Institute of Technology, Electronics and Communication Engineering, Izmir, 2004Includes bibliographical references (leaves: 56)Text in English; Abstract: Turkish and Englishxi, 96 leavesBlind source separation (BSS) methods, independent component analysis (ICA) and independent factor analysis (IFA) are used for detecting the signal coming to a mobile user which is subject to multiple access interference in a CDMA downlink communication. When CDMA models are studied for different channel characteristics, it is seen that they are similar with BSS/ICA models. It is also showed that if ICA is applied to these CDMA models, desired user.s signal can be estimated successfully without channel information and other users. code sequences. ICA detector is compared with matched filter detector and other conventional detectors using simulation results and it is seen that ICA has some advantages over the other methods.The other BSS method, IFA is applied to basic CDMA downlink model. Since IFA has some convergence and speed problems when the number of sources is large, firstly basic CDMA model with ideal channel assumption is used in IFA application.With simulation of ideal CDMA channel, IFA is compared with ICA and matched filter.Furthermore, Pearson System-based ICA (PS-ICA) method is used forestimating non-Gaussian multipath fading channel coefficients. Considering some fading channel measurements showing that the fading channel coefficients may have an impulsive nature, these coefficients are modeled with an -stable distribution whose shape parameter takes values close to 2 which makes the distributions slightly impulsive. Simulation results are obtained to compare PS-ICA with classical ICA.Also IFA is applied to the single path CDMA downlink model to estimate fading channel by using the advantage of IFA which is the capability to estimate sources with wide class of distributions

A Review of Inference Algorithms for Hybrid Bayesian Networks

Hybrid Bayesian networks have received an increasing attention during the last years. The difference with respect to standard Bayesian networks is that they can host discrete and continuous variables simultaneously, which extends the applicability of the Bayesian network framework in general. However, this extra feature also comes at a cost: inference in these types of models is computationally more challenging and the underlying models and updating procedures may not even support closed-form solutions. In this paper we provide an overview of the main trends and principled approaches for performing inference in hybrid Bayesian networks. The methods covered in the paper are organized and discussed according to their methodological basis. We consider how the methods have been extended and adapted to also include (hybrid) dynamic Bayesian networks, and we end with an overview of established software systems supporting inference in these types of models