Learning the Structure of Deep Sparse Graphical Models

Deep belief networks are a powerful way to model complex probability distributions. However, learning the structure of a belief network, particularly one with hidden units, is difficult. The Indian buffet process has been used as a nonparametric Bayesian prior on the directed structure of a belief network with a single infinitely wide hidden layer. In this paper, we introduce the cascading Indian buffet process (CIBP), which provides a nonparametric prior on the structure of a layered, directed belief network that is unbounded in both depth and width, yet allows tractable inference. We use the CIBP prior with the nonlinear Gaussian belief network so each unit can additionally vary its behavior between discrete and continuous representations. We provide Markov chain Monte Carlo algorithms for inference in these belief networks and explore the structures learned on several image data sets.Comment: 20 pages, 6 figures, AISTATS 2010, Revise

An Alternative Prior Process for Nonparametric Bayesian Clustering

Prior distributions play a crucial role in Bayesian approaches to clustering. Two commonly-used prior distributions are the Dirichlet and Pitman-Yor processes. In this paper, we investigate the predictive probabilities that underlie these processes, and the implicit "rich-get-richer" characteristic of the resulting partitions. We explore an alternative prior for nonparametric Bayesian clustering -- the uniform process -- for applications where the "rich-get-richer" property is undesirable. We also explore the cost of this process: partitions are no longer exchangeable with respect to the ordering of variables. We present new asymptotic and simulation-based results for the clustering characteristics of the uniform process and compare these with known results for the Dirichlet and Pitman-Yor processes. We compare performance on a real document clustering task, demonstrating the practical advantage of the uniform process despite its lack of exchangeability over orderings

Evaluation methods for topic models

A natural evaluation metric for statistical topic models is the probability of held-out documents given a trained model. While exact computation of this probability is intractable, several estimators for this probability have been used in the topic modeling literature, including the harmonic mean method and empirical likelihood method. In this paper, we demonstrate experimentally that commonly-used methods are unlikely to accurately estimate the probability of held-out documents, and propose two alternative methods that are both accurate and efficient

Topic Modeling and Text Analysis for Qualitative Policy Research

This paper contributes to a critical methodological discussion that has direct ramifications for policy studies: how computational methods can be concretely incorporated into existing processes of textual analysis and interpretation without compromising scientific integrity. We focus on the computational method of topic modeling and investigate how it interacts with two larger families of qualitative methods: content and classification methods characterized by interest in words as communication units and discourse and representation methods characterized by interest in the meaning of communicative acts. Based on analysis of recent academic publications that have used topic modeling for textual analysis, our findings show that different mixed‐method research designs are appropriate when combining topic modeling with the two groups of methods. Our main concluding argument is that topic modeling enables scholars to apply policy theories and concepts to much larger sets of data. That said, the use of computational methods requires genuine understanding of these techniques to obtain substantially meaningful results. We encourage policy scholars to reflect carefully on methodological issues, and offer a simple heuristic to help identify and address critical points when designing a study using topic modeling.Peer reviewe

Artificial Consciousness and Artificial Ethics: Between Realism and Social Relationism

I compare a ‘realist’ with a ‘social–relational’ perspective on our judgments of the moral status of artificial agents (AAs). I develop a realist position according to which the moral status of a being—particularly in relation to moral patiency attribution—is closely bound up with that being’s ability to experience states of conscious satisfaction or suffering (CSS). For a realist, both moral status and experiential capacity are objective properties of agents. A social relationist denies the existence of any such objective properties in the case of either moral status or consciousness, suggesting that the determination of such properties rests solely upon social attribution or consensus. A wide variety of social interactions between us and various kinds of artificial agent will no doubt proliferate in future generations, and the social–relational view may well be right that the appearance of CSS features in such artificial beings will make moral role attribution socially prevalent in human–AA relations. But there is still the question of what actual CSS states a given AA is capable of undergoing, independently of the appearances. This is not just a matter of changes in the structure of social existence that seem inevitable as human–AA interaction becomes more prevalent. The social world is itself enabled and constrained by the physical world, and by the biological features of living social participants. Properties analogous to certain key features in biological CSS are what need to be present for nonbiological CSS. Working out the details of such features will be an objective scientific inquiry

Genome-Wide Association Studies of Serum Magnesium, Potassium, and Sodium Concentrations Identify Six Loci Influencing Serum Magnesium Levels

Magnesium, potassium, and sodium, cations commonly measured in serum, are involved in many physiological processes including energy metabolism, nerve and muscle function, signal transduction, and fluid and blood pressure regulation. To evaluate the contribution of common genetic variation to normal physiologic variation in serum concentrations of these cations, we conducted genome-wide association studies of serum magnesium, potassium, and sodium concentrations using ∼2.5 million genotyped and imputed common single nucleotide polymorphisms (SNPs) in 15,366 participants of European descent from the international CHARGE Consortium. Study-specific results were combined using fixed-effects inverse-variance weighted meta-analysis. SNPs demonstrating genome-wide significant (p<5×10−8) or suggestive associations (p<4×10−7) were evaluated for replication in an additional 8,463 subjects of European descent. The association of common variants at six genomic regions (in or near MUC1, ATP2B1, DCDC5, TRPM6, SHROOM3, and MDS1) with serum magnesium levels was genome-wide significant when meta-analyzed with the replication dataset. All initially significant SNPs from the CHARGE Consortium showed nominal association with clinically defined hypomagnesemia, two showed association with kidney function, two with bone mineral density, and one of these also associated with fasting glucose levels. Common variants in CNNM2, a magnesium transporter studied only in model systems to date, as well as in CNNM3 and CNNM4, were also associated with magnesium concentrations in this study. We observed no associations with serum sodium or potassium levels exceeding p<4×10−7. Follow-up studies of newly implicated genomic loci may provide additional insights into the regulation and homeostasis of human serum magnesium levels