Variational Inference in Nonconjugate Models

Mean-field variational methods are widely used for approximate posterior inference in many probabilistic models. In a typical application, mean-field methods approximately compute the posterior with a coordinate-ascent optimization algorithm. When the model is conditionally conjugate, the coordinate updates are easily derived and in closed form. However, many models of interest---like the correlated topic model and Bayesian logistic regression---are nonconjuate. In these models, mean-field methods cannot be directly applied and practitioners have had to develop variational algorithms on a case-by-case basis. In this paper, we develop two generic methods for nonconjugate models, Laplace variational inference and delta method variational inference. Our methods have several advantages: they allow for easily derived variational algorithms with a wide class of nonconjugate models; they extend and unify some of the existing algorithms that have been derived for specific models; and they work well on real-world datasets. We studied our methods on the correlated topic model, Bayesian logistic regression, and hierarchical Bayesian logistic regression

The Discrete Infinite Logistic Normal Distribution

We present the discrete infinite logistic normal distribution (DILN), a Bayesian nonparametric prior for mixed membership models. DILN is a generalization of the hierarchical Dirichlet process (HDP) that models correlation structure between the weights of the atoms at the group level. We derive a representation of DILN as a normalized collection of gamma-distributed random variables, and study its statistical properties. We consider applications to topic modeling and derive a variational inference algorithm for approximate posterior inference. We study the empirical performance of the DILN topic model on four corpora, comparing performance with the HDP and the correlated topic model (CTM). To deal with large-scale data sets, we also develop an online inference algorithm for DILN and compare with online HDP and online LDA on the Nature magazine, which contains approximately 350,000 articles.Comment: This paper will appear in Bayesian Analysis. A shorter version of this paper appeared at AISTATS 2011, Fort Lauderdale, FL, US

Optimal extension to Sobolev rough paths

We show that every $\mathbb{R}^d$-valued Sobolev path with regularity $\alpha$ and integrability $p$ can be lifted to a Sobolev rough path in the sense of T. Lyons provided $\alpha >1/p>0$. Moreover, we prove the existence of unique rough path lifts which are optimal w.r.t. strictly convex functionals among all possible rough path lifts given a Sobolev path. As examples, we consider the rough path lift with minimal Sobolev norm and characterize the Stratonovich rough path lift of a Brownian motion as optimal lift w.r.t. to a suitable convex functional. Generalizations of the results to Besov spaces are briefly discussed.Comment: Typos fixed. To appear in Potential Analysi

The Choice Architecture of Choice Architecture: Toward a Nonpaternalistic Nudge Policy

This The goal of nudge policy is generally presented as assisting people in finding their “true” preferences. Supporters argue that nudge policies meet a libertarian paternalism criterion. This claim has provoked complaints that nudge policies are unacceptably paternalistic. This paper suggests that by changing the explicit goal of nudge policy to a goal of making the choice of choice mechanism an explicit decision variable of the subgroup being affected by the nudge one can have a non-paternalistic nudge policy that better fits with the values inherent in Classical liberalism. The goal of non-paternalistic nudge policy is not to achieve a better result as seen by government or by behavioral economists. The goal of non-paternalistic nudge policy is to achieve a better result as seen by the agents being nudged as revealed through their choices of choice mechanisms. Examples are given of how nonpaternalistic nudge policy will and will not differ from paternalistic nudge policy.libertarian, paternalism, nudge policy, choice architecture, behavioral economics

Quantum Theory of a Resonant Photonic Crystal

We present a quantum model of two-level atoms localized in a 3D lattice, based on the Hopfield theory of exciton polaritons. In addition to a polaritonic gap at the exciton energy, a photonic bandgap opens up at the Brillouin zone boundary. Upon tuning the lattice period or angle of incidence to match the photonic gap with the exciton energy, one obtains a combined polaritonic and photonic gap as a generalization of Rabi splitting. For typical experimental parameters, the size of the combined gap is on the order of 25 cm^{-1}, up to 10^5 times the detuned gap size. The dispersion curve contains a branch supporting slow-light modes with vanishing exciton probability density.Comment: 4 pages, 3 figure

Explaining Import Variety and Quality: The Role of the Income Distribution

We examine a generalized version of Flam and Helpman’s (1987) model of vertical differentiation that maps cross-country differences in income distributions to variations in import variety and price distributions. The theoretical predictions are examined and confirmed using micro data on income from the Luxemburg Income Study for 30 countries over 20 years. The pairs of importers whose income distributions look more similar have more export partners in common and more similar import price distributions. Similarly, the importers whose income distributions look more like the world buy from more exporters and have import price distributions that look more like the world.

Limiting Government Predation Through Anonymous Banking: A Theory with Evidence from China

China's economic performance of the past two decades presents a puzzle for the economics of transition and development: Enormous private business incentives were unleashed that have fueled rapid economic growth despite the fact that China has had very weak "conventional institutions" (such as the rule of law and separation of powers) to constrain the government from arbitrary intrusion into economic activities. We argue that one mechanism that has limited the government's ability for predation and harassment is commitment through information decentralization, where the key institutiton is "anonymous banking," that is, a combination of the use of cash for transactions and the use of anonymous savings deposits. The government's incentive for such a mechanism comes form the increased quasi-fiscal revenues collected from the state banking system through "financial repression," a combination of controls on international capital flows with restrictions on domestic interest rates. The major features of China's economy concerning its fiscal decline, financial deepening, and the sectoral dual-track can be better understood using this analytical framework.http://deepblue.lib.umich.edu/bitstream/2027.42/39659/3/wp275.pd