Variational Inference in Nonconjugate Models

Mean-field variational methods are widely used for approximate posterior inference in many probabilistic models. In a typical application, mean-field methods approximately compute the posterior with a coordinate-ascent optimization algorithm. When the model is conditionally conjugate, the coordinate updates are easily derived and in closed form. However, many models of interest---like the correlated topic model and Bayesian logistic regression---are nonconjuate. In these models, mean-field methods cannot be directly applied and practitioners have had to develop variational algorithms on a case-by-case basis. In this paper, we develop two generic methods for nonconjugate models, Laplace variational inference and delta method variational inference. Our methods have several advantages: they allow for easily derived variational algorithms with a wide class of nonconjugate models; they extend and unify some of the existing algorithms that have been derived for specific models; and they work well on real-world datasets. We studied our methods on the correlated topic model, Bayesian logistic regression, and hierarchical Bayesian logistic regression

Nested Hierarchical Dirichlet Processes

We develop a nested hierarchical Dirichlet process (nHDP) for hierarchical topic modeling. The nHDP is a generalization of the nested Chinese restaurant process (nCRP) that allows each word to follow its own path to a topic node according to a document-specific distribution on a shared tree. This alleviates the rigid, single-path formulation of the nCRP, allowing a document to more easily express thematic borrowings as a random effect. We derive a stochastic variational inference algorithm for the model, in addition to a greedy subtree selection method for each document, which allows for efficient inference using massive collections of text documents. We demonstrate our algorithm on 1.8 million documents from The New York Times and 3.3 million documents from Wikipedia.Comment: To appear in IEEE Transactions on Pattern Analysis and Machine Intelligence, Special Issue on Bayesian Nonparametric

Stochastic Variational Inference

We develop stochastic variational inference, a scalable algorithm for approximating posterior distributions. We develop this technique for a large class of probabilistic models and we demonstrate it with two probabilistic topic models, latent Dirichlet allocation and the hierarchical Dirichlet process topic model. Using stochastic variational inference, we analyze several large collections of documents: 300K articles from Nature, 1.8M articles from The New York Times, and 3.8M articles from Wikipedia. Stochastic inference can easily handle data sets of this size and outperforms traditional variational inference, which can only handle a smaller subset. (We also show that the Bayesian nonparametric topic model outperforms its parametric counterpart.) Stochastic variational inference lets us apply complex Bayesian models to massive data sets

From zonal flow to convection rolls in Rayleigh-B\'enard convection with free-slip plates

Rayleigh-B\'enard (RB) convection with free-slip plates and horizontally periodic boundary conditions is investigated using direct numerical simulations. Two configurations are considered, one is two-dimension (2D) RB convection and the other one three-dimension (3D) RB convection with a rotating axis parallel to the plate. We explore the parameter range of Rayleigh numbers Ra from $10^7 to$10^9$and Prandtl numbers$Pr$from$1$to$100. We show that zonal flow, which was observed, for example, by Goluskin \emph{et al}. \emph{J. Fluid. Mech.} 759, 360-385 (2014) for \Gamma=2$, is only stable when$\Gamma$is smaller than a critical value, which depends on$Ra$and$Pr$. With increasing$\Gamma$, we find a second regime in which both zonal flow and different convection roll states can be statistically stable. For even larger$\Gamma$, in a third regime, only convection roll states are statistically stable and zonal flow is not sustained. For the 3D simulations, we fix$Ra=10^7$and$Pr=0.71$, and compare the flow for$\Gamma=8$and$\Gamma = 16$. We demonstrate that with increasing aspect ratio$\Gamma$, zonal flow, which was observed for small$\Gamma=2\pi by von Hardenberg \emph{et al}. \emph{Phys. Rev. Lett.} 15, 134501 (2015), completely disappears for \Gamma=16$. For such large$\Gamma$only convection roll states are statistically stable. In between, here for medium aspect ratio$\Gamma = 8$, the convection roll state and the zonal flow state are both statistically stable. What state is taken depends on the initial conditions, similarly as we found for the 2D case.Comment: 26 pages, 12 figure

Electrical control of metallic heavy-metal/ferromagnet interfacial states

Voltage control effects provide an energy-efficient means of tailoring material properties, especially in highly integrated nanoscale devices. However, only insulating and semiconducting systems can be controlled so far. In metallic systems, there is no electric field due to electron screening effects and thus no such control effect exists. Here we demonstrate that metallic systems can also be controlled electrically through ionic not electronic effects. In a Pt/Co structure, the control of the metallic Pt/Co interface can lead to unprecedented control effects on the magnetic properties of the entire structure. Consequently, the magnetization and perpendicular magnetic anisotropy of the Co layer can be independently manipulated to any desired state, the efficient spin toques can be enhanced about 3.5 times, and the switching current can be reduced about one order of magnitude. This ability to control a metallic system may be extended to control other physical phenomena.Comment: 20 pages, 7 figures, Accepted by Physical Review Applied (2017

Top quark rare three-body decays in the littlest Higgs model with T-parity

In the littlest Higgs model with T-parity (LHT), the mirror quarks have flavor structures and will contribute to the top quark flavor changing neutral current. In this work, we perform an extensive investigation of the top quark rare three-body decays $t\to cVV (V=\gamma,Z,g)$ and $t\to cf\bar{f} (f=b,\tau,\mu,e)$ at one-loop level. Our results show that the branching ratios of $t\to cgg$ and $t\to cb\bar{b}$ could reach $\mathcal {O}(10^{-3})$ in the favorite parameter space of the littlest Higgs model with T-parity, which implies that these decays may be detectable at the LHC or ILC, while for the other decays, their rates are too small to be observable at the present or future colliders.Comment: 12 pages, 10 figure

Computation of the maximal invariant set of discrete-time linear systems subject to a class of non-convex constraints

We consider the problem of computing the maximal invariant set of discrete-time linear systems subject to a class of non-convex constraints that admit quadratic relaxations. These non-convex constraints include semialgebraic sets and other smooth constraints with Lipschitz gradient. With these quadratic relaxations, a sufficient condition for set invariance is derived and it can be formulated as a set of linear matrix inequalities. Based on the sufficient condition, a new algorithm is presented with finite-time convergence to the actual maximal invariant set under mild assumptions. This algorithm can be also extended to switched linear systems and some special nonlinear systems. The performance of this algorithm is demonstrated on several numerical examples.Comment: Accepted in Automatic