Latent tree models

Latent tree models are graphical models defined on trees, in which only a subset of variables is observed. They were first discussed by Judea Pearl as tree-decomposable distributions to generalise star-decomposable distributions such as the latent class model. Latent tree models, or their submodels, are widely used in: phylogenetic analysis, network tomography, computer vision, causal modeling, and data clustering. They also contain other well-known classes of models like hidden Markov models, Brownian motion tree model, the Ising model on a tree, and many popular models used in phylogenetics. This article offers a concise introduction to the theory of latent tree models. We emphasise the role of tree metrics in the structural description of this model class, in designing learning algorithms, and in understanding fundamental limits of what and when can be learned

Tree cumulants and the geometry of binary tree models

In this paper we investigate undirected discrete graphical tree models when all the variables in the system are binary, where leaves represent the observable variables and where all the inner nodes are unobserved. A novel approach based on the theory of partially ordered sets allows us to obtain a convenient parametrization of this model class. The construction of the proposed coordinate system mirrors the combinatorial definition of cumulants. A simple product-like form of the resulting parametrization gives insight into identifiability issues associated with this model class. In particular, we provide necessary and sufficient conditions for such a model to be identified up to the switching of labels of the inner nodes. When these conditions hold, we give explicit formulas for the parameters of the model. Whenever the model fails to be identified, we use the new parametrization to describe the geometry of the unidentified parameter space. We illustrate these results using a simple example.Comment: Published in at http://dx.doi.org/10.3150/10-BEJ338 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

Equations defining probability tree models

Coloured probability tree models are statistical models coding conditional independence between events depicted in a tree graph. They are more general than the very important class of context-specific Bayesian networks. In this paper, we study the algebraic properties of their ideal of model invariants. The generators of this ideal can be easily read from the tree graph and have a straightforward interpretation in terms of the underlying model: they are differences of odds ratios coming from conditional probabilities. One of the key findings in this analysis is that the tree is a convenient tool for understanding the exact algebraic way in which the sum-to-1 conditions on the parameter space translate into the sum-to-one conditions on the joint probabilities of the statistical model. This enables us to identify necessary and sufficient graphical conditions for a staged tree model to be a toric variety intersected with a probability simplex.Comment: 22 pages, 4 figure

Robust Decision Trees Against Adversarial Examples

Although adversarial examples and model robustness have been extensively studied in the context of linear models and neural networks, research on this issue in tree-based models and how to make tree-based models robust against adversarial examples is still limited. In this paper, we show that tree based models are also vulnerable to adversarial examples and develop a novel algorithm to learn robust trees. At its core, our method aims to optimize the performance under the worst-case perturbation of input features, which leads to a max-min saddle point problem. Incorporating this saddle point objective into the decision tree building procedure is non-trivial due to the discrete nature of trees --- a naive approach to finding the best split according to this saddle point objective will take exponential time. To make our approach practical and scalable, we propose efficient tree building algorithms by approximating the inner minimizer in this saddle point problem, and present efficient implementations for classical information gain based trees as well as state-of-the-art tree boosting models such as XGBoost. Experimental results on real world datasets demonstrate that the proposed algorithms can substantially improve the robustness of tree-based models against adversarial examples

Autonomous models on a Cayley tree

The most general single species autonomous reaction-diffusion model on a Cayley tree with nearest-neighbor interactions is introduced. The stationary solutions of such models, as well as their dynamics, are discussed. To study dynamics of the system, directionally-symmetric Green function for evolution equation of average number density is obtained. In some limiting cases the Green function is studied. Some examples are worked out in more detail.Comment: 12 page

Ising models on locally tree-like graphs

We consider ferromagnetic Ising models on graphs that converge locally to trees. Examples include random regular graphs with bounded degree and uniformly random graphs with bounded average degree. We prove that the "cavity" prediction for the limiting free energy per spin is correct for any positive temperature and external field. Further, local marginals can be approximated by iterating a set of mean field (cavity) equations. Both results are achieved by proving the local convergence of the Boltzmann distribution on the original graph to the Boltzmann distribution on the appropriate infinite random tree.Comment: Published in at http://dx.doi.org/10.1214/09-AAP627 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org