Modeling, inference and clustering for equivalence classes of 3-D orientations

Investigating cubic crystalline structures of specimens is an important way to study properties of materials in text analysis. Crystals in metal specimens have internally homogeneous orientations relative to a pre-chosen reference coordinate system. Clusters of crystals in the metal with locally similar orientations constitute so-called grains. The nature of these grains (shape, size, etc.) affects physical properties (e.g., hardness, conductivity, etc.) of the material. Electron backscatter diffraction (EBSD) machines are often use to measure orientations of crystals in metal specimens. However, orientations reported by EBSD machines are in truth equivalence classes of crystallographically symmetric orientations. Motivated by the materials science applications, we formulate parametric probability models for unlabeled orientation data. This amounts to developing models on equivalence classes of 3-D rotations. A Bayesian method is developed for inferencing parameters in the models, which is generally superior to large-sample methods based on likelihood estimation. We also proposed an algorithms for clustering equivalence classes of 3-D orientations. As we continue to work on this area, we found and studied an interesting class of Markov chains with state spaces partitions of a finite set. These Markov chains have some properties that make them attractive in their own right, and they are potentially helpful in Bayesian model-based clustering

Some Bayesian and multivariate analysis methods in statistical machine learning and applications

In this dissertation, we consider some Bayesian and multivariate analysis methods in statistical machine learning as well as some applications of Bayesian methodology with differential equation models to study dynamics during co-infections by Leishmania major and Leishmania amazonensis based on longitudinal data. First, we developed a new MCMC algorithm to integrate the curvature information of a target distribution to sample the target distribution accurately and efficiently. We then introduced a Bayesian Hierarchical Topographic Clustering method (BHTC) motivated by the well-known self-organizing map (SOM) using stationary isotropic Gaussian processes and principal component approximations. We constructed a computationally tractable MCMC algorithm to sample posterior distributions of the covariance matrices, as well as the posterior distributions of remaining BHTC parameters. To summarize the posterior distributions of BHTC parameters in a coherent fashion for the purpose of data clustering, we adopted a posterior risk framework that accounts for both data partitioning and topographic preservation. We also proposed a classification method based on the weighted bootstrap and ensemble mechanism to deal with covariate shifts in classifications, the Active Set Selections based Classification (ASSC). This procedure is flexible to be combined with classification methods including support vector machine (SVM), classification trees, and Fisher\u27s discriminant classifier (LDA) etc. to improve their performances. We adopted Bayesian methodologies to study longitudinal data from co-infections by Leishmania major and Leishmania amazonensis. In the proposed Bayesian analysis, we modeled the immunobiological dynamics and data variations by Lotka-Volterra equations and the linear mixed model, respectively. Using the posterior distributions of differential equation parameters and the concept of asymptotic stable equilibrium of differential equations, we successfully quantified the immune efficiency

Statistical learning with phylogenetic network invariants

Phylogenetic networks provide a means of describing the evolutionary history of sets of species believed to have undergone hybridization or gene flow during their evolution. The mutation process for a set of such species can be modeled as a Markov process on a phylogenetic network. Previous work has shown that a site-pattern probability distributions from a Jukes-Cantor phylogenetic network model must satisfy certain algebraic invariants. As a corollary, aspects of the phylogenetic network are theoretically identifiable from site-pattern frequencies. In practice, because of the probabilistic nature of sequence evolution, the phylogenetic network invariants will rarely be satisfied, even for data generated under the model. Thus, using network invariants for inferring phylogenetic networks requires some means of interpreting the residuals, or deviations from zero, when observed site-pattern frequencies are substituted into the invariants. In this work, we propose a method of utilizing invariant residuals and support vector machines to infer 4-leaf level-one phylogenetic networks, from which larger networks can be reconstructed. Given data for a set of species, the support vector machine is first trained on model data to learn the patterns of residuals corresponding to different network structures to classify the network that produced the data. We demonstrate the performance of our method on simulated data from the specified model and primate data.Comment: 27 pages, 8 figure

Properties, Learning Algorithms, and Applications of Chain Graphs and Bayesian Hypergraphs

Probabilistic graphical models (PGMs) use graphs, either undirected, directed, or mixed, to represent possible dependencies among the variables of a multivariate probability distri- bution. PGMs, such as Bayesian networks and Markov networks, are now widely accepted as a powerful and mature framework for reasoning and decision making under uncertainty in knowledge-based systems. With the increase of their popularity, the range of graphical models being investigated and used has also expanded. Several types of graphs with dif- ferent conditional independence interpretations - also known as Markov properties - have been proposed and used in graphical models. The graphical structure of a Bayesian network has the form of a directed acyclic graph (DAG), which has the advantage of supporting an interpretation of the graph in terms of cause-effect relationships. However, a limitation is that only asymmetric relationships, such as cause and effect relationships, can be modeled between variables in a DAG. Chain graphs, which admit both directed and undirected edges, can be used to overcome this limitation. Today there exist three main different interpretations of chain graphs in the lit- erature. These are the Lauritzen-Wermuth-Frydenberg, the Andersson-Madigan-Perlman, and the multivariate regression interpretations. In this thesis, we study these interpreta- tions based on their separation criteria and the intuition behind their edges. Since structure learning is a critical component in constructing an intelligent system based on a chain graph model, we propose new feasible and efficient structure learning algorithms to learn chain graphs from data under the faithfulness assumption. The proliferation of different PGMs that allow factorizations of different kinds leads us to consider a more general graphical structure in this thesis, namely directed acyclic hypergraphs. Directed acyclic hypergraphs are the graphical structure of a new proba- bilistic graphical model that we call Bayesian hypergraphs. Since there are many more hypergraphs than DAGs, undirected graphs, chain graphs, and, indeed, other graph-based networks, Bayesian hypergraphs can model much finer factorizations and thus are more computationally efficient. Bayesian hypergraphs also allow a modeler to represent causal patterns of interaction such as Noisy-OR graphically (without additional annotations). We introduce global, local and pairwise Markov properties of Bayesian hypergraphs and prove under which conditions they are equivalent. We also extend the causal interpretation of LWF chain graphs to Bayesian hypergraphs and provide corresponding formulas and a graphical criterion for intervention. The framework of graphical models, which provides algorithms for discovering and analyzing structure in complex distributions to describe them succinctly and extract un- structured information, allows them to be constructed and utilized effectively. Two of the most important applications of graphical models are causal inference and information ex- traction. To address these abilities of graphical models, we conduct a causal analysis, comparing the performance behavior of highly-configurable systems across environmen- tal conditions (changing workload, hardware, and software versions), to explore when and how causal knowledge can be commonly exploited for performance analysis

Reconstruction of Networks with Direct and Indirect Genetic Effects

Genetic variance of a phenotypic trait can originate from direct genetic effects, or from indirect effects, i.e., through genetic effects on other traits, affecting the trait of interest. This distinction is often of great importance, for example, when trying to improve crop yield and simultaneously control plant height. As suggested by Sewall Wright, assessing contributions of direct and indirect effects requires knowledge of (1) the presence or absence of direct genetic effects on each trait, and (2) the functional relationships between the traits. Because experimental validation of such relationships is often unfeasible, it is increasingly common to reconstruct them using causal inference methods. However, most current methods require all genetic variance to be explained by a small number of quantitative trait loci (QTL) with fixed effects. Only a few authors have considered the “missing heritability” case, where contributions of many undetectable QTL are modeled with random effects. Usually, these are treated as nuisance terms that need to be eliminated by taking residuals from a multi-trait mixed model (MTM). But fitting such an MTM is challenging, and it is impossible to infer the presence of direct genetic effects. Here, we propose an alternative strategy, where genetic effects are formally included in the graph. This has important advantages: (1) genetic effects can be directly incorporated in causal inference, implemented via our PCgen algorithm, which can analyze many more traits; and (2) we can test the existence of direct genetic effects, and improve the orientation of edges between traits. Finally, we show that reconstruction is much more accurate if individual plant or plot data are used, instead of genotypic means. We have implemented the PCgen-algorithm in the R-package pcgen.</p

A Survey on Causal Discovery Methods for Temporal and Non-Temporal Data

Causal Discovery (CD) is the process of identifying the cause-effect relationships among the variables from data. Over the years, several methods have been developed primarily based on the statistical properties of data to uncover the underlying causal mechanism. In this study we introduce the common terminologies in causal discovery, and provide a comprehensive discussion of the approaches designed to identify the causal edges in different settings. We further discuss some of the benchmark datasets available for evaluating the performance of the causal discovery algorithms, available tools to perform causal discovery readily, and the common metrics used to evaluate these methods. Finally, we conclude by presenting the common challenges involved in CD and also, discuss the applications of CD in multiple areas of interest

Deep Causal Learning for Robotic Intelligence

This invited review discusses causal learning in the context of robotic intelligence. The paper introduced the psychological findings on causal learning in human cognition, then it introduced the traditional statistical solutions on causal discovery and causal inference. The paper reviewed recent deep causal learning algorithms with a focus on their architectures and the benefits of using deep nets and discussed the gap between deep causal learning and the needs of robotic intelligence

Measuring uncertainty in human visual segmentation

Segmenting visual stimuli into distinct groups of features and visual objects is central to visual function. Classical psychophysical methods have helped uncover many rules of human perceptual segmentation, and recent progress in machine learning has produced successful algorithms. Yet, the computational logic of human segmentation remains unclear, partially because we lack well-controlled paradigms to measure perceptual segmentation maps and compare models quantitatively. Here we propose a new, integrated approach: given an image, we measure multiple pixel-based same--different judgments and perform model--based reconstruction of the underlying segmentation map. The reconstruction is robust to several experimental manipulations and captures the variability of individual participants. We demonstrate the validity of the approach on human segmentation of natural images and composite textures. We show that image uncertainty affects measured human variability, and it influences how participants weigh different visual features. Because any putative segmentation algorithm can be inserted to perform the reconstruction, our paradigm affords quantitative tests of theories of perception as well as new benchmarks for segmentation algorithms.Comment: 27 pages, 9 figures, 4 appendix, 3 figures in appendi